I think we need to be more precise about the terms used when describing "manager". To a large extent, financial managers or portfolio managers don't manage people, they manage the finance or portfolio. Somehow, in the IT industry, we've developed a different terminology around the term "manager", where IT managers are people who manage IT "workers" rather than managing IT itself.
If we go all the way up to CIOs and CFOs, we see a similarity in usage. A non-financial CFO would be kind of a joke, a non-IT CIO would be the same. However, these "C" suite officers don't necessarily then have the entirety of IT or finance reporting into them. Sometimes they only manage small teams to provide "guidance" and "leadership" and the bulk of the workers report up to a COO or CEO.
I think across all industries, you do have conflicting notions as to whether people managers should be more skilled in the task that their people are doing, or more skilled in managing people and organizations. These are different types of specialty skills and while it would be great to want all managers to have it all, there are availability constraints that would make it difficult to source omniscient intellect for every position.