The biggest issue is pay. You'll put up with a lot if you love what you are working on and can make a career out of it. But making a career out of being a university professor is becoming impossible, and that is the ultimate goal for many (if not most) folks getting a Ph.D.
Case in point: My wife has a Ph.D. and was an assistant professor for several years. Just getting a tenure-track position required working as a temporary adjunct for almost a decade and only being paid a base rate per class. That base rate varies by school, but is always absurdly low. She would typically be paid $1800 - $2500 per class per term (a 5 month semester). So, if she was teaching a full load, of 5-6 classes, the most she would make is around $45k/year (2 full terms + a summer term). But then there's the complication that no single school would give her more than 3 classes per term, so she'd have to look for jobs from multiple schools simultaneously. Also, there are no automatic renewals, so every term you have to basically find as many new classes as you can and just hope you get the contracts. Some terms were great and she'd have a full load, but often she might only get 3-4 classes total. We knew MANY adjuncts living in their cars.
But if you score a permanent position in a department, things must get better, right? Well, that's easier said than done. These jobs are incredibly competitive, and there aren't a lot of openings. My wife is very well known in her field, she's written dozens of papers and published her first 3 books before she even tried to get a permanent position. Even then, with around 45 applications over 3 years, she only made the shortlist for 4 jobs (each with 5-10 other candidates). After 3+ years of searching, she managed to find a position, but the issues didn't end there. The assistant professor position only paid $55k. She'd make more if she got tenure, but getting tenure is a 10-15 year process, and it's much harder to get as universities prefer to just let professors leave than try to retain them (they can just throw more temporary adjuncts at classes for next to nothing). She was at a large state school, but even they had a handful of departments with only 3-4 tenured professors each (and generally 1-2 dozen assistant profs. or adjuncts at any one time).
Long story short, after struggling within the university system for over 15 years, my wife now teaches high school. The pay for a high school science teacher started at $80k, about 50% more than she made as an assistant professor at a large state university.