(I work at Google, but not on search)
I'm afraid the idea, often expressed in this discussion, of "that's what most people want" sells us short. The whole point of a smart search engine is to give me what I want. What I want is not what most want.
When Google tries to do this, the same people start complaining about filter bubbles and either turn off personalization in their search settings, or turn to DDG, where a primary selling point is that they don't personalize. You really can't have it both ways, although Google comes very close with a simple toggle button for personalized results.
So as a monopoly it has started to ignore its users. It has even wound back features that were previously useful. Most of us could quickly list 10 things it could do to improve its service.
I don't believe you appreciate the difficulty of search given the current state of advanced [black hat] SEO; things that worked in the past (such as plain pagerank) would not work at all today. All search engines must run to keep in place. Also, economics plays a role -- can those 10 things be implemented in a practical way that scales and is cost effective.
I can 10 things on my car that I'd like, such as better fuel economy, more horsepower, better crash safety, better visibility, more convenience features, and a lower price. Unfortunately many of those things conflict, so in a practical sense it is likely that the car company had to strike a balance. From my armchair I am unlikely to know all of the things that went into those trade-offs.
 "filter bubbles" don't really apply to multi-answer ranking problems or are trivially broken with standard techniques from reinforcement learning to manage the "explore-vs-exploit" tradeoff. As far as I've been able to determine, the person who coined the bubble term has no formal background in statistics (in particular ranking problems) or machine learning (in particular reinforcement learning).
 An oberservation from a long-time logged-in user: In my search results, personalization hardly ever effects more than two results out of the first 10. In search at least, filter bubbles do not exist for me, and I've taken no steps to avoid them. They do happen when I listen to a music service for a while (where unlike search, only one song can be chosen as the next to play).