Back in high school, I thought I was the most badass programmer on this planet. Could hand assemble code and hyper-optimize my programs counting clock cycles everywhere. I got into college. There was a book on algorithms in the small library the computer room had. I learned about asymptotic behavior. So I realized that my cleverly optimized O(n^2) sorting assembly program was going to eventually lose badly to a straightforward interpreted implementation of quicksort. Clearly I had to become good at this. I started reading all books I could find on the topic and very quickly I noticed that they all included a sentence meaning "If you want to understand the subtle underpinnings of this, see TAOCP." So I got a copy of volumes 1 & 3. It was expensive (I was not in USA and books were VERY expensive because of currency exchange issues). Mom & Dad chipped in and I still thank them for that. I borrowed volume 2 from a friend.
I decided to go though the book solving all exercises up to level 30. Saturday was my TAOCP day. I cannot say I read ALL of the text nor that I solved ALL problems of difficulty below 30, but I solved all that I tried and they were way more than a few. I even solve some in the 35 level, whenever I considered them interesting.
FF to the beginning my PhD studies at a good school in the USA: When taking the grad level Algorithms class, I was doing VERY well. The professor, a scary smart guy (now full professor at Stanford) asked me how may courses on algorithms and information theory I had taken. The answer was none. My undergrad degree was in Systems Engineering, from a crappy school in South America. I was never formally taught what a big-Oh was. That professor become my thesis advisor. He used to joke saying I was a walking encyclopedia of algorithms.
Will it work for everybody? Probably not. If you want to do theoretical CS, as I did, you will have to handle that math and some more anyways. CLR is more up to date and time efficient for learning. If you want to be a software engineer, chances are you are not going to be doing the heavy math, you will be using off the shelf algorithms.
Still TAOCP is a hell of a reference. Funny thing is, after finishing my PhD, I browsed through it. It looked simple, not very formal at all. I guess PhD does mean "permanently head damaged" :-)