What I liked most about his show were his interviews. Yes, as a satire show he often showed absurdities in the political process, but the segment that made it special for me was his interviews.
While he did bring on celebrities from the media (film starts, bands), he brought on a lot of serious/reputed guests and gave them a chance to speak. I can honestly say I actually learned something from his show. While Colbert had interviews too, it seemed to me Colbert always tried hard to work jokes in to the interview instead of having a more serious conversation.
John Oliver also has a issue of the week in his show - serious issues peppered with light humor, but still informative. If he got guests on his show, it would be a great alternative to Jon Stewart's show.