The best thing about New Delhi is that it's an hour or two in a coach bus to Agra and the Taj Mahal
I think the best thing about Delhi is Hauz Khas, which is, of course, a beautiful historical building, within Delhi. Or a few old ones together. Quiet, somewhat run down and a peaceful spot in a city of awesome chaos.
The flip side is the horrendous air pollution, nonstop coughing for nine to ten days (the lengths of the trips I had there), leading to some of the worst nosebleeds I've had in my life.