Get rid of the sports that cannot measure the success of the competitors using the Olympic motto: higher, faster, stronger. That means no figure skating, no synchronized swimming, and, especially, no more rhythmic gymnastics. Essentially, nothing that requires assigning a number to a performance via a panel of judges. (I'm a little torn about any sport that chooses winners based on the points that they score on a particular day but when I think about the excessive coverage given to beach volleyball in the last few Summer Games I lean hard to the "drop them, too" side.)
Just think how much less expensive it would be to hold an Olympics would be if all those judged "sports" were taken out. The potential sites for the games would mushroom without a need for all the additional venues for the judged events. Cities that hold the Games can rarely afford to and the citizens wind up footing the bill for facilities that will rarely see use after the closing ceremonies. Plus, if it would get Bob Costas' interviews with prepubescent gymnasts off the air, we all win.