When my son was 2.5, I brought home a throwaway tablet from work. I set it up to launch web sites in "kiosk mode" like pbskids.org and our web photo album. I also disabled right-clicks and some other confusing UI gadgetry. He loves his moments of computer time (a couple sessions per week, max). I always sit with him, and he takes very good care of the machine. pbskids.org has games for all ages. Don't listen to the gut responses here. Anything interactive beats parking your kid in front of Baby Einstein or Disney.
My mistake was thinking that a tablet pen would be easier than a mouse! That pen requires steady hand that can be frustrating, even for me! When he got his hands on a mouse and tuxpaint, it took him about 2 minutes to master.
The future has great promise with touchscreens. I looked at the HP this year, but it looks like touch support beyond the built-in HP apps might be poor. I'm holding out for an Apple touch tablet!
"Life begins when you can spend your spare time programming instead of watching television." -- Cal Keegan