2 is young for having a computer. Even if your child is exceptionally bright, there's a lack of motor coordination that makes using the mouse impossible. So, don't be worried if your child doesn't take to the computer immediately. My kids didn't get interested in the computer until they were 3, and not really proficient until 4.
That being said, my advice is to buy a desktop with a cheap keyboard. Laptops break too easily, regardless of the brand. The biggest threat to the computer is from spilled food. Laptops are portable, thus easier to drag over to the dinner table. Even with pretty strict rules about eating and playing games, we've still lost a dozen or so keyboards to spilled milk, mac&cheese, etc.
Check online for games. Sometimes it's nice to buy an installable game, but there are really excellent games for pretty much every kids show. Look for the production company that makes your kids favorite shows (Noggin, Playhouse Disney, etc). They usually have links to a bunch of clever flash games. If that doesn't work, search for the show itself. Pretty much every show has good games. Eventually, the kids will learn to look for games under other shows, so their supply of games can be even larger than the number of shows they watch. A bonus is that these same games will work on Linux and Macintosh.
Check out webkinz. You buy a doll which gives you access to an online game site. It's a very clever site, much more involved than the games for the TV shows. 2 is waaayyy too young for it, though. You don't need to be able to read, but you need to be able to puzzle out the menus and remember which one gets you where. But, it's worth remembering for when your child is old enough.
Finally, be prepared for your kids to *hate* your games. They're just not interesting for kids. Or too violent. It'll be years before you both want to play the same games.