I think there's something to be said for learning programming on a very simple machine.
First, you need to keep in mind that kids have absolutely no idea how a computer works, at even the most basic of levels. It's a box with a keyboard and stuff happens on the screen. You need to cement the idea that you have to tell the computer to do stuff, and link the idea of coding to that stuff.
This is much easier to do when you have a computer that does pretty much nothing when you turn it on. A flashing cursor comes up and it's waiting for you to tell it what to do. 10 PRINT "HELLO" RUN and it does something. 20 GOTO 10 RUN and it does something else. You get the link between what you're telling the computer to do and what the computer is actually doing pretty quickly.
Compare that with booting up Windows/MacOSX/Linux, getting into your desktop environment, loading up a browser or IDE, creating a new project, explaining the UI of the IDE, making sure you have the right includes to do IO, directing your output to console or a UI object, etc...