I let my kids play with TuxPaint, a free paint app that is geared for kids. It doesn't require a keyboard so I put that out of reach. I also have a gamming optical mouse that let's you change the speed of the mouse with a small button on the mouse. I turn the mouse speed way down before I hand it over to them. Might need to wait until he/she's a little older.
"Who cares if it doesn't do anything? It was made with our new Triple-Iso-Bifurcated-Krypton-Gate-MOS process ..."