Our local school had several used computers donated to it (TRS-80's and the like). That summer they decided to have computer courses open to anyone who wanted to learn. I was going into kindergarden at the time and my parents signed me up for the course. I learned how to boot DOS, run a few commands and create a program in BASIC. I really enjoyed making the rocket scroll across the screen and I remember how proud I was when I changed the rocket to add my name onto it. Probably why I'm still programming these 30 years later.
That being said, I'm not sure there is much benefit in teaching children to code. Teach them how to use a computer well. If they are curious, teach them to make it crash constantly (aKa programming).