+1 to this. I know from personal experience that this is the way to go, especially for younger kids. Not only does it have a solid track for growth, from elementary until high school (FLL -> FTC -> FRC) , but it makes sure that you have other people to work with. Plus, there's generally no or very little cost to the student.
If that's not an option, I'd still recommend Mindstorms. It's more expensive, sure, but it really is leaps and bounds better than the alternatives. Younger kids (late elementary through middle school, 10 - 14 or so), tend to struggle with some of the less-developed kits, particularly if they lack a large community. Mindstorms is a great development kit, as you can see from all the /. articles about it. Adults and kids can make great use if them - I do all the time.
If that's still out of budget, VEX may be somewhat less expensive. I believe kits run about $200 and there's still a large community and yearly competitors and challenges to participate in. It's not quite the same community as FIRST, though.
Basically, there's no cheap way to get a (good) robotics kit. Even homebrew stuff (Arduino and the like), is going to be $100 at the absolute minimum. The cheapest way is to find a local team, or perhaps try starting one - many schools districts offer funding, support, or even full kits for new teams, in addition to lots of FIRST scholarships.
Disclaimer: I mentor FLL (Mindstorms) and FRC teams, after having been on several myself through middle and high school.