Seems to me that any language can allow for poorly written applications to run...that's like saying that you shouldn't teach a beginner about photography using a point & shoot camera because it allows you to take crappy pictures.
I used LOGO and VB to teach my nephew (he was a about 7 years old at the time) about programming. He really enjoyed it.
Things that are good about it: its visual, its dead simple, the syntax is easy, the language is verbose (so its a little easier to explain and remember for a kid), there are relatively few nuances (like ending lines with ; or case sensitivity).
Could you do it with Java, C#, Javascript, or Pascal? Sure, but that doesn't make VB bad for a beginner.
Now, if by beginner, you mean someone who is intending to becoming a programmer and earning a living doing so, maybe VB wouldn't be my first choice. But if I had to teach a bunch of 10 year olds how to program, I would rather use VB than C# or Java (although LOGO is pretty fun).