Well, there have been some great responses from folks who've been unschooled, and I don't think I can hope to match those, so I'll settle for making some comments from someone currently in "Year 11" of unschooling my daughter.
We started homeschooling purely due to concerns with the "roundedness" of current public school education. So no religious or "those people" concerns here.
I was initially skeptical of the idea, with one of my comments in arguing against it was that "I survived public school", which I realized later on was a pretty telling comment. Once I agreed to go along with it, at least initially, I've been pretty impressed with the workings of it.
In unschooling, the role of the parent is something of a guide. We're here to point out potential pitfalls (like: "unplug that toaster before you open it. In fact, let's have a discussion about the role of electricity & the human body!") and provide resources (like: "Here's Wikipedia, have fun!" (kidding!)). Learning is up to the child. JUST like it is in public/private/any school environment.
If I could pick out a primary tenet of "unschooling" it's that "School doesn't just happen in between the rings of a bell". My daughter responds to kids she hangs out with who say, with awe "You mean you never go to school?" with a sometimes wistful "No, it means I'm always in school." Really. When in your life after you leave formalized education do you study a subject, a bell rings and you change subjects? Sure it may happen from time to time (phone rings, time for an appointment) but not every hour of every day. This is what unschooling is against.
You do end up with some odd results. My daughter didn't start reading until what I consider rather late (8-ish). OTOH, now that she IS reading, she reads at a very high rate with high comprehension & retention. Weirdly, she also INSISTS that we taught her to read, when we did no such thing. (Unless you count us reading to her or helping her learn the spelling and structures of words).
Socialization? C'mon be serious. She's been in about 10 plays, does science workshops, is getting ready to do a project where kids take on roles in the running of a small "city". She's done philosophy clubs, baseball and rocketry, nature studies, fencing and archery, goes to YMCA summer camps - she's dealt with many of the "social" issues folks have brought up & in a much more controlled environment than "and the teacher mentioned this has been going on for weeks."
Downsides? Nerve-making for the parents (at least for me). There's no one else to "blame" if anything goes south, although so far, so good. It's a little hectic sometimes, but when I really sit back and think about it - I'd have killed to have had this kind of opportunity when I was a kid. Which is definitely something that makes me think it's worth it.
Anyways, like I said, just some thoughts - I'm just one parent who unschools, and figured I'd throw in my $.02.