The biggest technical problem is getting to Low Earth Orbit. Right now it is too expensive, though people are working on solving that.
The biggest overall problem as I see it is the socio-cultural approach we are taking with regards to space.
Back in the 1930's space flight was science fiction, though a few people like Goddard, Tsiolkovsky, and Oberth had been working on the technical aspects for a few years. People dreamed what it would be like to go into space and came up with some ideas on how to do it and what they might face. But it was just a dream that needed technology to make it real.
In the 1960's science fiction became science fact as the Space Race came into play. National pride and national security demanded that the money be spent on space. It got us places, but not cheaply. We learned a lot about what to do and what not to do. The dream was alive.
Since then we've been coasting, making little steps and sometimes stumbling. Space flight has become a 'so what' type of thing for a lot of people for a variety of reasons. It was also a fairly restricted 'club' limited to professionals and a small number of multi-millionaires. But the dream continued to live on and a few hundred ambitious people made it into space. They had to work hard to get those seats though.
In the not too distant future, as a result of private enterprise getting interested in space, 'normal' people will be making baby steps into space. These baby steps are much like the 'barnstorming' flights where pilots sold airplane rides. You went up, you came down, and you talked about it to all your friends. That creates an greater cultural awareness of what could be done with airplanes and it will do the same with suborbital space ships. And if people like Rutan and Branson can make a profit at it, an greater economic awareness will develop.
When the ball gets rolling in the suborbital area, people will start looking at orbital flights with a greater degree of seriousness. Once you reach orbit, you're halfway to everywhere.