You're kinda illustrating what I meant by my parenthetical, but I guess I didn't explain it very clearly - Libertarians often focus on keeping the government from restricting your rights, rather than focusing on ensuring that the government protects your rights.
For example, their platform reads "Members of private organizations retain their rights to set whatever standards of association they deem appropriate," which means that businesses should be free to openly discriminate against employees of a certain race, sexuality, gender, etc, in order to protect the business owner's freedom of association. Liberals generally believe that the government should be in the business of ensuring that people get a fair shot, in order to protect the employees' ability to earn a wage.
(Also, they want to entirely privatize schools, because nothing says "land of opportunity" quite like being too poor to afford first grade.)
Saying basically that "It's fine, as long as it's not government doing it" doesn't really seem "socially quite liberal" to me - it seems, frankly, like an anarchic appeal to the state of nature, in which only the strongest is secure in their rights.