Just wanted to say hi to a fellow take-in-new-information-and-change-opinions person. There need to be more of us, or if there are more of us already, we need to be more vocal, so the world knows both that it's OK to be reasonable (lots of people are doing it!) and more especially that reason will can actually get you somewhere with people.
My personal history: I started out, when I was very young, assuming that someone was obviously in charge of the whole world and all that was needed to make things better was to get whoever was in charge to do it, or put someone new in charge who would do it; to make things more fair, thoughts like "if society is going to be designed such that you need a car to function in it, then cars should be given to everyone when they're old enough to need them, otherwise it's just not fair". I was basically (and called myself) a straight up communist.
In my teens my perspective shifted as I realized that there really isn't and actually can't be anybody in charge setting up "the system" as a whole, there are just a bunch if interacting individuals and any system that there might be emerges organically from their interactions, so the best we can do is just keep people from trampling over each other, protect individuals' rights, and leave them to their self-determination. I considered myself a libertarian then.
Then as I became an adult and had to actually get by in the real world on my own, I realized like you that even given ideal libertarian freedom, success and failure are frequently, I'd say even predominantly, highly uncorrelated with hard work and skill. There is random chance to factor in, like you point out, and I've been hit with more than my fair share of bad luck to prove that point to me, but there are also systematic factors allowed by traditional American right-libertarianism that perpetuate inequalities, giving a hand up to those who need it least, and holding down those in greatest need.
In the years since then I've looked for other alternatives, finding some sympathy for left-libertarianism a.k.a. libertarian socialism, which is a term I now apply to myself in lieu of any other, although I am still strongly propertarian in contrast to them. I am also very sympathetic to distributivism and it's motto that "the problem with capitalism is not that there are too many capitalists, but that there are too few", i.e. purely free markets would work great if there were a society where everyone owned e.g. their own homes and businesses, and not a class division between the non-working owners and the non-owning workers. I mostly focus on contracts of rent (including the special case of rent on money, interest), and possibly contracts in general (besides transfers of ownership), as the root of the problems with traditional American right-libertarianism, and have extensive original arguments on how they perpetuate that owner/worker class division that would otherwise naturally dissolve in a truly free market, and thus how broadly libertarian ideals could be realized while still achieving socialist (egalitarian) ends, if only that feature (rent and interest) was omitted.
But I've also had to learn to separate idealism from pragmatism. I don't have any party I can get behind when it comes to representing my ideals, because they require refactoring large social structures in ways that most Americans simple cannot conceive ("libertarian socialist" being a blatant contradiction in their minds, never mind things like "stateless governance"). But that's all long-term, and the only work that can really be done there is to spread the ideas. In the short term, practical considerations outweigh ideals, especially since one way or another the ideals simply are not going to be realized in my lifetime. So in the short term, I've backed away from my libertarianism and accepted some more mainstream state-socialist concepts as the best thing that we can do right now with the political climate what it is, though that "best" is still informed by libertarian ideals. Things like: given that taxation is bad and ideally should not happen at all, but that it is going to happen anyway, if it happens, it should be done in the least-harmful way; namely, by taking from those who can best bear that burden, the rich, and spending on those who most benefit from it, the poor. In other words, taxes are bad but necessary, therefore progressive taxation is the best option.
So for the most part I end up disappointedly voting for Democrats, or Greens when they're available, and wishing there were better options. I really really wish the Libertarian party would just destroy the Republicans already and the mainstream debate could be between Libertarians and Democrats, because then maybe we'd see some movement in the right direction. (Basically everything those two parties agree on, I agree with; while basically everything the Democrats and Republicans agree on, I disagree with).