You couldn't care less...
Maybe you should know that the big banks who do HFT also co-locate inside the exchanges and front run orders making hundreds of billions per year.
Also, you might want to know that if the market crashes and restarts like today the big banks can get their losing trades reversed and you can't.
All the profit they're making has to come from somewhere. Are you so certain it doesn't come out of your pocket?
HFT exists to take advantage of arbitrage situations. They are reactive to changes in the market and colocating inside exchanges helps them to react quicker. It is certainly a profitable venture, as you can see.
Big Banks do not get any sort of priority over anyone else when having trades reversed. For starters, you are certainly not trading on the exchange for yourself, are you? You have a brokerage account, right? In that case, broker XYZ made the trade in their name. Your ownership of that stock exists as a notation on their books. That's it. Fidelity trades with Ameritrade. Bob does not trade with Joe. If some crazy event happens and trades are going to be reversed, Fidelity will ask for their affected trades to be reversed. All of them. Including the ones they made for you. But now, if you were trading for yourself, you can also ask for the trades to be reversed. I'm not going to paste a link here, but you can find it with less than 3 minutes of googling. Each exchange has a form on their website. Fill it in and email it to them. They can't refuse you just because you're a nobody. Really, they can't.
Now, as far as who's pocket HFT profits come from - that's a good question. First of all, there is no way to take any money from your pocket unless you are a party to the transaction. I had lots of money "in the market" as part of my retirement planning, but I didn't have any buy or sell orders. So I was completely unaffected by this flash crash. Right? My retirement was in jeopardy for 4 minutes. I didn't notice. I was working. Then the market recovered and my retirement plans were back on! Yay! Alright, so before HFT, buy/sell spreads were large. Now they are small. That's actually a very large benefit to me when I buy or sell stock. So no, I am not so sure that HFT is pulling money out of my pocket. Perhaps they are pulling money out of the pocket of the people that benefited when the buy/sell spread was large. Brokerages? I am not sure..