You are wrong. Your premise is correct, but your conclusion is wrong. You're advocating what's effectively a position of anarchy, and if you know history, you'll know that anarchy is not a course that will lead to progress.
See, there's something called economies of scale. A new product is expensive because it costs a lot to manufacture. It costs a lot to manufacture because the company manufacturing it does not know exactly how well it will sell, and hence will only allocate a certain amount of resources into making it. I.e., it will not make very many units.
This is true of all industries that deal with material goods.
Once a company has sales data, it can then begin to ramp up its production. Prices will then fall. How well a company does this, how quickly it responds to the rise or fall of demand, is a significant competitive advantage. This is why a lot of companies are focusing on point-of-sale tracking, and why you're seeing all these advertisements from IBM about "analytics".
A large company with much more resources (material property and capital) can ramp up much more quickly than a small business. This is a fact. Companies with lots of resources that cannot do this die very quickly and sometimes violently. It's a matter of survival of the fittest, and the fitness, like natural processes, is determined by how quickly and efficiently the organism can adapt.
Without patents, your small inventor is going to be ultimately completely screwed. While the new product is still in a niche market, the small inventor's small company will thrive. At this time, large companies will take notice. They will begin to copy the product. But they probably won't sell the copied product immediately. Instead, they'll sit on it and wait until the market leaves its niche status. If this never happens, the copy is shelved and the large company focuses on a different product. But once the product has sufficient market, the large company will jump in. It will out-manufacture the small company (who by now, isn't very small, but is still tiny when compared to the large company), and be able to sell at much lower prices.
Now, if the small company can ramp up production quickly enough, and with enough marketing dollars, the small company can regain much of this lost market within a short amount of time. However, the large company has a lot of revenue streams. And it can still thrive even if one product is losing money. The small company, with far fewer revenue streams, cannot afford to take a loss on their best-selling product. So the large company will go into a price war with the small company. And the large company will win.
But say that multiple large companies jump in at the same time, and that they will all be fighting to gain control of the market of this product. That still doesn't help the small company, because all that will do is drive the price down to its bare minimum, and again, based on economies of scale, the small company will not be able to sell as low as the large company.
The only way out for the small company is to out-innovate the large companies. The inventor who owns the small company needs to be one step ahead of the mass of resources the large company can bring to the table. Sometimes, this is possible. Most of the time, it is not.
Now, this is purely an economic viewpoint of what would happen if patents are removed. From a social viewpoint, it will discourage innovation. Why? Because people will realize that if they want to bring to market a new invention, they're facing a significant uphill battle, one that will not end until they become a large company themselves, or their company dies. So what they will do is work for an existing large company instead. It is a much easier way out, and people with starving kids waiting at the dining table for them to come home with the bacon will take that way.
So all of the potential inventors and innovators will go work for large companies. Large companies are risk adverse. Car companies spent millions to bury new inventions that threaten their existence. This includes engine technology, and battery technology. Back at the turn of the 20th century, the tobacco industry spent a ton of money lobbying congress to make marijuana illegal. Large companies are not interested in innovation, only in keeping their existing revenue streams alive. And the most efficient way to do that is to put money into having to do nothing. They can spend $100K a year lobbying congress to keep their business alive, or $100M to expand into something new, while incurring significant risk that the new product fails. (Kodak is a lesson every CEO in the world is learning from--how a company managed to innovate itself out of business.) Which avenue do you think a large company will take?
Progress will crawl without patents. It will all but disappear. It may not completely disappear, because there will always be a few who will get through the cracks. But it will be slowed significantly. And if it's not enough to overcome our natural state of continuous regression, then society itself will regress.
The system needs patents. Or it needs significant, stringent corporate regulations which effectively duplicate the purpose of patents, but would add a level of complexity to the law that yet again, enables the haves and disables the have-nots.
The solution is not to do away with patents completely. It is not to throw the small inventors to the wolves and see who survies.
A better solution is to take a page from trademark law, and separate patents by industry. Industry categories will have shorter or longer patent times based on how quickly a product can be manufactured. And they will have separate grantability requirements that will be based on the obvious standards of that particular industry. Couple that with explicitly limiting patents to things that are manufactured, and whose manufacturing chain ends in an artificial (man-made) machine, and that will bring the patent system back in line with its intention, to promote the useful arts and sciences.