There is a lot of pseudo-science BS and greenwashing in the organic community. But I also think your impression of it is a common misconception. Organic practices are constantly progressing through scientific research, or best practices being rediscovered to enhance yield. It isn't all anti-science back to nature romanticism.
Yields don't have to be much lower. Growing organic is more expensive because it requires more labor hours, which is the big reason industrial farms were so easily honeymooned by Monsanto and other agri-businesses to use the latest pesticides with minimally studied safety profiles, fertilizers, and genetically shotgun spliced crops that cost a fortune to the farmer and lock them in to business with those companies. It also allowed farmers to forget the hard won knowledge over generations of how to maintain their soil and do proper crop rotation to prevent degradation of their land.
My wife grew up in rural Illinois. She remembers having to come in when the spray planes would fly overhead, and the stench that lingered in the air afterward. She got a non-cancerous brain tumor that will probably reoccur over her life. She has had lots of friends from the area live with and die from various cancers before they've turned 40. There is no hard scientific evidence this spraying was the cause. But those chemicals are generally studied in isolation, in labs for several years, not together in the environment with humans over decades. She has some concern that we may learn many decades later there is a link.
And as one other poster alluded to, if you are concerned about calorie yield per acre, you'd do much better by not (or rarely) eating beef as it is very land intensive to grow the feed for cows and they have a high global warming contribution through their methane emissions.