This may be YOUR objection, but it is by no means most peoples first justification. I've participated in many a discussion (in person AND online) regarding this topic, and the nature of the modifications rarely comes up in the first half dozen objections. When it does come up, my response is generally "So What?"
1. Improvements in crop production efficiency lead to cost reductions, the majority of which trickle down to end consumers. This is a large part of the reason that the rate of food cost inflation over the last 20 years has been much lower than for the rest of the US economy. It's also why the environmental foot print per unit of crop yield is an order of magnitude less than it was 20 years ago. Consumers are absolutely benefiting directly, just not in a flashy way.
2. Food products engineered to be more directly beneficial to consumers HAVE been developed, but have for the most part tanked in the market up until now (nutridense corn, low phytic acid crops, etc. come to mind). There is a new round of consumer focused GM crops hitting the market, and I'm hoping they fare better, but I won't be surprised if they tank as well. AquaAdvantage salmon have the potential to dramatically reduce the cost of salmon, reduce the environmental impact of farm raised salmon, and spare wild salmon stock from further over fishing. Arctic Apples (non-browning) stand to help consumers by increasing the at-home shelf life of apples, and thus reducing their waste. However, both of these are brand new on the market and it remains to be seen..
3. Golden rice was also developed years ago, is consumer focused, and literally free for personal use, and yet it has been opposed at every turn by the anti-GMO crowd.