The days of the mom&pop run farms is over. A large percentage of the North American food supply is controlled by corporations. You can't lay that at the feet of GMOs or claim that if GMOs went away then so would the corporations.
As far as reduced biodiversity, agricultural sustainability and overuse of pesticides go, these risks are all still there if GMOs go away AND some GMO 'products' have no risks related to any of these. Likewise, many invasive species are more harmful than GMO's; try zebra muscles, purple loosetrife, sparrows, domestic cats....
With that said, I do think corporations, and independent mad scientists, should be accountable for what they release on the world. Anything created to increase the use of a pesticide is not in the Earth's best interest. Anything created that can out-compete and eliminate natural species should be highly regulated with the potential for fines large enough to reverse damage done.