We have 3 kids in car seats, and an Odyssey.
When we lived in town, it was great. Back then, my only serious gripe with the Odyssey is that if you are running a second set of wheels (e.g. for permanently mounted snow tires), and don't fit a 2nd set of expensive TPMS sensors to those wheels, the VSA (stability control) cannot be defeated via the console switch.
This is a problem because the VSA implementation sucks and is frankly unsafe when accelerating on surface transitions - for instance, when you are waiting on a gravel road and are about to pull onto a paved highway, the VSA system senses differing levels of wheel grip between the wheel on pavement and the wheel still on gravel, and cuts power, precisely when you need maximum power to quickly get to highway speed.
Last fall we moved to a rural area, and now poorly maintained roads (deep snow in the winters until I clear it, deep ruts whenever there are rains) has really shown me the shortcomings of the vehicle. My wife has gotten it stuck 4 times in our first winter.
The Odyssey needs 2 things to be superlative. Air suspension with adjustable ride height (it is a very low vehicle, for ease of entry/exit for small kids), and a proper AWD system.
My wife is now desperately wanting an AWD vehicle. But to get a proper AWD system (e.g. locking transfer case or at least a torsen differential), and the useful seating capacity of a minivan, you need to be looking at full-size truck based SUVs, like the Excursion or Sequoia.
I'm aware that the Sienna comes in an AWD version, but its particular AWD system and ride height doesn't inspire me that they will be foolproof enough to want to make the switch.
Sadly, my wife also refuses to drive a Mercedes G-wagen :)
As an aside, the Odyssey towing capacity isn't really sufficient. It's 3500lbs, and it requires upfitting the vehicle considerably with things that don't come factory - PS cooler, ATF cooler, hitch wiring, etc. (In addition to the actual hitch receiver).
When we were considering camping options, essentially nothing that had enough floor space for a family of 5 could be towed behind an Odyssey.