As sibling says, bridges and hills are a problem. Major waterways are generally constructed so that bridges are either really high (as in 30ft+) or have some part that can be opened.
Anything that fits on a truck is easy to transport over land, but stuff that is significantly larger and can't be moved in parts is difficult over land. On the water, major ports and waterways are pretty wide. For example, the coast pilot linked a boatnerd.com [1] (who doesn't love that url) says for the mississipi - illinois waterway connection:
(10) Mississippi River-Illinois Waterway-
(11) depth, 9 feet (2.7);
(12) width, 80 feet (24.38 meters);
(13) length, 600 feet (182.88 meters);
(14) vertical clearance 17 feet (5.18 meters);
So, you can transport something that is roughly 20x150 meters. Some random internet site [2] says that "The standard barge is 195 feet long, 35 feet wide, and can be used to a 9-foot draft. Its capacity is 1500 tons. Some of the newer barges today are 290 feet by 50 feet, double the capacity of earlier barges." So, if we get one of them "newer barges', we can transport something that is 75 x 15 meters and weighs 3000 tonnes using equipment that is standard on the infrastructure.
A random wiki quote [3] says that "In the United States, 80,000 pounds (36,287 kg) is the maximum allowable legal gross vehicle weight without a permit.". So, with standard equipment you can transport something on the ground up to 36 metric tons, or about 0.1 percent of what fits on the barge*. Of course, you *can* transport something bigger than that, but then you get into serious logistic operations with special equipment, road closures, etc etc, while the barge can just be loaded up and sail away.
tl;dr: roads are made for fast and flexible transportation of relatively small amounts of cargo; shipping is made for slow transportation of bulk and large items.
[1] http://www.boatnerd.com/facts-figures/cpgreat.htm
[2] http://www.caria.org/barges_tugboats.html
[3] https://en.wikipedia.org/wiki/Semi-trailer_truck
*) I'm totally ignoring any possible short tonne, long tonne, metric tonne etc errors here, since that won't make a dent into a 3 orders of magnitude difference...