The Express editions have a bunch of arbitrary limitations in them.
The two that bit me were:
1. You can't install plugins. I don't currently use any I can't live without, but several features in VS2013 -- e.g. NuGET, the thumbnail view replacing the scroll bar, better refactoring, visual indent level indication -- started out as plugins. Even if you take the view that eventually, all third-party plugin features eventually make it into the retail version, you're opting into being years behind the current state of the art.
2. The Express editions are artificially siloed into several versions, none of which has all of the features. If you need two features that are in different versions, at best you have to keep bouncing between the editions. If you need both features simultaneously, you're stuffed.
For me, the two features I needed simultaneously were the ability to create a mixed C# and F# program that ran on the desktop. To make a C# desktop app, you naturally need the desktop edition, but that edition doesn't include any F# support. For some demented reason, that's off in the Web edition, where it seems focused on ASP.NET development, not desktop development.
(And if you ask me why F#, well, this is Slashdot, isn't it? If I'd said Haskell instead, you'd just be nodding now. :) )