a) 64 bit processors can do 64-bit arithmetic in a single cycle.
b) The 64-bit processors in question have more named registers (fewer stack spills), and a significantly more efficient function calling convention (ABI)
c) 64-bit ABI doesn't touch the old x87 register set, which is another net performance win. (Not that VS2015 will use this much.)
Ergo: most of the time they are faster.
The only way to make a 64-bit program slower than a 32-bit one is to have enough pointer-chasing and associated irregular dynamic data that the change in pointer size materially affects the data cache miss rate. Certainly there is some code like that: VS2015 might even be an example.
How fast do you need your IDE to be though, and how much is performance really the instruction set's fault? Versions of Visual Studio have been produced that run in everything from
The issue is almost certainly that LLP64 is a dumb idea, and the code base will have lots and lots of pieces of historical code that assume that you can manipulate pointers with long arithmetic, and all of those are going to have to be found and fixed by hand, often involving real understanding and design decisions.