Forcing SSL makes any hardware endpoint compression/optimization tools pretty much useless (Look at Riverbed products). You also put more of a strain on anything with smaller MTUs (VPN tunnels, PPPoE, dialup [Yes, it still exists]) or with higher latency (client in China, satellite users).
Additionally, you need one IP address per https website you want to host. This isn't an issue with IPv6 (Yay IPv6) or when using a webserver/client that can use Host headers before the SSL transaction (which all older browsers do NOT support). The main problem is that not everyone has a metric 'shitton' of IPv4 addresses and the software isn't wide spread enough to reliably host multiple SSL websites on a single IP with vhosts.
Now, there are some other circumstances that people will state that isn't really valid IMO. Some people will state that SSL certs cost money. I recently purchased a handful of wildcard certs from StartSSL for $20 (IIRC). The only thing that cost money was the identity check and even that was pretty painless. I highly recommend them for small/medium businesses or individuals. Their root certs are in iOS, Android, RHEL as far back as I cared to check (RHEL 4), Windows, Ubuntu, etc. </slashvertisement>