When you download email from Google, it's still cached on the local machine so you can view it. When you're downloading email from your own POP account, it has to be transferred across a firewall, switches, and so forth, some of which might cache the information. They would not have to contact you in order to obtain access to your email, and they would not have to contact your email provider or someone who you have entered into a secure agreement with. They would simply have to contact the person who controls the router between you and your email server. (Some of which are already controlled by the government.) In regard to SSL, some corporate firewalls are using the client key to decrypt the emails and web pages to transfer them more quickly through their networks since SSL is a huge taxing process on the system.
Whatever you do on the internet or in email is trackable and traceable. They don't have to touch your computer to find out what you are doing. Also since you are licensing your operating system from a company that makes operating systems, I'm sure there's another loophole there as well.
If you aren't doing anything wrong, then there is nothing to worry about.