I have been doing this for 12 years. For the first 10 years I alternated between using Linux (which at times made it very difficult to interact with Windows clients) and using Windows, with all its attendant problems.
Here is what I have been doing for the past two years: A Ubuntu desktop system with multiple Windows virtual machines, using VirtualBox.
From the Linux system I can do most of my desktop work; PPTP VPN into customers and our other offices; RDP into Windows Servers; ssh into Linux systems; and LogMeIn into Windows desktops.
From the Windows VM I can run MS Outlook (when your organization runs Outlook you can make life VERY hard for yourself by using any other email client) and I use Cisco VPN and Cisco AnyConnect when necessary. I know there is a Cisco VPN client for Linux but I never got it working smoothly. As far as I know there is not an AnyConnect client.
When it comes to document compatibility, I can use MS Office in the Windows VM, or OOo in Linux -- they seem to interoperate very smoothly. But when needed I can run Visio or PowerPoint in MS Office.
The part I love best about this solution is the stability and restorability. I have multiple Windows VMs. One is my "real" work environment, so I back it up carefully (it's just a file!) and I don't install any suspect software. Another VM is a sandbox for when I need to load some software with a big footprint, or something I may not keep. I also have VMs for Windows Server, Windows development, etc. Limited only by the size of my disk.
Oh the other best part: many of those Cisco VPN connections force all of your network traffic through their tunnel. That can be fatal to productivity. So fire up a Windows VM and VPN from there. Your "real" desktop system is not affected -- you can still email, IM, surf porn, whatever.