I think what your looking for is sandboxie.
Its sort of like solaris containers, AIX's WPARS or LXC.
Others listed here: http://en.wikipedia.org/wiki/Operating_system-level_virtualization
The problem with these solutions is that when you want to actually share data (think clipboards, or word documents) you have to poke huge holes in the security model to get them in/out of the container. That is the fundamental issue with windows, either you allow applications the ability to extend the environment (think adding thumbnail viewers for proprietary image formats), share data, or even do some level of application embedding/etc it becomes very difficult to secure that environment from a rogue application.