I used to work out after work, and that was a whole different scene (it, in fact, WAS a "scene"). The "pick-up" crowd tends to show up after work, and makes it annoying when you're actually trying to exercise and not socialize.
If you find you can't accommodate what you'd like to do (cardio, weights, whatever) at home, then I'd suggest hitting the gym in the morning. If you feel you need to add another level of distance from everyone else, then put on some headphones, even if you don't listen to them. Most people won't even give you a second glance, if they give you a first glance to begin with.