Not giving notice will look bad to both your new employer and your old. Why care about your old? You may end up working with some of those people again. Best to leave on a good note.
During my most recent job change, my old employer expected 4 weeks notice. Obviously, my new employer was not happy to hear that. In the end I compromised and gave 3 weeks. While no one was ecstatic, they both understood that I was trying to be as fair as possible.