So the bad guy just got the password database from hacking slashdot and sees your password is sahcorrecthorsebatterystaple. The bad guy pulls up another password leak from hellokittyislandadventure.com, and sees an account with the same email address uses the password hlocorrecthorsebatterystaple as a password. It's entirely possible they'll figure it out given enough data points. You're right that it's an edge case, since nowadays the bad guys aren't doing much of that since there are so many users using "letmein" and "Password1", so you have to make a decision. Given the number of places you're reusing your password strategy, your knowledge (or lack thereof) of trends in identity theft via password leaks, and the value you place in your online identity, is it worth using password management software instead of memorizing a password algorithm?
In favor of password managers, when banks do stupid stuff like that you can use the software to make truly random passwords that follow those requirements. No need to modify your algorithm to fit within stupid restrictions.