Hypothetically, if this were to happen to me, and I got a message from my bank asking me to call them to verify information, and I called this automated system, what message would I be presented with?
Are the phishers going to look up the phone number stored under 'Bank' and see which bank it actually is and then record the voicemail using my bank's actual name, and then ALSO have an automated system with my bank's actual name in it?
What if you were using a small-town bank? I highly doubt the phisher would be able to accurately determine your address to get the city and state required to lookup said bank, unless you also had that stored in your address book.
As a previous poster said, too many ifs and maybes.