This is not really an issue when you switch phones, the problem is the other users has set to always to use iMessage. Normally, if a "message" don't reach an iPhone user in a set amount of time, the system defaults to sending it as a text. That's the default behavior of iOS out of the box. Some users have turn it off in Messages settings therefore the iMessage never delivers the message and continues to wait it out.
Why would someone turn off "Send as SMS"?
Few reasons:
1) they're on Verizon or ATT and it's $10 minimum text plan,
2) the limit before it sends as SMS is really short (something around 5 seconds), and you don't want to annoy everyone with multiple iMessage and SMS for the same message
3) the user like knowing if the message actually arrived