1. Do you offer a specific Drupal distribution that you manage, or do you just give the customer a LAMP stack and let them set up Drupal any way they want?
2. How much do you charge?
Drupal can be set up to be extremely scalable, but someone has to do that setup.
If you offer a Drupal distribution and it doesn't scale, that is your fault. If making it stable and scalable is "too much work", charge more.
On the other hand, if your customers are setting up Drupal by themselves, then it isn't your problem if they don't set it up correctly. If you don't want cheap customers that hire cheap agencies, charge more.
I know this sounds glib, but you wouldn't believe how often I have seen talented people bogged down with problems because they simply weren't willing to ask for the money they were worth. Higher rates = more money = less clients = less work. It may seem scary, but if you really are good, the good clients will stick with you despite the cost, and the problem clients will go be a problem for your competition.