I solder SMT components by hand as well. Don't even need a microscope; just head-mounted magnifier glasses is plenty. Make sure you have good light and plenty of flux and you're good to go.
But the problem is the board. Sure, if you have a finished design already, and you intend to actually use it in the future, then sending off for a finished PCB is good. But if it's just a hobby, and you're prototyping or just playing around to better understand a particular circuit, then spending a good chunk of money and weeks of time for a board is simply not feasible. You really want to set something up, try it, then tear it down and try the next idea.
With that said, I don't know that this is the answer either. Hand-drawn does not sound precise enough to handle SMT, and a whole separate device just making prototype boards sounds like too much money and space for a hobbyist. Perhaps the answer is desktop mills that become cheap and precise enough that you can use them to cut out boards from copper blanks along with other building tasks. At least that would not be a single-purpose gadget.