For simple manual stepping, etc. debugger speed is probably not that important. Have you ever tried to really use watchpoints? Given up after your program slowed to a crawl?
We had a debugger that could run many watchpoints at almost full program speed some time ago, see http://www.netilium.org/~mad/dtj/DTJ/DTJK07/ for details. The key point is that Parasight was fast enough to make tools like this useable.