It's only cheaper because our capital punishment process is so badly broken. It should not take decades to complete the process; that's just dumb. On the other hand, there are flaw in how it's applied currently (moving to the second part of your issue with it), so those also need to be fixed. I support the death penalty, but with some pretty major reforms. And as a strong advocate of it, I would be open to suspending it until said reform has changed the process to one which is much faster, cheaper, more humane, more fair, more evidence-based, and more regulated. For starters, take all the stuff the Innocence Project is doing and integrate it directly into the process and provide wide open access to all information going into the process to any third-party groups wishing to provide sunshine/oversight.
Some individuals are so dangerous and destructive that all members of society (including prison guards, staff, and other prisoners) deserve permanent protection from them. I have no issue with extinguishing the existence of those who are so fundamentally broken that we can't contain their violence. However, we need to bend over backwards to ensure the process to do that is applied fairly, reasonably, and is designed to make it as close to impossible to execute an innocent person as we can reasonably make it.