Actually, the structure of quantum mechanics is quite simple--objects (states) that are defined on a linear vector space are about as simple as you could possibly get. The axioms of the theory have analogues in optics and wave mechanics--are those any less beautiful or elegant?
Some of the quantum numbers are in fact not arbitrary--they are constrained by the solutions to the partial differential equations from which they are derived (example: the spherical harmonics in the solution to the hydrogen atom--not Mills' version...).
Quantum mechanics, and its relativistic extension into quantum electrodynamics has provided us with the most accurate measurement of any physical quantity (the anomalous magnetic moment of the electron). I'd say the theory deserves a little more than only being correct "mathematically speaking".
Feynman's (and Dirac's) "sum-over-histories" path integral approach to quantum mechanics is actually quite beautiful, and even has applications in "nonquantum" physics. It might be the genesis of the theory that will eventually subsume quantum physics.
It took Euler, Lagrange and Hamilton a century or so to bring classical mechanics to a new level. Don't be too hard on the quantum. It's still quite young.