ignore the DAC the amp the source and everything... ...except the speaker drivers themselves. even the best in the world are wildly non-linear.
and then there's the air between your ears and the speakers
Best source? .0001% THD. best amp? .0001% THD. Speakers? 1% THD haha good luck.
There is a fundamental problem with your argument, and that is the failure to take into account the nature and type of the distortion. It's not your fault - you share the misconception with most audio engineers, (who ought to know better), that THD figures correlate well with listening tests.
Quoting from my own comment in an earlier Slashdot story:
"THD measurements are taken as the ratio of the total power of all harmonics to the power of the fundamental, with no weighting of any kind applied. The trouble is, human hearing doesn't respond to harmonic distortions in this linear fashion - our ears find higher order harmonic distortions much more apparent and objectionable. This deficiency was noted by prominent BBC engineers D.E.L. Shorter and Norman Crowhurst in the 40's and 50's, when they proposed weighting harmonics by the square or the cube of the order; but their voices were drowned out by market forces that wanted a simple, flattering figure of merit that made the newer, more powerful pentode-based amps, (with lots of negative feedback), look better on paper than their lower-powered triode predecessors. The market won out over scientific and technical accuracy, (it usually does), and today engineers the world over, ignorant of this history, mistakenly believe that low THD is the gold standard for measuring and defining audio amplifier quality. (For a good technical analysis of distortion and the sound of an amplifier, see Lynn Olson's excellent investigation)."
Yes, speakers are hugely non-linear - but their non-linearity doesn't make distortions earlier in the reproduction chain inaudible, even though those distortions can be several orders of magnitude smaller. And that applies to all earlier distortions, whether they originate in an amplifier, a DAC, the digital encoding, or the recording equipment itself. Also, an amplifier with 1% THD can sound much better than one with 0.001% THD, not because the distortion in the 'poorer' amp sounds good, but because the distortion in the supposedly 'blameless' amplifier sounds bad. Then there are Intermodulation Distortion and Transient Intermodulation Distortion, which are difficult to measure thoroughly and seldom appear in amplifier specs, yet are often audible.
Audio quality isn't nearly as simple as THD figures imply, nor as simplistic as most manufacturers of audio equipment would have you believe.