Well, Jack will most certainly do more than "Single pipe in/single pipe out"
I used to have my box set up running brutefir (a filter program) in jack. I would run the outputs from my buses in ardour to both channels 7-8 on my soundcard for monitoring on my headphones, as well as to the inputs of brutefir for separation into Sub/Woofer/Tweeter channels, which ran out of brutefir to channels 1-6 on my card.
That aspect of it is great. The problems are that jackd (and the apps that depend on it) crash far too easily.
The odd time that I did something really stupid and caused an underrun jack would usually crash. I'm not sure if it's jack itself that started the crash, or brutefir dropping out causing it to crash, but anyway you look at it, it meant killing all of the audio apps (which frequently hung when they lost their connection to jackd), restarting everything, and then reconnecting all of my flows.
Obviously I shouldn't be getting underruns to begin with, but if I do, I should get a report, and a botched recording, rather than a large conglomeration of crashed and hung apps.
One of the biggest things required is a consistent standard for linux audio. Maybe a jack-like framework implemented in the kernel.
Basically, we just need something that everyone can actually use, rather than varied support for the many sound daemons allowing only certain sub-sets of programs to work together without a lot of hassle.