It's hard to do, but perhaps not as hard as you make out.
You can't really compare an interstellar exploration or colony ship to a planetary probe. Probes have little or no propulsion, relying mostly on the initial boost, directional thrusters, gravitational slingshots, and time to get where they're going. Plus, you can't speed them up too much, because you need to achieve a sensible orbit when you reach your destination instead of just hurtling past it.
Instead of thinking in terms of speed, think in terms of acceleration. If you just apply constant acceleration -- even weak acceleration using means that are feasible today --for a long period of time, you can achieve fantastic speeds. If you hit relativistic speeds, time dilation will reduce the length of the trip even further (for the spacecraft occupants, but not for earth).
Really, all we need are efficient, high-specific-impulse rockets and a crapload of fuel, enough for a long sustained acceleration burn and then some deceleration at the end. (I suppose you could use gravity assist to slow down at the other end too.) I haven't done the math, but I suspect you could reach Alpha Centauri within 20 years using technologies that are already on the drawing board. Maybe even ion engines could get it done.