There is a crap-ton of evidence placing quasars at cosmological distances. Arp's idea is one of the DISCARDED ideas about what quasars are for really good reasons.
Starting with - why are there no BLUE shifted quasars? If they are ejected from galaxies, we should should see ones coming at us as well as receding from us.
We have images of gravitationally lensed quasars while necessarily places them FURTHER AWAY than the galaxies acting as lenses. We've even witnessed time delayed changes in the multiple images from those lenses.
We have pictures of some of the galaxies quasars are embedded in - which have the SAME redshift as the associated quasar! Quite the coincidence that, eh?
We can measure adsorption lines in their spectrums from intervening clouds of gas. Again, allowing us to place minimum distances on the quasars since they MUST be further away than the clouds of gas.
We can measure all kinds of properties - and they all agree: Quasars are at cosmological distances.