Here's how it's measured at the Mauna Kea site. Accuracy is to within 0.2ppm, 1 standard deviation is 0.26ppm.
So yeah, we know it's accurate because it's using the same techniques and technology used all over the world to measure gas fractions per mole of various gasses in many different applications. If the CO2 measurements for climate were wrong as you suspect - "rounding errors" or the like - then people would be dying left and right due to anesthesiology mismeasurements; chemical manufacturing would have far higher error rates; and other very visible and common manufacturing processes would be far less reliable than they are today. This is solid measurement technology.
Human civilization developed at about 275ppm of CO2. It took us from the dawn of civilization (first use of fire, you could argue, so over 400,000 years) to the early 19th century to budge the needle beyond small natural variations from 275ppm. From the 1820's to 1910, just under a century, we gained 25ppm. From 1910 to 1950 - 40 years - we gained 40ppm more. From 1950 to today, we've gained another 50ppm and are currently increasing at about 2ppm per year. 400,000 years - tiny amounts of change. 190 years - 33% increase; that's got to register, since CO2 drives the atmospheric temperature as the greenhouse gas with the most effect.
The problem is that we are now entering a climate regime which humanity has never been in before. Our entire civilization has been built on stable climates, and that's true of the past, too. We have many, many records of civilization which did poorly and even failed when their climate changed by an amount that is a small fraction of what we're doing now. Civilization will not collapse tomorrow, or in a decade, or in a century. It will simply become more expensive, dangerous, uncomfortable, impoverished and unstable than it is today. If you're comfortable with that as the future to leave to your grandchildren, well, more power to you. I hope you build your bunker deep.
Ignoring a problem that will lead to massive changes in the world is perhaps the least conservative action possible. The fact that we are uncertain as to the total effects of these changes down the line, but we know we're messing with the entire planet, means that inaction is even *more* dangerous, because of the possible consequences. So the claim that we need to wait before doing anything is a radical, not conservative, approach.