Yes, it's a bit of a slippery slope in terms of the government taking private property, but did you read the shit the drug cartels do? I fear my government like any good American, but I'm way more afraid of the drug cartels, And I don't live anywhere near the Mexican border.
You know that stupid argument people use to justify pot... like, "well, alcohol and cigarettes are way worse so weed should be allowed, too!" Just because we let one bad thing go on doesn't open the door for every other bad thing, even if it's objectively demonstrable to be less bad.
So, if drug cartels do horrible things without consequences, does that mean the next step is to allow the government to do terrible things because the alternative has the potential for being worse?