I'd expect drivers are slowing down because the road is less safe without the lines
You've just described virtually all "traffic-calming" techniques, in a nutshell.
Let's make roads narrower; let's add more curves to reduce visibility; let's have tall plants grow right up to the pavement so you can't see the deer; let's add random obstacles like rotaries-without-intersections; and now, let's take away the lines that let people know where and how many lanes exist!
All these will make traffic slow down, because they all make the road substantially more dangerous. You really want to make the road safer? 100% divided, with 100% gated fencing to keep kids and wildlife from wandering into the road. The rest of this just amounts to games, by nannies desperate to pretend that the real problem has anything to do with "speed" .
On the bright side, this suggestion at least does have one positive side-effect, no doubt totally unintentional: without lines, Everywhere becomes a passing zone.