Your father was wrong.
I somehow am not allowed to play M rated games, despite being 40 years old, if I have children with me?
Am I not allowed to buy alcohol, birth control or donuts, either?
Granted, your dad worked at Wal-Mart, which probably wouldn't sell me the birth control with or without children...but the point is, I am a mature, responsible adult, and I like to play video games. I also have the self control to play them appropriately, and don't need some stranger deciding for me that I'm not allowed to own it.
As you said, there is such a thing as games for adults. As an adult, I should be allowed to play that game, without interference from your father. Unless he felt like babysitting for me for free? Special needs child as well, he'd need additional training, much more than his Wal Mart job!
Ridiculous. You should be embarrassed to tell this story. Your father is an ass.