Playing loud music at 3AM is nothing like a natural right exercised without imposing on others - it is a clear over-the-line case, as in one person violating someone else's rights (the "quiet enjoyment" part of natural property rights). As the saying goes, your right to swing your fist ends at my nose.
So how loud? At what hours? Who decides that? Say one is a light sleeper working night shifts and the other a heavy metal fan who thinks the heavy beat is essential, are they going to agree on where the fist ends and where the nose is? Property rights are rather simple but I imagine the standard of "quiet enjoyment" is quite fuzzy.
What if I want to grow vegetables in my front lawn, but the local government has passed a zoning rule that says I can't.
I'm not sure what the deal is with the vegetables, but let's say it's a city ordinance against having trash lying around that might attract rodents and if there's rodents near your house they'll probably be around my house that might possibly at some point in the future get a rodent problem where your trash habits might have been a contributing cause. How far can I extend my interests before they'd clash with your interests to not give a damn?
Most libertarians I've met seem to have this simple world image where they live in their own bubble with their own property, like it doesn't interact with anyone else ever. Your car pollutes, should I have the right to tell you to stop polluting my air and get a bicycle? What about speed limits, who decides exactly what's reckless driving? I'm sure I can find some reason why you can't have a vegetable patch too, if I try hard enough.