I attended a course by Yale Patt from U. Austin, who is one of the "popes" of Computer Architecture research (see this ranking, for example), where he discussed Moore's Law.
He argued how Moore's Law was not a physical law, nor a technological or market-driven law. But it was a real law and had a very large impact.
Instead, he argued very accurately that Moore's Law was actually a psychological Law: given that it provided the baseline for the expected performance (or transistor count) increase, every company would struggle to get to that point, knowing that otherwise the others would get it and make a much better product. Therefore, Moore's Law became somehow a self-fulfilled prophecy. Note how the ITRS still provides yearly predictions for the (still exponential, in some cases) improvements in semiconductor technology fabrication.
So definitely you are right in that it was a mere observation -- but this observation drove the evolution of electronics for more than 50 years (enough to be labelled as "Law", isn't it?)