He was the one who kicked off European colonization and exploitation of the place. Other Europeans who came made only a tenuous foothold. Columbus was the one who said, "There's a place over there, and it's worth living in and taking stuff." He's the reason Europeans in general came to know about it.
It's not entirely out of keeping with other uses of "discover". The OED's first definition is "To disclose, reveal, etc., to others". The fact that it's first is historical, rather than a matter of present usage; the present use "to find out" is also very old. But it also includes notions of "finding out for oneself", i.e. not necessarily being the very first.
All told the OED gives over a dozen different shades of meaning for "discover", and I don't think this one is entirely wrong. It can be misleading, since as you say there were already people there and other Europeans had lived there, but he was an important "first" whatever word one applies.