Because our technology was not advancing as fast as it is now, or capable of the level of destruction that it soon will be. The answer to the Fermi Paradox is that species destroy themselves when technology gets to point where true space travel is possible. Technology advances technology, to the point that technological evolution advances far faster than societal evolution. As a result, technology is created that a species is not responsible enough to handle, and it destroys itself either in the form of weapons or its inability to control the technology it creates.
How long do you think our species would last if a way was found to create a fusion device from household products that could be used as a weapon? We're one discovery away from killing ourselves off, and the faster our technology advances, the more likely that discovery is to be found.
I'm not saying that we need to stop scientific advancement, I'm just saying that it's very likely to lead to our own destruction due to our immaturity at handling the technology we have and are on the verge of discovering.