I think you are thinking of two different things, one being The Fermi Paradox. Basically the idea goes that even if just one planet in this entire galaxy bore life, and it took at least as long as it took our planet to develop advanced life forms, they would of easily had enough time in the billions of years that have passed to colonize every single planet a thousand times over in the entire galaxy (Even if the maximum space travel speed they reached is far less than we have already achieved). Hell, even colonizing other galaxies is easy in that time-frame. Space in incredibly tiny compared to the amount of time that has passed. So even if we did not find aliens themselves, we would expect to find Alien mining complex ruins on mars, etc.
Now that is not an absolute proof. There has been a theory that would render it completely mood. The common theory is that life evolved from nothing to us in 4 billion years. Some people are saying that not only did life start here as space microbes from asteroids, but that perhaps 4 billions years is not nearly enough time, that this life already had billions of years of evolution to get to this stage. So perhaps right now is the first time in the history of the universe that non-microbial life has existed anywhere, and it potentially got spread to and created in multiple locations relatively simultaneously.
Not that these theories need to be complimentary. It could be that life arrived here from Mars or further, but it did not save a significant amount of time in an evolutionary sense and does not invalidate The Fermi Paradox.