So let me see if I got this straight. Less than 300 years ago the fastest method of transportation was horse and buggey and sail ships on the Ocean.
In 300 years we now can put ships in space that can travel at 87,000 mph (143kph), and your best reasoning borrowed from Astrophysicists no less, is that aliens can't be here or make it here because the distances are just too vast ?!?!?
This lacks basic reasoning at minimum and surely imagination.
Imagine if you can how fast we'll be able to travel in space another 300 years from now. Now imagine how fast an alien race could travel if they were 5,000 years more advanced than we are. What about 100,000 years? 1 Million? 10 Million? Pretty sure after even a few thousand years the problem of going across these vast distances will be solved.