It could be stopped by allying with USSR before the latter allied with Germany. USSR did in fact explore the option of an alliance with UK and France first, specifically over the issue of providing protection guarantees for Poland. The USSR specifically wanted a clear and firm guarantee that if Poland were to be attacked, the Western countries would enter the war with Germany alongside with the Soviets in more than a token effort (i.e. they didn't want to end up being the only ones facing the Germans there). As the Phony War has shown later, it was not an unreasonable fear. The French were interested in exploring such an approach, but the Brits still believed the war was not imminent, and refused on the fear of being dragged into a war on behalf of someone else.
Additionally, Soviets wanted to extend the agreement beyond just Poland, and in particular to prevent the then-independent Baltic states from openly allying with Germany by treating such an act as German aggression that would trigger the joint military intervention provisions of the agreement.
While Soviets were already in talks with the Germans, it was the collapse of the tripartite agreement talks that prompted them to switch gears and seek a full fledged treaty with the Germans, which resulted in the Molotov-Ribbentrop pact.
The run-up to WW2 was really quite a mess in terms of who was supporting whom. Most people do remember the pact, but fewer are aware of the fact that Poland has participated in the partitioning of Czechoslovakia in 1938 on Nazi side, for example.