One of the fundamental tenants of the (literally) pre-historical concept of sovereignty, by which the people ordain and establish the U.S. Constituion, includes the ability to controls one's borders. The federal government, through the 14th amendment, has supremacy in defining citizenship of the United States, and through Article 4, at least a parallel duty to defend the States against invasion. You object that invasion means a group meaning to overthrow the goverment - a definition not found in the Constitution. Another definition of invasion is infrigment by intrusion, which ties back to the ability of the people's goverment to control borders.
So, historical pre-Consitutional ideas of sovereignty give the people the power to form a nation and the ability of that nation to control its borders, Amendment 14 specifically empowers the federal government with defining who is and is not a citizen of the United States, and Article 4 specifically empowers the federal goverment to protect the state's borders (against invasion, which necessarily posits who can and cannot legally cross the border) . You can add to that the Preamble, which states that the purpose of the Constitution is to insure domestic tranquility - legality of residence being central to national domesticity.
Perhaps you're a 10th amendment supporter getting confused about primacy in federalism, and instead of objecting to the argument that Federal law on border security preempts all State law on the topic (i.e. arguing that it does not and States also have the parallel authority to secure the borders), are thinking the inverse: that the federal government has NO rights to secure the State's borders? Which doesn't seem to be correct, because no one, not even 10th Amendmentists, are currently arguing that the federal government has NO right to deport non-citizens not legally in the U.S.; the argument is that the States should be able to do so independently of and in parallel to the federal government.