Actually while slavery was highlighted (especially by the North and now the left) it was indeed about the States losing their rights. As with other rights such as our personal right to speech, you have the right to misuse it. There can and often are consequences to misusing it, however we should never allow the government to remove the right itself. So again it wasn't about slavery it was about rights that the States used to have and the Federal government removed.
Fighting against slavery is a noble cause, as is fighting against terrorists and all the other bad things in the world that governments use as pretexts to removing rights. Sounds as though you agree that it was acceptable to remove the States' rights in order to fight slavery in the past. Do you also agree it is acceptable to remove our rights today to fight terrorists?
If you looked into the laws being passed by Congress prior to the Civil War you would find that the South was being put in a serious bind. They were being forced to sell their raw goods to factories in the North at a lower price than they could get from European factories. Additionally laws were enacted that made creating their own factories difficult. And yes in addition to those items there was concern that they would be forced to give up the slave trade. It is much easier for most of us to do something that is good or good for us when we at least have some say in making that decision. Have you really never balked against doing something for no better reason than someone else was making you do it? I know I have and I have seen countless others do it also.
The North and the South were both borrowing heavily from Europe for the first two years of the war. Can you guess what caused a very anti-slavery Europe to stop funding the South? That's right the North only struck down slavery because it would help them win the war not because of altruistic reasons.