Actually, no, they are not "private affairs."
Senator Kerry voted for, and Bush signed, a law heavily regulating the broadcast of political commercials (BCRA). The Commission on Presidential Debates is in violation of that law -- they are operating as a 501(c)(3) non-profit, while making HUGE donations to the Bush and Kerry campaigns in the form of commercial advertisements -- the "debates."
If all of the candidates were allowed to participate, the "debates" would be "debates," or at least legitimate non-profit educational expenditures. Since all candidates are not invited, the "debates" are actually campaign commercials for Bush and Kerry, illegally funded to the tune of millions of dollars by the CPD.
As the situation now stands, the CPD, Bush and Kerry are in violation of the law. Far be it from me to put down Badnarik or Cobb for attempting to help Bush and Kerry to live within the law that they created, even though I think that law is stupid.