1) The US has not ratified UNCOLS, it does not care what is considered international waters.
2) Even if the US has ratified it, military would be allowed "innocent passage" subject to local regulations. Launching an unannounced missile is neither innocent nor regulated.
3) The Channel Islands are not international waters, they are archipelagic waters. The location of this thing was even pinpointed by a damn news station, it's right next to Santa Barabara Island. Well within US territory.
4) The trajectory of a weapon is irrelevant. Are you perfectly fine with someone sneaking up behind you and firing a gun in the opposite direction? The trajectory never crossed you, therefore a crazy man with a gun is not a threat? Bull
5) If this was an unannounced demonstration by another country, there is no international convention that would prevent the US from destroying or attempting to capture the ship.
6) If this was an announced demonstration then the ship would have been refused passage due to its non-innocent nature, meaning there is still no international convention keeping it from being destroyed.
7) The premise of this being a demonstration is that it was meant to demonstrate the ability to evade detection (we already know people can hit us with missiles, who would bother to demonstrate that?). That is antithetical to actually launching a missile, which immediately reveals your location. Also, if you REALLY wanted to demonstrate your sneakiness by launching a missile, why use a big expensive rocket? Send up something short-range, cheap, and shiny. The message is the same.
It's a US Missile (or at least US affiliated, either private or an allied country) and the agency which launched it has not been revealed yet, I don't see any other feasible option.