I can't tell if you need to be already on the network to use it based on the video.
But, Chromecast had in its FAQ even that anyone on your network had access.
They had some shot in the video of disconnecting and reconnecting to the phone, was that resetting it to search remotely? seems likely that it is possible to do that (play videos on someone else's device in range), but hardly devestating. People have TVs with IR remotes too, they can be "hacked" in pretty much the same way.