I think the idea of not going beyond line of sight is that you'll still be able to recover from FPV equipment failure, and also so you can be more aware of what's around the craft in the area you're flying (e.g. visually inspect what it's flying above).
You can simply look up, or take off your goggles (if you're using them instead of an LCD). In addition, some countries require a second person as a "spotter" to keep an eye out, visually, for things the pilot can't see.
Go beyond line of sight and you don't know what's around the craft other than what you can see through the (usually low quality) camera. Also the majority of autopilot mechanisms available today have no object detection and avoidance, so they can fly into solid objects (including people).