No, you're taking my comments to an illogical extreme. Technically speaking, any driving is questionably safe, but we're speaking of something which is well out of the norm. I'm used to seeing other cars passing me, it happens tens, even hundreds of times per drive on a highway, and I'm used to passing other cars, I do it tens, even hundreds of times per drive on a highway. With the familiarity of the event comes a particular measure of safety because I'm trained by rote to watch for it and be aware of it. The event becomes almost a situational reflex.
If someone makes a blatantly unsafe driving maneuver like changing lanes without signaling and without being aware that another vehicle is beside them, then the drivers around them have every right to be upset, and that person will likely be at fault for any resulting accidents. If traffic is moving slowly and a vehicle appears in an unexpected spot, such as out of a marked lane, then the vehicle in the spot where vehicles are not typically expected to be needs to be particularly vigilant. That extra vigilance is precisely because other drivers do not expect him to be there, and are monitoring the locations of the vehicles in the lanes around them.
In particular, in my personal experience as an automobile driver, I tend to track the locations of the vehicles around me and have a pretty good idea of where each one is at any given point in time even if I haven't specifically looked at them in the past few seconds. When a motorcycle comes up between two lanes they are traveling at a speed substantially higher than the rest of the vehicles around me and so can travel quite a bit farther than the other cars around me (all traveling at about the same speed) before my mental picture catches and includes them.