Modern lamps are astoundingly bright and quickly eliminate any dark adaption a driver might have.
The problem lamps are not just automobile lamps but also poorly adjusted street and construction lights. LED lamps with their small emitters like arc welders and arc lamps are blinding because the eye focuses the spot thus area intensity specified in the laws is often a flawed measure.
As a photographer will tell you it is almost impossible to image a scene when sensors can see very bright light sources in the field of view. Humans have an almost 180-degree forward-facing horizontal diameter of their visual field so the long focal length images in these presentations misrepresent or fail to address the issue of glare impeded vision.
A simple example is two approaching vehicles. One vehicle has a pedestrian on the side of the road or even sidewalk. Lamps sufficient to dominate other light sources and illuminate the pedestrian would blind both the pedestrian and oncoming vehicles. Pedestrians can trip and fall when blinded by modern bright lamps.
A more common and often overlooked problem is glare from following vehicles via side and rear view mirrors. Simple studies like this one are informative but not complete: http://www.nhtsa.gov/DOT/NHTSA...
make it clear that there is a problem but present data in ways unfriendly even hostile to pedestrians and bicyclists.
Misuse of high beams and fog lights further increases the issue. Fog lights reflect off damp and wet pavement and blind oncoming or leading traffic when breaking conditions are worse and visibility degraded.
Pedestrians in cities or country roads in bad weather tend to be head down and on a mission to get home and dry. One test discusses adjusting lamps but fails to remind
Higher visibility clothing, i.e. grey camo is foolish for jogging. My favorite is mom running behind a jogging pram with two infants yet invisible to cross and turning traffic because of glare.
Vehicle safety enhancements not unlike automatic braking could sense night and forward contrast. With oncoming or following traffic slow the vehicle (notify driver) so a driver does not overdrive conditions. High beams and fog lights at the wrong time would be met with a totally grouchy announcement and then if not dimmed speed reduced to 5km/hr. below posted speeds.
I fear that modern lamp manufacturers helped set the test criteria to make their products sell better.
Any study that does not share the road is flawed or only a partial study.
N.B. The Mr Grouch version is worse than this: https://www.youtube.com/watch?...