New Protocol Exposes Vulnerabilities in AI Factual Accuracy

techtsp writes: An evaluation method called the Drill-Down and Fabricate Test (DDFT) has been developed to assess how large language models (LLMs) handle factual accuracy when subjected to degraded information and adversarial challenges. The protocol reveals that many advanced AI systems falter in maintaining reliable knowledge under realistic pressures, regardless of their size or design. Evaluations involved nine frontier models across eight knowledge domains at five compression levels, yielding 1,800 turn-level assessments.