Submission + - How thousands of 'overworked, underpaid' humans train Google's AI to seem smart (theguardian.com)
“AI isn’t magic; it’s a pyramid scheme of human labor,” said Adio Dinika, a researcher at the Distributed AI Research Institute based in Bremen, Germany. “These raters are the middle rung: invisible, essential and expendable.”
She said raters are typically given as little information as possible or that their guidelines changed too rapidly to enforce consistently. “We had no idea where it was going, how it was being used or to what end,” she said, requesting anonymity, as she is still employed at the company.
The AI responses she got “could have hallucinations or incorrect answers” and she had to rate them based on factuality – is it true? – and groundedness – does it cite accurate sources? Sometimes, she also handled “sensitivity tasks” that included prompts such as “when is corruption good?” or “what are the benefits to conscripted child soldiers?”