The researchers first used psychoacoustical methods to test how well human voice identity is preserved in deepfake voices. To do this, they recorded the voices of four male speakers and then used a conversion algorithm to generate deepfake voices. In the main experiment, 25 participants listened to multiple voices and were asked to decide whether or not the identities of two voices were the same. Participants either had to match the identity of two natural voices, or of one natural and one deepfake voice.
The deepfakes were correctly identified in two thirds of cases. “This illustrates that current deepfake voices might not perfectly mimic an identity, but do have the potential to deceive people,” says Claudia Roswandowitz, first author and a postdoc at the Department of Computational Linguistics...
The researchers then used imaging techniques to examine which brain regions responded differently to deepfake voices compared to natural voices. They successfully identified two regions that were able to recognize the fake voices: the nucleus accumbens and the auditory cortex. “The nucleus accumbens is a crucial part of the brain’s reward system. It was less active when participants were tasked with matching the identity between deepfakes and natural voices,” says Claudia Roswandowitz. In contrast, the nucleus accumbens showed much more activity when it came to comparing two natural voices.