Comment Re:Shocked! (Score 1) 3
Yeah. There's been widespread misrepresentation of this (preprint!) in the media. What they actually found: test scores correspond to how much time you spend doing homework, regardless of whether you use AI or not. Which should not be a surprising result! Of course, the AI people, esp. those who had been using AI for more than a few months, stopped spending much time on their homework, and then their test scores fell to match the non-AI users who didn't spend much time in their homework either.
The TL/DR is, in today's age, you cannot use test scores as a proxy of how much time a user is studying or knows the material.
There are of course some caveats, beyond the fact that it's just a preprint - notably that there's nothing to separate why the user started using AI to do their homework, so there's potential confounders in there. For example, if a student is in a poor family, maybe they had to get a job to help pay the bills. But working a job itself can lower test scores - for example, having to work late the night before an exam. So there are some confounder problems with the study. That said, I'm sure the general conclusion is correct: time spent doing the thing = amount of learning the thing (relatively independent of what tools you're using in the process).