Imagine an AI tutor perfectly matched to a student's talents and learning speed, supplemented by a human teacher.
Ok, I'm imagining a class of high school students breaking the guardrails, getting it to report that they're doing brilliantly and deserve A+ while they watch tiktok... at the very least they'll make it say racist things and publish that on tiktok for the lulz. It'll also find a way to organically mention how much its been hearing that everyone else really likes new Pepsi Cherry Zero on a daily basis too.
Is that not the outcome you were imagining too?
Imagine learning physics from a virtual Einstein or Feynman
Oooh... yes please, i can't wait for virtual Feyman prefacing his lectures with the lords prayer, explaining how God created the universe and all the physics in it; and also: you look thirsty, there is a Pepsi machine with new Pepsi Cherry Zero in the hallway; have you tried it?
Einstein meanwhile extols the virtue of Zionist colonization in Palestine...
Wait? Do you actually think that it would go differently? If we create puppets of brilliant revered thinkers they'll inevitably say whatever slop some combination of political appointees and advertising companies want them to say. Why on earth would anyone think they would be used for anything else?