I started using Google Gemini to evaluate applications of transfer students for credit for specified courses at my university.
What a person in my role is supposed to do is email faculty colleagues teaching the course in question, but good luck with that. A faculty member wants such requests to just go away. Even if you know the contents of a course you have taught for years, how do you know that Cow College's offering is anywhere close?
The AI not only has a lot to say on how equivalent a pair of courses, if it is just making stuff up, I don't see how it can be worse than what my colleagues say.
Maybe if I don't call attention to what I am doing, no one will notice the difference. Maybe it is an improvement. A grad student turned in a form and left out which courses they wanted credit for. It came to me but requesting an undergrad-level course, and this topic is in the "wheelhouse" of the student's PhD advisor who approved. I sent it back that my "research" indicates that the course taken at MIT is advanced-graduate level at our Cow College and suggested an advanced grad level course here that according to the AI is a "great match."
I sent this back to the student--is the student going to complain?