He is using Torch, anytime anyone says they wrote things by hand using python I'm suspicious. Truth is there is no reason not to use a machine learning framework, so you are always using something. (Whether that be PyTorch, Tensorflow, or something else)
I find it interesting that the model definition is probably the important part (assuming the model is using standard transformers or existing techniques)
Say someone else does the training, you just download the weights, it kind of makes the training process a chore. The code using a framework means the code is mostly throw away boilerplate. The only thing that matters is the results, and the model that generated them. I suppose the training data being available is important as well (as we see with ChatGPT) But then again, I think I would have moral qualms with what they are doing regarding copyright