Comment Optimise all the things! (Score 1) 17
They built a custom engine for a specific model and it went 30% faster. That's great, but it's inflexible and of limited use.
Of course these days, coding agents make designing and writing new code very cheap. With the right tools and toolchains it shouldn't be hard to automate the process of building a custom engine for each new model that comes along. The cost of doing this would be small compared to the cost of training the model in the first place, so really we should just be doing this as a matter of course. For a 30% speed-up it would certainly worth the effort of building those toolchains.