Comment Re:So no software updates then (Score 1) 26
Would it be feasible to burn into the chip only a few layers of a transformer, for instance the first layers, and then train the rest? In this way you pair some speed-up and some trainability.
I'm not an expert, but as far as I know there are contexts where basically only training the outer layer gives you a decent-enough neural network.
I'm not an expert, but as far as I know there are contexts where basically only training the outer layer gives you a decent-enough neural network.