Comment Why? (Score 4, Interesting) 14
That said, the huge Chinese models we've seen lately aren't single models, they're MOEs - Effectively a hive-mind of many smaller models working in concert. Bytedance hasn't mentioned that detail yet but it would somewhat mitigate the training issue. This is still absurdly over-spec'd though. There's simply no way to train it any more than all the rest starving for data.