News
A large model of byte training with 1 billion parameters is exposed, which may surpass Mythos 5. Zhang Yiming and Liang Rubo have spoken out.
2 min read
Source: zhidx.com
Zhidongxi Author | Wang Han Editor | Yunpeng Zhidongzhi reported on August 7 that today, the Financial Times quoted people familiar with the matter as reporting that ByteDance is training an AI model with up to 10 trillion parameters. And this number was still “over 5 trillion” in LatePost’s exclusive report yesterday evening. Even if it is 5 trillion, the scale of this model has exceeded the Kimi K3, the largest model released in China, and is close to Anthropic's Fable 5; if it is 10 trillion, the scale of Byte's new model will exceed Mythos 5, which has about 8 trillion parameters. As of the publication of the article, ByteDance had not responded to this. ▲The Financial Times reported that according to foreign media reports, one of the people familiar with the matter revealed that Byte's model is still in its early stages and is currently undergoing pre-training for a period of 3 to 6 months. If all goes well, it will be fine-tuned and released later. The specific model size will be determined at a later stage. According to LatePost reports, Byte’s new model will be led by Xiang Liang, head of Seed Foundation, and will cooperate with Shen Ke, head of large language model pre-training data. The revelation of the "Big Mac" model of Byte has attracted the attention of many netizens on the Internet. Some netizens used the classic line from the Marvel movie "Spider-Man" "With great power comes great responsibility." and said, "The greater the power, the greater the competition. It is not surprising that (AI) has reached this point. If their model can continue to evolve like Kimi K3, it will force everyone to speed up to catch up, otherwise they will be eliminated." ▲ Netizen comments (Source: X) Some netizens believe: "The accounting logic of the inference has been rewritten before the 10 trillion parameter (model) has been officially launched." ▲ Netizen comments (source: decide on this split