ByteDance training 10-trillion-parameter model vs Anthropic

👁 119 reads

ByteDance is in the early pre-training stage of a model with as many as 10 trillion parameters, three times larger than Moonshot’s Kimi K3 - the biggest Chinese model released to date - according to three people with knowledge of the matter. The pre-training phase typically takes three to six months, after which the model would be fine-tuned and released if development proceeds as planned. The exact final parameter count will only be confirmed at a later stage.

For context, Anthropic does not disclose the size of its models, but industry estimates cited in reporting by Ars Technica put Mythos 5 at roughly 8 trillion parameters and Fable 5 at about 5 trillion. A larger parameter count sets a model’s fundamental capacity to store information, though capability also depends on data quality and training methods.

ByteDance’s model-development unit, Seed, is led by former Google DeepMind scientist Wu Yonghui and employs around 2,000 people covering research, infrastructure, data labeling and translation. The team has spent more than a year building models without distillation - the common practice of training a smaller model to mimic the outputs of a larger one. Founder Zhang Yiming has pushed this independent approach, telling the Seed team in an internal meeting two weeks ago to target “world-leading model capabilities” over the long run without becoming preoccupied with short-term competitive positioning.

ByteDance already operates at scale in consumer AI: its Doubao assistant has 324 million monthly active users in China, its SeeDance model ranks among the top globally for video generation, and its cloud unit Volcano Engine sells AI solutions to enterprises. The company has also invested in custom AI chip development.

Practitioner takeaway: For developers and businesses evaluating Chinese AI APIs, ByteDance’s push toward frontier-scale models suggests Volcano Engine’s enterprise offerings could become more competitive with top-tier Western APIs within the next year or two. The company’s rejection of distillation means any future model release would represent genuinely independent capability rather than a derivative - worth noting when assessing benchmark results or building products on top of Chinese model providers.

// read moreSony Music and Warner Sue Anthropic Over AI Training CopyrightNewsNash-MOPD: Balancing Training Across Multiple TeachersNewsPentagon Replaced Anthropic With Zero Disruption, Palantir SaysNews

Kinetyra