Thomas Wolf, co-founder and Chief Science Officer of Hugging Face, opens by mapping the open-model landscape — GLM at #2 on Artificial Analysis and the field of Chinese "AI dragons" (DeepSeek, Moonshot/Kimi, Z/GLM, MiniMax) — then hosts a fireside with MiniMax's Olive on M3. Released in June as the top open-source model at the time, M3 is a compact ~400B total / ~20B active (cited as 428B/23B) model with native multimodality (text, image, video) and a functional 1M-token context powered by MiniMax Sparse Attention (MSA). The conversation traces MiniMax's long-context lineage (M1, MiniMax-01), its from-the-first-step multimodal training, its intern-driven research culture, and its commitment to open source.
Wolf opens by situating the moment: attendees just saw GLM, currently number two on the Artificial Analysis table, and the session lines up the top open-source models in a row. He introduces MiniMax's Olive, who did a PhD at NYU in Yann LeCun's lab working on JEPA before joining MiniMax rather than Hugging Face.
He notes the sheer breadth of the field — around 64 labs ('nails') worldwide — so it's forgivable not to know them all.
Wolf groups the leading Chinese open-model labs under the banner of AI dragons: the well-known DeepSeek, Moonshot (which makes Kimi), Z / GLM, and now MiniMax. He characterizes them as extremely good, extremely talented teams all fighting for first place, setting up MiniMax M3 as the latest headline release.
The framing transitions into the fireside: M3 shipped in June as the top open-source model at the time — ~400B total / ~20B active parameters — combining coding, vision (images and video), and a 1M-token context via the openly shared MiniMax Sparse Attention. Wolf teases the deeper technical discussion of long context and attention efficiency that follows on the companion page.
The MiniMax representative introduced M3 as a model released earlier that month with around 400 billion total parameters and 20 billion activated, later restated as 428 billion total and 23 billion active. Despite its relatively small size, it is described as strong at coding while also understanding video and images.
Wolf emphasized that M3 is, in his view, still the only top-five open-source model that is genuinely multimodal. The team framed coding, agentic capability, longer context, and multimodal understanding as the four pillars they deliberately combined because they believe all four will matter for future AI applications.
The long-context story goes back to MiniMax M1 and MiniMax-01, which could handle up to 10 million tokens of context but were not agentic. The team realized that longer context unlocks capabilities when agents interact with users and environments, receiving tool responses across many rounds where shorter context would fall short.
For M3 they built MiniMax Sparse Attention (MSA), described as scalable with a simple design. At a high level it has an index branch that selects, at a higher level, what matters most in the context, and a sparse attention branch that performs computation over the selected blocks. Wolf drew a parallel to the field's earlier N-squared and linear attention research that faded when flash attention arrived, and welcomed the return to first-principles thinking about efficient attention.
The model card notes M3 was trained as a multimodal model from the first step rather than after text pre-training. The representative called this native modality. Many labs add vision via adapters after text pre-training or through continued pre-training partway through, but the team found this harms text performance, causes vision understanding to converge poorly, and is highly recipe-sensitive and hard to scale.
Training from the very first step is the most natural approach but risks the model collapsing after a few steps. The team solved this with extensive work on the ViT, on interleaved data (keeping images and videos in the data instead of masking them out), careful cleaning and masking, and strong reward modeling, allowing it to scale without collapse.
The MSA architecture was reportedly designed by an intern, which the representative noted is unusual since interns at many labs lack access to the data and work. MiniMax provides good foundations and infrastructure so anyone can experiment with the model, propose evaluations, find weaknesses, and lead projects that can run for weeks or even months before shipping into final training.
On products, MiniMax has been model-first from day one, with the CEO's dream of AGI and a model that understands all modalities predating ChatGPT. Its apps reportedly reach more than 300 million people across roughly 200 countries and over a million companies. The team remains committed to open source, valuing community feedback and PRs, and internally runs automated research harnesses using M3 for kernel optimization and data generation, with M3.1 already in development.
"You just saw GLM which is current number two on the artificial analysis table."
"For those who maybe don't know all the labs around the world, you're forgiven, because I think there's like 64 labs right now."
"M3 we released M3 earlier this month and it is a smaller model with 400 around 400 billion total parameters and 20 billion activated"
"it has a super long context of 1 million with our new architecture called MSA minimax sparse attention"
"the story about long context went back to even Miniax M1 and Miniax01 where the model was actually was able to perform tax of 10 million token context"
"the architecture was actually designed by an intern"
| Time | Topic |
|---|---|
| 00:00 | Thomas Wolf welcomes MiniMax's Olive |
| 00:32 | GLM at #2 and the open-model lineup |
| 01:03 | The 64 labs and China's AI dragons |
| 01:35 | Introducing MiniMax and M3 |
| 02:05 | M3 specs: parameters, coding, and vision |
| 03:37 | Long context lineage: M1 and MiniMax-01 |
| 05:08 | How MiniMax Sparse Attention works |
| 09:49 | Native multimodality from the first step |
| 12:27 | Apps, scale, and open source |