MAI-Base-2: Microsoft details its new training infrastructure

Microsoft unveils MAI-Base-2, its 2.8 trillion parameter model, pretrained on 16,000 NVIDIA GB200 GPUs over 38 days.

2.8 trillion parameters, 16,000 NVIDIA GB200 GPUs, and 38 days of training. With MAI-Base-2, Microsoft AI details the development of its new foundation model, built from its own infrastructure and training data. The system activates 80 billion parameters per token and natively integrates visual understanding. Microsoft also claims higher training efficiency than DeepSeek-V4-Pro-Base in its internal evaluations. Pre-training is complete, but the model has yet to go through an alignment phase before being offered to users. 2.8 trillion parameters, including 80 billion active

MAI-Base-2 relies on a Mixture-of-Experts (MoE) architecture, in which different groups of parameters are routed depending on the information being processed.

The model totals 2.8 trillion parameters, but only 80 billion are activated for each token. This architecture distributes knowledge across multiple experts while limiting the amount of computation required for each operation.

Unlike MAI-Base-1, Microsoft chose to place MoE blocks at every layer of the new model. This change required a reorganization of computing and communication operations between GPUs to reduce performance loss.

MAI-Base-2 succeeds the model that served as the foundation for MAI-Thinking-1, introduced in June. The new generation is larger, sparser in its parameter activation, and trained on an expanded dataset. 38 days of training on 16,000 GPUs

Microsoft trained MAI-Base-2 for 38 days, between August and September 2026, on 16,000 NVIDIA GB200 GPUs.

The company reports two metrics regarding the utilization of this infrastructure.

The first, called Model FLOPs Utilization (MFU), measures the proportion of the processors' theoretical capacity actually leveraged for computation. It reaches an average of 21% across the entire training run.

The second, goodput, represents the proportion of time during which training effectively progresses. Microsoft reports a rate of 89%.

These results account for the constraints encountered during very large-scale distributed training, including data transfers, hardware synchronization, and potential interruptions.

Teams also improved the execution of MoE blocks. Their utilization rate rose from 24% to 35% following several optimizations aimed at limiting redundant operations and memory loss. Between four and twelve times more efficient than DeepSeek, according to Microsoft

Microsoft presents MAI-Base-2 as more efficient to train than DeepSeek-V4-Pro-Base, its main point of comparison among open-weight foundation models.

According to published evaluations, Microsoft's training method would require between 4.2 and 12.3 times less computation to achieve comparable quality, measured on held-out evaluation datasets.

This estimate is based on curves linking training resources to achieved performance. Microsoft compared several experimental versions of its models to estimate the resources needed to match DeepSeek's results.

Compared to MAI-Base-1, the company claims an efficiency improvement between 6.3 and 12.7 times.

These figures do not mean that MAI-Base-2 was trained with twelve times fewer total resources, nor that it will cost twelve times less to run. They describe the estimated efficiency of a training method at equivalent quality.

Microsoft also indicates that MAI-Base-2 outperforms selected open-weight models in its evaluations across text, code, mathematics, and scientific knowledge.

These comparisons primarily focus on the pre-training phase. Some competitors are evaluated in their post-trained versions, which limits the scope of direct comparisons. A method to predict performance before full training

One of the technical works detailed by Microsoft involves scaling ladders—families of models of different sizes trained using a common method.

The lab uses these intermediate models to determine training hyperparameters before deploying a much larger infrastructure.

A challenge arises when small models encounter the same data multiple times, whereas large models see a greater diversity of data.

To limit this bias, Microsoft developed a method called source scaling. It adjusts the amount of unique data used during small-scale experiments to maintain proportions comparable to those of the final model.

Teams also established new mathematical relationships to select learning rates, regularization, and other technical parameters.

These experiments made it possible to anticipate MAI-Base-2's results more accurately before its full training run, and to select configurations tailored to its size. Optimizations to prevent memory loss

Distributing tokens among MoE experts creates specific constraints. Some experts can receive far more data than others, causing imbalances and memory issues.

Microsoft developed a multi-pass execution system with fixed memory capacity for each group of experts.

When the volume of tokens exceeds the available capacity, additional operations are split across multiple passes instead of being dropped.

The system can also reuse unallocated capacity from other experts within the same group.

This setup helps avoid certain outages related to memory saturation, while minimizing unnecessarily reserved resources.

In parallel, Microsoft optimized GPU-to-GPU communication using NVLink, operation grouping, and better coordination between transfers and computations. Muon and a new training stabilization method

MAI-Base-2 also uses Muon, a parameter optimization method that Microsoft deems more stable and efficient than AdamW in its large-scale experiments.

Teams adapted its design to distribute operations across thousands of GPUs.

After optimization, Muon accounts for 2.1% of the average training iteration time, or about 165 milliseconds.

Microsoft also developed a gradient clipping method applied to the model's intermediate activations.

Rather than globally reducing adjustments when an anomalous value appears, this technique allows for more targeted intervention.

It was also used to detect issues within the data. Researchers notably