IFM releases K2 Horizon with its weights, data, and training recipes

The Institute of Foundation Models has unveiled K2 Horizon, a family of six open models for personal devices, workstations, and servers, with Apache 2.0 weights, long-context support, training data, and code.

Six models developed through the same training program, ranging from a compact system intended for personal devices to a 375-billion-parameter model designed for servers. With K2 Horizon, the Institute of Foundation Models is releasing more than a single general-purpose model. It is offering a lineup designed to distribute workloads according to the capabilities and computing resources available.

Introduced on September 3, 2026, by the institute affiliated with Abu Dhabi’s Mohamed bin Zayed University of Artificial Intelligence, K2 Horizon includes models with 0.9 billion, 3.7 billion, 7 billion, 32 billion, 36 billion, and 375 billion parameters. Their weights are available on Hugging Face under the Apache 2.0 license.

The family primarily targets reasoning, coding, tool use, and agents capable of completing multistep tasks. It supports conversational exchanges, separate reasoning outputs, and function calls in a format compatible with OpenAI-style interfaces.

IFM describes the release as the largest fully open-source model launch in AI history. That claim cannot be verified without a precise definition of what constitutes a single launch and a comparison with every model family released to date. The range of resources announced is nonetheless unusual: final weights, intermediate checkpoints, pretraining and post-training code, data or construction methods, training logs, and detailed evaluation results.

The six model names use rounded sizes or focus on their main parameter counts. Hugging Face reports slightly higher tensor counts for some repositories, at approximately 1 billion, 5 billion, 9 billion, 35 billion, 37 billion, and 379 billion parameters. This does not mean the models changed between their announcement and publication. The difference comes from how certain components, including input and output layers, are counted.

The 0.9B, 3.7B, 7B, and 32B versions use dense architectures. All their parameters participate in every generation step. The two largest models use sparse Mixture of Experts architectures, in which only part of the system is activated for each token.

K2 Horizon 375B-A23B therefore stores 375 billion parameters but activates approximately 23 billion at each step. The objective is to benefit from a large total capacity without always incurring the computing cost of the entire model.

The 36B-A4B version pushes this approach further. It contains 36 billion parameters in total while activating approximately 4 billion per token. IFM presents it as an intermediate option for workstations and local infrastructure, with performance intended to approach that of the dense 32-billion-parameter model.

This version introduces an architecture called Mixture-of-Value Attention, or MoVA. Expert-based models typically apply their selection mechanism to layers responsible for processing and reorganizing representations. MoVA extends that selection to the computation of values used by the attention mechanism.

Several specialized groups are available, but only some are engaged depending on the content being processed. The system remains compatible with FlashAttention, grouped attention, and sparse methods used by deployment tools.

IFM says the 36B-A4B model performs slightly below the dense 32B model when both are trained under the same conditions, even though it activates about eight times fewer parameters. That theoretical efficiency does not automatically produce an equivalent reduction in real-world costs. Routing operations, memory transfers, and still-limited support for MoVA can restrict the benefit depending on the hardware and inference engine.

The four intermediate models, 3.7B, 7B, 32B, and 36B-A4B, were each trained on exactly 22 trillion tokens. They used the same data and a common methodology, providing a more controlled basis for studying the effects of scale and architecture.

The documentation more broadly refers to approximately 20 trillion tokens per model across the family. The published details do not yet confirm whether the 0.9B and 375B versions followed exactly the same data volume and sequence of training stages as the other four.

The data combines web content, code, mathematics, scientific publications, multilingual documents, and specialized sources. IFM says nearly 17% of the pretraining corpus consists of problem-solving trajectories containing explicit reasoning.

Approximately 10 trillion tokens were reportedly produced through the institute’s own generation pipelines. Some mathematical reasoning trajectories were rewritten in several forms, including dialogues and instructional guides, to prevent a particular reasoning pattern from becoming tied to a single writing structure.

Synthetic production used millions of combinations of instructions, diversity settings, and documents retrieved through an internal search engine built on the web corpus. IFM says the resulting diversity approaches that of high-quality natural web text, based on a compression metric developed internally.

That measurement is not an external evaluation of the documents’ relevance or the absence of semantic repetition. A corpus can be varied in form while still reproducing errors, similar reasoning patterns, or cultural imbalances.

Post-training does not begin only after pretraining has ended. Long documents, instructions, reasoning examples, and agent trajectories are introduced during an intermediate phase. IFM says it generated more than 100 million distinct tasks using problem taxonomies, diversity settings, and document searches.

The subsequent process includes supervised fine-tuning, model merging, and specialized reinforcement learning. Instead of following a single path toward a final assistant, the training program creates several branches focused on reasoning, code, tools, or agent environments.

The release of checkpoints is intended to help researchers identify when a capability appears, improves, or deteriorates. They can compare a branch before and after a particular training stage