Fern can read half a million lines of code in a single context window

With Fern, Mirendil is developing a language model capable of processing up to 10 million tokens, to analyze code and long AI agent traces.

An entire software repository, several days of an agent's activity, or thousands of pages of technical logs gathered into a single context window. With Fern, its first language model, the Mirendil laboratory is exploring an approach to artificial intelligence in which increasing the amount of information accessible to the model matters as much as its size. Its architecture supports up to 10 million tokens—about 23 MB of code or half a million lines—without having to split this information into separate conversations.

Trained from scratch, the model received a relatively short initial post-training phase under the name Fern 0.1. It is intended for agentic tasks involving vast codebases, execution histories, and voluminous technical documents. Mirendil is also presenting two larger models, still currently in training, though none of these systems are yet publicly accessible. Betting on Context Rather Than Model Size Alone

The progress of language models is often presented through their parameter count, their benchmark results, or their ability to reason longer. Mirendil proposes giving comparable importance to another dimension: the amount of information a model can consult simultaneously.

An expanded context window allows for preserving more elements specific to a task. For a software development agent, this can include multiple components of a repository, their dependencies, user instructions, test results, and the various modifications already made. In analysis work, it can also gather execution logs, traces from other agents, or large textual datasets.

Fern was designed to reach 10 million tokens of context, a capacity that Mirendil compares to that required to read a software project the size of Megatron-LM. This window also represents several dozen, or even several hundred complete agent traces, depending on their length.

However, the value of such a capacity depends on the actual use of the information. A model can accept several million tokens without successfully retrieving a detail correctly, establishing relationships between distant elements, or leveraging all of the content with the same precision. Predictions That Continue to Improve Across Very Long Distances

To examine whether Fern actually utilizes this capacity, researchers measured its code prediction performance on a series of software repositories. The protocol consists of progressively presenting more context to the model, then measuring the accuracy with which it predicts the remainder of the text.

The results are expressed in bits per byte, a metric that allows for comparing models using different tokenizers. The lower the value, the better the prediction.

In the published evaluations, Fern's performance continues to improve as the context increases, up to a window of approximately 10.2 million tokens. Mirendil compares this evolution to that of several models, notably Kimi K3, GLM-5.3, DeepSeek-V4-Pro-Base, Nemotron 3 Ultra Base, Beam, and Inkling. The latter are tested on windows close to one million tokens, corresponding to their announced capacities or neighboring limits.

The comparison here focuses on statistical code prediction, and not on the completion of a full programming task. It indicates that the additional context remains usable for improving local prediction, but is not sufficient to demonstrate that an agent will know how to reason correctly over ten million tokens. Retrieving Information Is Not the Same as Reasoning About It

Mirendil also evaluated its base model with RULER, a benchmark designed to test various long-sequence processing capabilities. The results show a clear difference between information retrieval and tasks that require more reasoning.

In the test consisting of retrieving a single piece of information hidden within a long context, Fern maintains 100% accuracy up to 10 million tokens. It also achieves 100% on a task consisting of retrieving a set of multiple scattered pieces of information.

Other exercises reveal more difficulties. Multi-key to multi-value retrieval reaches 90% at ten million tokens, while variable tracking fluctuates depending on the length of the context. On a multi-step reasoning question-answering test based on HotpotQA, accuracy drops from 65% at 32,000 tokens to 45% at ten million.

These figures are based on series of 100 examples per configuration, reduced to 20 examples for the ten-million-token window. The results at very long lengths must therefore be interpreted taking this sample size into account.

Overall, this shows that Fern can retrieve elements within an extremely broad context, but that leveraging this information to solve complex questions remains more difficult. Memory capacity and reasoning quality do not necessarily progress at the same pace. Still Uneven Results on Coding Benchmarks

The first post-trained version, Fern 0.1, was also tested on several coding agent evaluations. Its performance is encouraging on certain exercises, but remains far from the top-performing models on the market.

On DeepSWE v1.1, Fern 0.1 achieves 44.2%, ahead of Beam at 43.3%, but behind Kimi K3 at 67.3% and Claude Opus 5.5 at 74.2%. On SWE Atlas QnA, the model reaches 53.2%, a result higher than those of Beam, Inkling, and Nemotron 3 Ultra in the table published by Mirendil.

The gaps become larger on other benchmarks. Fern scores only 6.1% on Terminal-Bench 4.0, compared to 57.9% for GPT-6 Astra and 66.4% for Claude Opus 5.5. On SWE-Bench Verified, its score of 73.4% remains below those of several competitors.

These comparisons mix results from model spec sheets, existing leaderboards, and, when these sources are insufficient, evaluations conducted by Mirendil. They do not all correspond to an independent campaign conducted under a single protocol.

Furthermore, Fern 0.1 benefited from only a short post-training phase. Its current positioning is more that of a research model specialized in handling very long contexts than a generalist competitor to the most advanced coding models. A