Carbon-X: Hugging Face trains an AI model to understand DNA and explain mutations

Hugging Face releases Carbon-X, an open 14B DNA-language model trained on 300 billion tokens to analyze genetic sequences and mutation effects.

A language model can explain the function of a gene using scientific literature without necessarily being able to interpret its underlying DNA sequence. Bridging that gap between biological knowledge and genomic data is the purpose of Carbon-X, an open model developed by Hugging Face and its scientific partners. Based on Qwen3-14B, the system underwent additional training on 300 billion tokens combining genetic sequences, biology literature, and general English. Researchers can submit a sequence, ask about the potential effects of a mutation, or request DNA sequence completion. Across five specialized evaluations, Carbon-X achieved an average score of 66.7%, compared with 67.5% for GPT-6.1 Sol, at an estimated inference cost approximately 1,800 times lower. Bringing DNA and Natural Language Into One Model

Genomic foundation models learn patterns in DNA sequences, including structures associated with gene regulation and genetic variation. They can estimate sequence properties but do not necessarily have the conversational abilities required to explain their predictions. General-purpose language models, meanwhile, acquire considerable biological knowledge from scientific publications while receiving relatively little training on raw genomic sequences.

The team behind Carbon-X sought to combine these capabilities within a single architecture. The model retains Qwen3-14B's general design while learning to process DNA and natural language in a shared token stream. Researchers can include a genetic sequence in an English-language question and receive an answer connecting the sequence to a written explanation.

The approach builds on Hugging Face's earlier Carbon-3B genomic model and previous research combining language and biology. Some earlier systems used a separate DNA encoder connected to a language model responsible for generating explanations. Carbon-X instead incorporates both functions within a single model.

The project was developed jointly by researchers from Hugging Face, Zhongguancun Academy, and the Telethon Institute of Genetics and Medicine, affiliated with the University of Naples Federico II. A Vocabulary Designed for Genetic Sequences

DNA is composed of four nucleotide bases, A, C, G, and T. General-purpose language-model tokenizers are not specifically optimized for these sequences. They may break genetic strings into irregular fragments, using more tokens than necessary to represent their content.

Carbon-X uses a hybrid tokenizer. Ordinary language continues to rely on Qwen3's byte-pair encoding, while DNA is represented in groups of six bases, known as 6-mers. The researchers extended the vocabulary with 4,096 six-base combinations and additional tokens marking the boundaries of genetic sequences.

This representation compresses DNA more efficiently. According to the team, genomic sequences require approximately one-third as many tokens as they would using Qwen3's original tokenizer. It also gives the model consistent units from which to learn genetic patterns while retaining its ability to communicate in natural language.

Training required balancing those two domains. Experiments with smaller models showed that learning DNA too aggressively could damage existing English-language capabilities. The researchers adjusted learning rates and extended training on Qwen3-14B using a mixture of genomic sequences, biology texts, and general English. Why Learning DNA Before Reinforcement Learning Matters

The team then applied supervised fine-tuning and reinforcement learning to improve how the model answered biological questions. Early experiments revealed an important problem: better benchmark scores did not necessarily produce trustworthy explanations. Some responses invented database references or incorrectly identified genes.

To address this, the researchers worked with domain experts to develop deterministic checks that could verify selected claims against the supplied DNA sequences. Those checks supported the creation of more grounded training responses, followed by reinforcement learning using rewards tied to verifiable outcomes.

The experiments also highlighted the importance of DNA-specific intermediate training, or midtraining. On FrontierVEP, a benchmark focused on the effects of genetic variants, reinforcement learning applied directly to Qwen3-14B failed to exceed the 50% random-chance baseline during the reported training runs. When the same approach started from a checkpoint already trained on genomic sequences, performance improved beyond chance.

The result suggests that additional reasoning training cannot necessarily compensate for missing representations of the underlying data. The model first needs to learn how DNA is structured before general reasoning capabilities can be applied effectively. Near GPT-6.1 Sol Across Five DNA Benchmarks

The researchers compared Carbon-X with general-purpose models on five benchmarks covering sequence understanding, mutation effects, and nucleotide prediction. With extended reasoning disabled, Carbon-X scored 66.7% on average, compared with 40% for its Qwen3-14B starting point. GPT-6.1 Sol reached 67.5%, while Claude Opus 5.5 scored 60.9% under the selected evaluation configurations.

Performance varies significantly by task. On BioReason Coding, which evaluates the effects of variants in protein-coding regions, Carbon-X scored 74.9%, ahead of GPT-6.1 Sol's 71%. On NextBase, which tests prediction of subsequent DNA bases, it achieved 39.4%, compared with 32.9% for GPT-6.1 Sol.

Frontier general-purpose models retained an advantage elsewhere. On BioReason Non-SNV, covering genetic variations more complex than single-base substitutions, Carbon-X scored 85.1%, compared with 91.7% for GPT-6.1 Sol and 94.8% for Claude Opus 5.5. On FrontierVEP, Carbon-X reached 70.4%, below GPT-6.1 Sol's 74.9%.

These differences matter when interpreting the overall result. The five benchmarks evaluate different abilities using different metrics. An average score of 66.7%