COBOL, Fortran, Visual Basic: OpenAI is making progress on the languages AI struggles with

OpenAI is improving its models across 20 programming languages and achieves a 97.9% success rate in a COBOL migration test with GPT-6.1 Sol.

From banks to government agencies, a portion of critical IT systems still runs on software written several decades ago. Their maintenance often requires working in languages that are sparingly represented in artificial intelligence model training data, such as COBOL, Fortran, or Visual Basic. In a research publication dated October 19, 2026, OpenAI presents the progress of its models on these environments through two new evaluations: building a database engine in twenty languages and migrating COBOL programs without altering their functionality. On the latter task, GPT-6.1 Sol achieves a 97.9% success rate with high reasoning effort, compared to 5.4% for GPT-5.6 Sol with low effort in the presented protocol. Legacy languages at the heart of critical systems

Artificial intelligence applied to programming has largely developed around languages like Python, JavaScript, or Java, which are abundantly represented in public repositories and accessible documentation. The situation is different for COBOL, Fortran, or certain Visual Basic environments, a large portion of whose code remains embedded in older, sometimes proprietary and sparingly documented enterprise systems. These languages therefore have fewer readily usable learning resources, even as they continue to perform essential functions.

In particular, OpenAI points out the role of COBOL in banking systems and financial transactions, as well as the use of Fortran in weather forecasting or certain analyses related to nuclear safety. The challenge is not simply understanding their syntax. These programs can incorporate business rules accumulated over several decades, whose original specifications have sometimes disappeared. An apparently minor modification can then affect payment processing, the precision of a calculation, or the way an application handles missing data.

The modernization of these systems thus relies on two distinct skills: being able to develop new features in an unfamiliar language and knowing how to transform an existing program without modifying the expected results. OpenAI designed two evaluations corresponding to these needs in order to measure the progress of its models beyond routine code generation exercises. Recreating a SQLite engine in twenty languages

The first evaluation, MultiLangBench, asks models to build a SQLite-compatible database engine from scratch. The goal is not to reproduce a few isolated queries, but to incrementally develop a program capable of managing database files, processing SQL statements, maintaining data state, and handling errors. Each model must structure its code and add features while preserving those that are already working.

To compare implementations written in very different languages, OpenAI uses database files as a common interface. The program receives an initial database and a SQL query, and must then produce a file whose content is benchmarked against the result obtained with SQLite. The tests, adapted from the software's reference suite, make it possible to measure the actual execution of the programs rather than their mere resemblance to correct code.

At the maximum level of reasoning effort, GPT-6 Astra achieves an average test pass rate of 87.4%, compared to 26.8% for GPT-5.6 Sol. Progress spans all twenty languages studied. In COBOL, the score increases from 10.9% to 72.1%, while Fortran advances from 21.4% to 83.5%. Results also reach 85.8% in OCaml, up from 25.6% previously, and 89.4% in Visual Basic, up from 27.5%.

Astra exceeds an 83% test pass rate in nineteen of the twenty languages. COBOL remains the most difficult, despite a strong improvement. This difference illustrates the difficulty of generalizing programming capabilities to environments with their own conventions, constraints, and execution characteristics. GPT-6 Luna improves results at a lower cost

The performance gain is not limited to the most powerful model. With GPT-6 Luna, OpenAI introduces a less expensive option that also outperforms GPT-5.6 Sol across each of MultiLangBench's twenty languages. Its average test pass rate reaches 40.3%, compared to 26.8% for the previous model.

Gains remain more modest than those of Astra, particularly in the most difficult languages. In COBOL, Luna achieves 17.1%; in Fortran, 31.4%; in OCaml, 34.1%; and in Visual Basic, 41.6%. These results show that a general improvement in the model can translate into measurable progress without necessarily being sufficient to produce complex and reliable programs in every environment.

The difference appears primarily in estimated costs. On MultiLangBench, GPT-6 Luna costs an average of $0.77 per sample, compared to $3.48 for GPT-5.6 Sol, representing a reduction of roughly 78%. This is the estimated API cost within the context of the trials, not a flat rate for developing an application or modernizing an enterprise codebase.

OpenAI is publishing the MultiLangBench evaluation setup and its test suite on GitHub so developers can consult results by language and examine the selected tasks. However, the protocol evaluates a specific type of development—rebuilding a SQLite-compatible engine—and does not cover the full set of constraints encountered in enterprise applications. Migrating a COBOL program without altering its behavior

The second evaluation focuses on a different problem: converting an existing COBOL program into C++, Java, Kotlin, Python, or Rust while preserving its exact behavior. This requirement may seem obvious, but translating a legacy program is not merely a matter of replacing statements with their syntactical equivalent in a newer language.

Certain COBOL rules produce effects that can escape a model during a rewrite. OpenAI notably cites fixed-point arithmetic: when an operation produces excess decimal digits, COBOL can truncate them if rounding is not explicitly requested. A translated program that automatically rounds these values may run without errors while altering the results of financial calculations. Conventions for