What Is OpenAI's Improved COBOL and Fortran Support? MultiLangBench and COBOL Migration Results

Screenshot of the official OpenAI — MultiLangBench page — OpenAI — MultiLangBench · SourceCaptured: October 20, 2026

Screenshots show the official pages on the capture date and may differ from the pages at publication. Select an image to enlarge it.

On October 19, 2026 (9 p.m. the same day, Japan time), OpenAI published a post titled "Advancing coding in low-resource programming languages." It says its latest AI models have become much better at writing programs in, and converting programs out of, long-established programming languages such as COBOL, Fortran and Visual Basic. Alongside the post, the company released a new evaluation framework, MultiLangBench, on GitHub.

The figures in this article are those that matched across several news reports on OpenAI's announcement, along with the description of the benchmark published on GitHub. All of the scores cited were measured by OpenAI itself.

What "low-resource languages" are

AI models learn to program by studying large amounts of code and explanations, much of it found on the internet. Widely used languages such as Python and JavaScript have plenty of examples to learn from, but languages such as COBOL and Fortran have far fewer public programs and documents, and AI has long struggled with them. Languages like these, with little material available for training, are called "low-resource languages."

COBOL has mainly been used for business data processing, and Fortran mainly for scientific and engineering calculations. According to reports, OpenAI's post notes that COBOL underpins more than 40% of online banking, 80% of in-person card payments and 95% of ATM transactions, and explains that Fortran is used for weather forecasting and nuclear reactor safety analysis.

The new benchmark, MultiLangBench

MultiLangBench has AI models build SQLite, a piece of database software, from scratch in 20 languages. SQLite is a small, widely used database engine.

According to the description published on GitHub, it works as follows.

The model is given only the documentation for SQLite 3.40.1 and asked to write a program that behaves the same way, in a specified language
The model has no internet access, and SQLite is removed from the environment beforehand
About 24,000 tests built from SQLite's own test suite check whether the program gives the same results as the real SQLite (272 of them are used by default)
Writing in a language other than the one specified, or mixing several languages, counts as a failure

The 20 languages are C, C#, C++, COBOL, Common Lisp, Fortran, Go, Java, JavaScript, Lean, OCaml, PHP, PowerShell, Python, Ruby, Rust, Scala, Swift, Visual Basic and Zig. The tests and environment are released under the Apache License 2.0.

Scores published by OpenAI

According to reports, the main results OpenAI published are as follows. The figures are the share of tests passed.

GPT-6 Astra: 87.4% on average across the 20 languages, and above 83% in 19 of the 20
GPT-6 Luna: 40.3% on average, at a cost of about $0.77 per run
GPT-5.6 Sol: 26.8% on average, at a cost of $3.48 per run

Looking at individual languages, GPT-6 Astra's scores reportedly rose as follows compared with GPT-5.6 Sol.

COBOL: from 10.9% to 72.1%
Fortran: from 21.4% to 83.5%
OCaml: from 25.6% to 85.8%
Visual Basic: from 27.5% to 89.4%

Even for GPT-6 Astra, COBOL was the lowest-scoring of the 20 languages.

Evaluating the move of COBOL programs to other languages

OpenAI also shared results from a second evaluation, a "COBOL migration evaluation." It has models rewrite programs written in COBOL into C++, Java, Kotlin, Python or Rust, and checks whether the output is byte-for-byte identical to that of the original program. It measures whether behavior can be preserved during migration, the work of moving old systems to newer languages.

According to reports, the results are as follows.

GPT-6.1 Sol (low reasoning setting): 86.6%
GPT-6.1 Sol (high reasoning setting): 97.9%
GPT-5.6 Sol (low reasoning setting): 5.4%

As far as CoAI's editorial team could confirm, it is not known how many programs this evaluation used, what they contained, or whether the evaluation framework will be released.

Where it's available, and caveats

Reports say the improvements are reflected in ChatGPT as well as in the latest models available through the API. This is not a new product launch but an announcement that existing models have become more capable. As far as we could confirm, there is no specific guidance on how this applies to Codex or on Japanese-language information.

OpenAI cautions that real-world system migrations still require review by experts and testing tailored to each system's business rules and its connections to other systems.

What it means for Japan's legacy systems

In Japan, too, dealing with old core systems (the systems that support a company's central operations) has long been a challenge. In its 2018 DX Report, the Ministry of Economy, Trade and Industry (METI) warned of the "2025 Digital Cliff," the problem of old systems holding back DX (transformation through digital technology). In a summary report from its Legacy System Modernization Committee, compiled in May 2025, METI noted that 61% of companies that responded to its survey still have old systems (legacy systems), and that the pool of IT talent keeps shrinking as experienced engineers leave.

Many old systems running on mainframes (large business computers) are written in COBOL. If AI can read and write such languages correctly, it could help people understand old programs whose specifications have been lost and help move them to newer languages.

CoAI's take

Until now, evaluations of AI programming have centered on widely used languages such as Python and JavaScript. This announcement from OpenAI is significant because it turns attention to COBOL and Fortran, which still run in banks, government and elsewhere, and because it releases the evaluation framework as well. Making the tests available to anyone makes it easier to compare models from other companies under the same conditions.

On the other hand, the figures were measured by OpenAI itself, and we have not been able to verify the details of the COBOL migration evaluation. In systems that handle money or personal information, even small differences in behavior can lead to serious problems. Even when using AI for migration, it is best to assume, as OpenAI itself says, that expert review and thorough testing are required.