In math, what does an AI do in “autoformalization”?

Translates human-language math into a proof language

Reads a human-language proof and judges if it’s right

Translates a proof-language proof into human language

AlphaProof (Google DeepMind’s AI that writes proofs in Lean’s language) practiced on about 80 million problems that an AI had translated from human language into Lean. An AI’s translations may include some that don’t mean the same as the original problems. What does the paper say about such translations?

They are still useful for practice

They were removed from the practice

They were fixed by people before use

An AI translating the problem “for every integer n, n × n is 0 or more” into Lean’s language treated n as “a whole number 0 or above.” If the proof of this translated statement passes in Lean, what can you say?

It proved an easier problem than the original

It proved the same problem as the original

It proved a harder problem than the original

Home › Glossary › AI

What is autoformalization in AI and math?

Also called: Auto-formalization

Term explainer · Checked November 10, 2026 · 13 min read

In one minute

Autoformalization is when an AI automatically translates math written in human language (ordinary sentences and formulas), such as a problem or a proof, into a formal language (one in which the meaning of every symbol is fixed exactly, so a computer can check it line by line).

Proofs are usually written in human language and checked by people reading them, but in a long proof, mistakes can slip through. Written in a formal language, a proof is checked step by step by proof-checking software such as Lean or Isabelle, which won’t accept it if anything is wrong. But translating a big proof by hand is a big job that takes experts years. So researchers have been having large language models (LLMs, text-writing AIs like ChatGPT) do it, and it is also used to train AIs that write proofs.

The trap is that Lean can’t tell when the translation’s meaning is wrong. Then, even if the proof passes, it has solved a different problem.

A common mix-up

“If Lean has checked an AI’s proof, it is certainly right.”

Only partly right. Lean (software that checks proofs) checks the steps of a proof one by one, mechanically, so that part can be trusted. But all Lean checks is whether the statement written in Lean’s language has been proved, not whether that statement means the same as the original problem. Say the problem is “for any integer, multiplying it by itself gives 0 or more,” and an AI translates it as “for any whole number 0 or above, multiplying it by itself gives 0 or more.” The proof of this statement passes in Lean, but it never considers negative numbers such as −3 × (−3). It has only proved a different, much easier problem. Mistranslations turn up even in problem sets translated by people. miniF2F, a problem set often used to test AIs, contained translations that had turned into false claims and so could not be proved, and statements whose assumptions contradicted each other, and researchers at Google DeepMind made a corrected version. A statement with contradicting assumptions is one like “x is greater than 5 and less than 3,” where no number fits the assumptions. Since no number fits, nothing can go against whatever conclusion you write, so the proof passes. To go from “it passed in Lean” to “the original problem is solved,” you need a separate check that the translation is right.

Explained in pictures

This flow chart runs from autoformalization to a checked proof. The left box is a problem in human words, “two even numbers add up to an even number,” and the next box is the statement an AI translated it into for Lean (software that checks proofs); the part grouped by the orange bracket is autoformalization. To the right of a “:” you write what the thing on its left is. (a b : ℕ) means “a and b are whole numbers from 0 up,” and (ha : Even a) is the assumption “a is even,” with ha as the label given to that assumption. After the last “:”, Even (a + b), “a + b is even,” is what has to be shown. The proof, Even.add ha hb, uses a theorem in Mathlib, Lean’s math library, that says even plus even is even, handing it the assumptions ha and hb. A person or an AI writes this proof, and Lean checks it step by step, accepting it if every step follows the rules and rejecting it if not. But Lean checks only the steps of the proof, not whether the translated statement means the same as the original problem.

A figure of one problem translated into Lean in two ways (example made up for this page; only part of each Lean statement is shown). Title: “A wrong translation passes as another problem.” Top box, “Original problem”: “For every integer n, n × n ≥ 0,” and “Includes negatives, like −3 × (−3) = 9.” Arrows lead left and right. Left box, solid navy outline, “Right translation”: (n : ℤ) : 0 ≤ n * n, “ℤ = integers (negatives too),” the tag “Lean: passes ✓,” and “→ Proves the real problem.” Right box, dotted orange outline, “Wrong translation”: (n : ℕ) : 0 ≤ n * n, “ℕ = only 0, 1, 2, …,” the tag “Lean: passes ✓,” and “→ Another problem, no negatives.” The letters ℤ and ℕ stand out in bold orange. Band at the bottom: One letter apart. Lean passes both. — This figure shows one problem translated in two ways into the language of Lean (software that checks proofs). The top box is the original problem, “for every integer n, n × n is 0 or more.” The left box with a solid outline is a correct translation: (n : ℤ) means “n is in ℤ (the integers, negatives included),” and 0 ≤ n * n means “n × n is 0 or more.” The right box with a dotted outline is a wrong one, which makes n a member of ℕ (whole numbers from 0 up). They differ only by this one letter for the kind of number n is. Both proofs pass in Lean, but the right one is a different, much easier problem that never has to deal with negative numbers like −3 × (−3). Even if a proof passes, a wrong translation means the original problem hasn’t been solved.

This figure sets numbers from a 2022 paper side by side. The 150 squares on the left are 150 math competition problems that an AI (Codex) translated into the language of Isabelle (software that checks proofs), each checked by people; the 38 orange squares were fully correct translations (25.3%). The two bars on the right show the share of 244 problems, kept for scoring, that a separate proof-writing AI (Thor) could prove in miniF2F (a set of 488 problems translated into formal languages by people). Before training it was 29.9%, and after training on 3,363 translations (the ones with a valid form out of 3,908 the AI translated) it was 35.2%. The “previous best” of 29.6% is a number an AI from another study had reached earlier. Only about 1 in 4 translations was fully correct, yet they helped with training.

You’re free to use these figures in lessons, slides and posts if you credit Naru News (naru.news). CC BY 4.0.

Think of it like this

Copying a problem from the board, then solving it

Imagine a very strict teacher who checks the answers in your notebook line by line against the rules of math. This teacher never misses a single wrong line. But the teacher looks only at the notebook, never at the problem on the board.

Students copy the problem from the board into their notebooks and then solve it. The copying is autoformalization, and the teacher checking the notebook is Lean. If a student copies “for any integer” on the board as “for any whole number 0 or above,” the teacher will say every answer is right. But what the student solved is a different problem from the one on the board. That is why someone has to compare the copied problem with the board separately.

But the comparison has limits. Copying from the board keeps the same language, while autoformalization rewrites human language into a formal language in which the meaning of every word is fixed in advance. Writing just “number” isn’t enough; you have to decide whether it means integers, whole numbers from 0 up, or numbers with decimals too. Most mistakes come not from careless copying but from choosing the wrong meaning. And Lean doesn’t read with understanding like a teacher; it only checks mechanically that the rules are followed.

Key points

Why translate math into a formal language?

In math, a “theorem” is a statement that has been proved true. Proofs are usually written in human language and formulas, and other mathematicians check them by reading. But human language has vague spots and steps skipped as “obvious,” and in a long proof, mistakes can slip through.

A formal language is one in which the symbols you may use and the rules for combining them are all spelled out exactly. Rewriting math in such a language is called formalization. Lean and Isabelle are tools that have a computer check proofs written in a formal language (they are called proof assistants). They check that each step of a proof follows the rules and reject the proof if any step doesn’t. In Lean, this checking is done by a small core called the kernel. The kernel is like a guard at a checkpoint: every proof passes through it in the end, and any step that breaks the rules is stopped there. Lean also has Mathlib, a large library of math with more than a million lines of definitions and theorems.

A proof checked in a formal language gives almost complete certainty that it is correct. Even very long proofs, such as that of the four color theorem (any map can be colored with four colors so that neighboring countries differ), have been formalized.

It also helps with checking AI proofs. Large language models now solve math problems too, but they can make convincing mistakes (hallucinations). Reading through a long proof an AI wrote in human language is hard work for people. If the AI writes it in a proof language instead, Lean finds the mistakes in the steps.

Why not translate by hand?

It can be done by hand, but it takes great effort. Formalizing a big proof is usually years of painstaking work by specialists. Translating needs both an understanding of the math and knowledge of the formal language’s fine rules.

As a result, math written in formal languages is still tiny compared with math written in human language. A 2022 paper (Wu et al.) notes that the Archive of Formal Proofs, a collection of Isabelle proofs, is about 180 MB, less than 0.18% of the data the AI Codex learned from, and that there are almost no pairs of the same math written in both human and formal language. The 2025 AlphaProof paper (Google DeepMind) also says that the number of problems people have formalized in Lean is many orders of magnitude smaller than the number in human language.

So the idea was that having large language models (LLMs) do the translating might cut the heavy cost of formalization. That is autoformalization.

How accurate are AI translations?

A 2022 paper (Wu et al.) had large language models translate problems from MATH, a collection of problems from American middle and high school math competitions (such as the AMC and AIME), into the language of Isabelle (software that checks proofs). They weren’t specially trained to translate; they were only shown a few model translations. When people checked 150 translations one by one, one of the AIs (Codex) had translated 38 of them fully correctly, or 25.3%.

Most of the mistakes came from a mismatch between the meaning in human language and the definitions in the formal language. For example, the models often couldn’t write a problem’s “greatest possible value” with Isabelle’s set ways of writing it (Greatest or Max), so the translated statement didn’t properly say “greatest.” In a problem saying “f is a linear function,” the AI made up a word Isabelle doesn’t have, linear, as if it already existed, and so failed to translate the condition that f is linear.

Translations have improved since then. In AlphaProof (Google DeepMind, paper in 2025), the AI in charge of translating (built on Gemini) gave one translation per problem for 50 representative International Mathematical Olympiad (IMO) problems, and 60% of them were rated correct. But the problems, languages and ways of checking all differ, so 25.3% and 60% can’t be compared directly.

How are translation mistakes caught?

Lean and Isabelle check that a translated statement follows the formal language’s rules (that its form is valid). But they don’t check whether it means the same as the original problem. Catching mistakes in meaning takes a separate check. The surest is to have someone who knows both languages read it. But people can’t check a great many by hand.

So there are ways to have AIs do the checking. A 2024 study called Lean Workbook (Ying et al.) had an AI translate the Lean statements back into human language and had another AI judge whether each said the same as the original problem. Statements that failed in Lean were dropped, and those judged different were sent to people for fixing. When experts checked problems sampled from the resulting set of about 57,000 that passed both checks, 93.5% were correct translations.

The study also lists a mistake that kept coming up. In Lean, if you write 1/3 without saying what kind of numbers they are, Lean treats it as division of whole numbers and drops the decimal part of the answer, giving 0. To have it work with numbers that can have decimals, you write (1:ℝ)/3 in the formula, which says “this 1 is in ℝ (the real numbers, decimals included).” AIs repeatedly left that out, wrote 1/3 inside a formula, and ended up with a formula different from the original.

Another way is to compare against a statement fixed in advance. In a September 2026 announcement by Anthropic (an AI company) of an AI’s formalization of the proof of Fermat’s Last Theorem, the company says a tool confirmed that the final statement the AI proved matches the statement of this theorem already in Mathlib (a company announcement).

Is a wrong translation still useful?

Yes, as practice problems for training an AI that writes proofs. A 2022 paper (Wu et al.) had an AI (Codex) translate 3,908 practice problems from MATH into Isabelle’s language and used the 3,363 whose form was valid to have a proof-writing AI (Thor) practice searching for proofs again and again. On the test problems of miniF2F (a set of 488 problems from math olympiads and elsewhere, translated into formal languages by people), the share it could prove rose from 29.9% to 35.2%, beating the previous best (29.6%).

Google DeepMind’s AlphaProof (announced in 2024, paper in 2025) took this much further. About 2,500 problems that people had translated into Lean were the starting material for teaching an AI to translate, and that AI turned about 1 million problems in human language into about 80 million Lean problems. It made many different translations of each problem to make it more likely that a correct one was among them. AlphaProof then practiced proving with reinforcement learning (learning by being rewarded when it does well). The paper says every translated statement, whether or not it is faithful to the original problem, is a proper formal problem it can try to prove (show true) or disprove (show false), and so is useful as practice material. It also says the wrong translations were often settled quickly during practice, and some even turned out to be interesting problems in their own right.

But to say the original problem has been solved, the translation has to be right. A translation mistake matters much less in a practice problem than in a problem you actually want answered.

Check yourself

Q1In math, what does an AI do in “autoformalization”?
Answer and explanation
Answer: C Translates human-language math into a proof language
Autoformalization is when an AI translates math written in human language (ordinary sentences and formulas), such as a problem or a proof, into a proof language that a computer can check, like Lean or Isabelle. What checks whether a proof is right is a tool such as Lean, after the translation, not the AI that translated it. Translating a proof language into human language is translation in the opposite direction. It is sometimes used to catch translation mistakes.
More on this → In one minute
Q2AlphaProof (Google DeepMind’s AI that writes proofs in Lean’s language) practiced on about 80 million problems that an AI had translated from human language into Lean. An AI’s translations may include some that don’t mean the same as the original problems. What does the paper say about such translations?
Answer and explanation
Answer: A They are still useful for practice
The paper says every translated statement, whether or not it is faithful to the original problem, is a proper formal problem it can try to prove (show true) or disprove (show false), and so is useful as practice material. That is because a proof about such a statement is still a correct proof, even if it differs from the original problem. People did not fix all of some 80 million translations. But to say the original problem has been solved, the translation must be right.
More on this → Key points: Is a wrong translation still useful?
Q3An AI translating the problem “for every integer n, n × n is 0 or more” into Lean’s language treated n as “a whole number 0 or above.” If the proof of this translated statement passes in Lean, what can you say?
Answer and explanation
Answer: B It proved an easier problem than the original
With only whole numbers from 0 up, negative numbers don’t need to be considered, so it becomes a different problem, much easier than the original. Cases such as −3 × (−3) = 9 were never checked. Lean checks that the proof of the translated statement is correct, but not whether that statement means the same as the original problem. Only the range of numbers got narrower, so the problem did not become harder than the original either. Even if it passes in Lean, whether the translation is right has to be checked separately.
More on this → Explained in pictures

Large language model
Hallucination
Reasoning model
Reinforcement learning
Machine learning
Lean
Proof assistant
Formal verification
Formal proof

Sources and how we checked

AI wrote this page from the sources above, and a different AI checked it against the text of those sources (November 10, 2026). A person with a master’s degree in science and a Japanese high-school science teaching license also read and checked it before publishing. Numbers are as of that date. If anything was hard to follow, you spot a mistake, or there’s a term you’d like explained, please write to [email protected].