← Blog

Guide17 min read27 sources

How to check a Lean proof that comes with a paper: build, sorry, axioms, and what stays open

A paper says its proof is checked in Lean. How to rebuild it, find sorry and extra axioms, re-check it, read the statement, and what Paperlane's builds show.

Share

To check a Lean proof that comes with a paper, pin the exact commit and Lean version, rebuild the project from a clean copy on an isolated machine, look for sorry, print the axioms of the main theorem, re-check the compiled proofs with a second checker, and read the Lean statement and its definitions against the paper's theorem. The last step is reading, and no tool does it. On 16 November 2026 the Proofs and Prompts newsfeed reported a free tool, Paperlane, that runs the build step for every new arXiv mathematics paper that links a Lean project on GitHub, and publishes the result with a count of sorry warnings.3

What none of these checks establishes is that the Lean theorem is the paper's theorem. Paperlane says so itself: a passing build "shows the project compiles. It does not show that the Lean statements match the theorems in the paper."2 No tool checks the paper's prose, and no check here says whether a result is new, important or peer reviewed. The gap matters more as AI-written mathematics arrives with Lean code: according to the mathematician Kevin Buzzard, around 45% of the 722 papers OpenAI released in October came with some.24

Below: how Paperlane's builds run, key terms, the checks with commands from Lean's documentation, what each leaves open, cases where a compiled proof and a paper's theorem came apart, what Paperlane's counts say, and what to record. Every source was checked on 17 November 2026.

What does Paperlane check, and what do its Lean tags mean?#

Paperlane, developed by Pragyaan Gaur, lists each weekday's new arXiv mathematics papers in one searchable issue. It is free and needs no account.2 The newsfeed's summary of its Lean feature: it "clones every Lean project linked by a new upload and runs it on a clean machine."3

The code on GitHub shows how. A paper enters the check when its abstract or arXiv comments link a GitHub repository that contains a lean-toolchain file or a lakefile.7 A job on a GitHub-hosted Ubuntu 24.04 machine makes a shallow clone of the repository's default branch, records the commit, installs the Lean version that lean-toolchain names, downloads Mathlib's compiled files when the project uses Mathlib, and runs lake build. It then counts the declaration uses 'sorry' warnings in the build log.5 The job gets no token and no secrets, because, in the workflow's words, "The build runs the authors' code".6

The first build stops at 90 minutes, and a new project that runs out of time gets one longer build of up to 300 minutes.6 The about page gives an hour,2 and stored results include builds stopped at 45 minutes,8 so the limit has changed over time. A checked repository is normally rebuilt only when a new version of a paper links it, at least seven days after the last check.5

ResultWhat it means, in Paperlane's words or codeRepositories
builds, no sorry"the build passed with no sorry"48
builds with sorry"Lean reported declarations proved with sorry, which means some proofs are missing"7
failslake build stopped with an error5
too largethe build ran past the time limit5
could not be checked"the build machine stopped before the build finished"2
Table 1 Paperlane's Lean build results, from its lean builds page (updated 16 November 2026, 17:18 UTC), about page and data file.

A label, lists its assumed inputs, appears when a repository holds a file named for assumptions, external inputs, axioms or hypotheses.5 The project's page then says: "The build checks the Lean code around those inputs. It does not check the inputs themselves, so the result holds on the condition that they are true."10 The label comes from a file name, so open the file: in one tagged project, Axioms.lean12 holds #print axioms commands, an axiom report rather than a list of assumptions. Papers that mention Lean with no repository found get their own tag: "The paper says it is formalised in Lean, but no Lean repository was found."2

Key terms#

  • Lean: a language in which proofs are written as code and checked by a computer.
  • Kernel: in the reference manual's words, "a small, robust implementation of a type checker for the core type theory".18
  • Mathlib: a library for Lean, maintained by its users, that "contains both programming infrastructure and mathematics".22
  • Lake: "the standard Lean build tool". It fetches dependencies and builds the code.16
  • Toolchain: a specific Lean version. A project names it in its lean-toolchain file, and Elan, Lean's toolchain manager, installs and runs it.17
  • sorry: "a temporary placeholder for an incomplete tactic proof". Lean warns about each use and records it with an axiom called sorryAx.15
  • Axiom: a statement accepted without proof. "Lean automatically tracks the axioms that each proof depends on so that they can be audited." The three standard ones are propext, Classical.choice and Quot.sound.14
  • Statement fidelity: whether the formal statement means what the paper's theorem means. Lean's manual asks readers to distinguish "does the theorem have a valid proof" from "what does the theorem statement mean".13 The Yang–Mills group calls the second question validation.27

How do you check a Lean proof that comes with a paper?#

The Lean reference manual has a page for this, "Validating a Lean Proof". It sets out an escalating sequence of checks and separates honest proof attempts from malicious ones, among which it counts "un-reviewed AI-generated proofs and programs".13 The steps below follow that page, with one step added before it and one after.

Step 6 / 6

  1. Pin the version. Find the repository and the commit the paper cites, and read lean-toolchain and lake-manifest.json. Together they fix the Lean version and every dependency.
  2. Build a clean copy in isolation. On a throwaway machine or container, run lake exe cache get if the project uses Mathlib, then lake build.
  3. Read the warnings. Each declaration uses 'sorry' warning marks a proof left open. Note which files they come from.
  4. Print the axioms of the main theorem. #print axioms should list only propext, Classical.choice and Quot.sound. Anything else is a gap or an assumption.
  5. Re-check with a second checker. Replay the compiled proofs with leanchecker, and on Lean 4.35.0 or later run lake check or lake comparator.
  6. Read the statement against the paper. Find the final theorem, read every definition and hypothesis it uses, and compare them with the theorem in the paper.
Fig. 1 Six checks for a Lean proof that comes with a paper, from pinning the version to reading the statement.

Which commit and Lean version should you check?#

Repositories change after papers are posted, and Paperlane builds whatever the default branch holds when it checks.5 If the paper or its arXiv comments name a commit or tag, check that one. If not, write down the commit you checked and the date.

git clone https://github.com/OWNER/REPO
cd REPO
git checkout COMMIT
cat lean-toolchain

The lean-toolchain file "contains a single line with a valid toolchain identifier", and Elan installs that version for the project.17 The manifest, lake-manifest.json, "tracks the specific versions of other packages that are used in a package", and Lake brings each dependency to that version before building.16 Do not run lake update on a project you are checking: it rewrites the manifest, "downloading and upgrading packages as needed", and you would then build something the authors did not.

By our count from Paperlane's data file, its 65 checked projects with a toolchain file name 20 different Lean versions, from v4.19.0 to v4.36.0-rc2.8 Which of the commands below exist depends on that version.

How do you build it without putting your machine at risk?#

Building a Lean project runs its code. The manual warns that "Lean tactics and other meta-code can perform arbitrary actions when run."13 Buzzard's rule: "do not run random Lean code on your local machine unless you sandbox it first."24 Paperlane builds in a job with "no secrets and no write access" for the same reason.4

lake exe cache get
lake build

For a project that depends on Mathlib, run lake exe cache get first; Mathlib's guide gives it "to save time compiling all of mathlib and its dependencies".23 Then lake build builds the project's default targets.16 If the file holding the main theorem is not among them, build it by name with lake build +Module. The manual says that running it and "observing success without error messages or warnings" gives the assurance of the check marks in Lean's editor: the kernel accepted a proof of the statement.13 Large projects need large machines: Buzzard needed 96 cores and 500 gigabytes of memory to compile Anthropic's 13-million-line Fermat formalization.24

How do you find sorry and extra axioms?#

Lean "will give a warning whenever a proof uses sorry", the tactic reference says.15 Older versions print it as declaration uses 'sorry', newer ones with backticks around the word, and Paperlane's script matches both.5

A warning count is blunt, because some sorries are deliberate. Comparator setups keep them on purpose: the theorems they name "should be complete in the solution, but sorry in the challenge."16 One project in Paperlane's list builds with three sorry warnings, and its page carries the author's note that they "come from Challenge.lean, which the Comparator workflow needs."9 The Yang–Mills group's clean build of OpenAI's submission likewise counted one Challenge.lean sorry among 2,453 warnings.27 And the count says nothing about axioms, because the script looks only for the sorry warning.5

The check that covers both is #print axioms, which "displays all the axioms that a definition transitively relies on."14 To run it without editing the authors' files, save a short file of your own as Check.lean in the project folder, with the project's own module and theorem names:

import Project.Main
#print axioms Project.mainTheorem

Then run lake lean Check.lean, which "Builds the imports of the given file and then runs lean on it".16 The manual's page on validation explains the output:13

  • Only propext, Classical.choice and Quot.sound: these are "standard axioms of Lean's logic, and benign".
  • sorryAx: "this theorem or one of its dependencies uses sorry or is otherwise incomplete".
  • An axiom with _native in its name: "native evaluation is used", through native_decide or bv_decide. The tactic reference warns that native_decide "adds the entire lean compiler to the trusted part".15 On Lean 4.28.0 or earlier the same thing shows up as Lean.trustCompiler, which Lean 4.35.0 removed along with Lean.ofReduceBool.19
  • Any other axiom: "a custom axiom was declared and used, and the theorem is only valid relative to the soundness of these axioms".

How do you re-check the proofs with a second checker?#

A build and an axiom report both trust what Lean stored as it went. The manual lists problems they miss, among them "Meta-programs or tactics intentionally bypassing that state".13 The notes in Physical Superintelligence's (PSI) Aaronson–Ambainis repository show one: with an option that switches the kernel off, lake build accepts a proof of False, and #print axioms reports no axiom for it.25 The remedy is to replay the compiled proofs through a kernel again.

lake env leanchecker --fresh Project.Main

The checker the manual calls lean4checker ships with every Lean toolchain as leanchecker from v4.28.0, and its old repository is archived. With --fresh it replays everything the module rests on into an empty environment. Its README states the limit: "This is not an external verifier, as it uses the Lean kernel itself."20

lake check --paranoid

Lean 4.35.0, released on 12 October 2026, added two commands.19 lake check "builds the default targets of the current project, exports them, replays the export through the kernel, and fails if anything uses an axiom other than the standard ones". The --paranoid flag adds every external checker that ships with Lean: leanchecker-paranoid, lean4lean, nanoda, con-leche and con-ron. Both commands build the project inside a sandbox, which "is available only on Linux".16

The second, lake comparator, judges a proof against a challenge: a trusted file that states the theorems, and a configuration listing the permitted axioms. It "establishes that every named theorem in the solution proves the same statement as the challenge, that the solution uses only permitted axioms, and that it is accepted by Lean's kernel as well as by every configured external kernel."16 The manual calls this the gold standard, meant for "high risk scenarios (proof marketplaces, high-reward proof competitions, unaligned AI)".13 Someone still has to write or read the trusted statement. A project pinned to a Lean version before 4.35.0 lacks both commands, and the separate comparator tool is archived, its README pointing to lake comparator.21

How do you compare the Lean statement with the paper?#

No command performs this step. Where there are doubts about what a theorem means, the manual says "its statement and all referenced definitions must be investigated carefully, in particular with regard to custom notation and type classes."13

In practice: find the headline theorem, which the README, Main.lean or a challenge file usually names. Read its statement, then every definition it uses that is not in Mathlib. List its hypotheses; each must also appear in the paper's theorem or be proved elsewhere, and a hypothesis that can never hold lets any conclusion follow. Then compare with the paper, noting whether the Lean theorem is the main result or an intermediate one.

What does each check establish, and what does it leave open?#

CheckWhat it establishesWhat it leaves open
Paperlane's build tagThe project, at the commit cloned, compiled on a clean machine in time; how many sorry warnings it printedAxioms; which theorem is proved; whether the commit is the paper's
Your own lake build of the cited commitEvery file built was elaborated, and the kernel accepted each declaration as it was addedCustom axioms; code that bypasses the kernel; what the statement means
sorry warningsWhere proofs were left open in the files builtWhether a sorry is deliberate, as in a challenge file; axioms
#print axioms on the main theoremEvery axiom the theorem depends on, including sorryAx, native-evaluation and custom axiomsDeclarations added with the kernel switched off; what the statement means
leanchecker --freshEvery declaration behind the module passes Lean's kernel againA bug in Lean's own kernel; crafted compiled files; what the statement means
lake check --paranoidLean's kernel and five external checkers accept the default targets, in a sandbox, with standard axioms onlyWhether the theorem proved is the paper's
lake comparatorThe named theorems prove exactly the statements of a trusted challenge, with permitted axioms onlyWhether the challenge says what the paper says
Reading the statement and definitionsWhether the formal theorem is the paper's, with which hypotheses and conventionsA human judgment; record who made it
Table 2 What each check establishes and leaves open, from Lean's reference manual and Paperlane's code.

Why doesn't “it compiles” mean the paper's theorem is proved?#

Because the kernel checks a proof of whatever statement it is given. The Yang–Mills group's assessment of OpenAI's construction puts it in two sentences: "Lean kernel acceptance determines whether the proof follows from its formal definitions and permitted axioms. It does not settle whether those definitions express the intended natural language problem."27 Public documents show how the two come apart.

The Lean theorem is another theorem. Buzzard reports that of the 290 OpenAI theorems with Lean code, "around 38 of them have the property that the Lean code does not even state, let alone prove, the main theorem of the paper." He adds that OpenAI never claimed they did. For one paper on Lech's multiplicity conjecture, the headline theorem of the Lean code is Corollary 5.4 of the paper.24 Our Fermat post sets his four questions against Anthropic's two runs.

The checked target is weaker than the proof. After its submission, OpenAI told the Yang–Mills group that its public comparator target "omitted some of the connections to the lattice construction established by its proof." The group based its assessment on a stronger target of its own, ran comparator on macOS without process isolation or an external kernel, and reports "extensive (but not exhaustive) human review of the Lean definitions". It also writes: "We do not claim to fully understand the proof yet."27 More in our post on the assessment.

Definitions follow Lean's conventions. Physical Superintelligence's paper-to-Lean map lists points of the formal language that matter when its statements are read. The first: in Lean, 0⁻¹ = 0 and x / 0 = 0. Where the paper inverts a sum, the project proves the sum positive separately, "so that these inequalities do not hold merely by this convention".26

Assumed inputs ride in as hypotheses. One of that project's final theorems takes a bound from an earlier paper by Agarwal and Ben-David as a hypothesis. The audit note says "It is not proved in this library", and it is not declared as an axiom.25 So #print axioms shows only the standard three, and the theorem still holds only if the bound does. The Piatetski–Shapiro primes project in Paperlane's list does the same openly: "The external analytic and arithmetic inputs remain unproved and uninstantiated. No mathematical axiom was added".11 Paperlane lists it as building with no sorry, and flags its assumed inputs.10 Our Aaronson–Ambainis post reads PSI's main statement against the conjecture.

The paper and the Lean proof are separate texts. The same audit note calls its map between paper and Lean "a guide for the reader; no machine has checked it."25 Buzzard notes that for OpenAI's Navier–Stokes work, on which all his tests have been run, "there are instances where the pdf proof and the Lean proof diverge".24 When a statement rests on a large library of AI-written definitions, reading it is a project of its own, as in Axiom's finite simple groups formalization.

What do Paperlane's counts say about Lean in new papers?#

Less than a headline might suggest. On 17 November the lean builds page, last updated at 17:18 UTC on 16 November, listed 67 repositories: 48 built with no sorry, 7 built with sorry, 5 failed, 5 were too large for the time limit and 2 could not be checked.1

By our count from Paperlane's data files, its 15 stored issues, from 23 October to 16 November 2026, held 7,026 new mathematics submissions, plus revisions and cross-listed papers. Of all these, 69 linked a GitHub repository with a Lean project, 40 of them new submissions. Another 113 matched the tool's pattern for a mention of Lean with no repository found.8 That is a few weeks of uploads, read by a tool that, in its README's words, "is new and changes quickly".4

The numbers have limits beyond their size. Paperlane reads links only from the abstract and the arXiv comments, and opens only GitHub repositories, so Lean code linked from inside a PDF or hosted elsewhere is not counted.7 Each result is one commit on one machine under one time limit, so a failed or timed-out build may say more about the machine than about the proof. The tags record nothing about AI use, and none says a paper is correct. The counts change with each issue.

What should a research record say about a Lean check?#

Whoever publishes or reviews a Lean-checked result can write down, with dates: the repository and the full commit hash checked; the Lean version from lean-toolchain and the Mathlib version from the manifest; the build result, with the machine and the time it took; the full output of #print axioms for the main theorem; which re-checks ran, with the challenge file if there was one; who read the Lean statement against the paper, and what they concluded; and which AI tools were used and for what, as the authors report them.

On Papex, an AI app can prepare a private draft of such a record, which the researcher reviews, signs and decides whether to publish (how AI apps work with Papex). Such a record does not make the proof right. That stays with the kernel, the checkers and the people who read the statement.

Sources#

  1. Paperlane, Lean builds (Pragyaan Gaur; page updated 16 November 2026, 17:18 UTC), checked 17 November 2026 ↩
  2. Paperlane, About: what the tags mean, how it updates, checked 17 November 2026 a b c d
  3. Newsfeed, item of 16 November 2026 on Paperlane (Proofs and Prompts), checked 17 November 2026 a b
  4. pragyaangaur/Paperlane: README (GitHub), checked 17 November 2026 a b
  5. pragyaangaur/Paperlane: paperlane/leancheck.py, the Lean build check (GitHub), checked 17 November 2026 a b c d e f
  6. pragyaangaur/Paperlane: Lean build workflows, lean.yml and lean-long.yml (GitHub), checked 17 November 2026 a b
  7. pragyaangaur/Paperlane: paperlane/signals.py, how Lean projects are found (GitHub), checked 17 November 2026 a b
  8. pragyaangaur/Paperlane: data/lean.json and data/days, stored results and issues (GitHub, commit 5ad0e7f0), checked 17 November 2026 a b c
  9. Paperlane, Lean build page for the Pointwise Ergodic Theorems project, with the author's note, checked 17 November 2026 ↩
  10. Paperlane, Lean build page for the Piatetski–Shapiro primes project, with its assumed inputs, checked 17 November 2026 a b
  11. docs/ASSUMPTIONS.md, Piatetski–Shapiro primes formalization (GitHub, commit ab2c7e6e), checked 17 November 2026 ↩
  12. Axioms.lean (magbenji/discrete-modified-gradient-lean, GitHub, commit c3e4688e), checked 17 November 2026 ↩
  13. Validating a Lean Proof (The Lean Language Reference), checked 17 November 2026 a b c d e f g h
  14. Axioms (The Lean Language Reference), checked 17 November 2026 a b
  15. Tactic Reference: sorry and native_decide (The Lean Language Reference), checked 17 November 2026 a b c
  16. Lake, including lake check and lake comparator (The Lean Language Reference), checked 17 November 2026 a b c d e f g
  17. Managing Toolchains with Elan (The Lean Language Reference), checked 17 November 2026 a b
  18. Elaboration and Compilation: the kernel (The Lean Language Reference), checked 17 November 2026 ↩
  19. Lean 4.35.0 release notes (The Lean Language Reference, 12 October 2026), checked 17 November 2026 a b
  20. leanprover/lean4checker: README (GitHub, archived), checked 17 November 2026 ↩
  21. leanprover/comparator: README (GitHub, archived), checked 17 November 2026 ↩
  22. leanprover-community/mathlib4: README (GitHub), checked 17 November 2026 ↩
  23. Using mathlib4 as a dependency (Mathlib wiki, GitHub), checked 17 November 2026 ↩
  24. Kevin Buzzard, …and the proof is checked in Lean (Xena, 12 November 2026), checked 17 November 2026 a b c d e
  25. Axiom audit, notes/AXIOM_AUDIT.md (physical-superintelligence/aaronson-ambainis, GitHub, commit 2a6ceae), checked 17 November 2026 a b c
  26. Paper to Lean, notes/PAPER_TO_LEAN.md (physical-superintelligence/aaronson-ambainis, GitHub, commit 2a6ceae), checked 17 November 2026 ↩
  27. Validation of the OpenAI Yang–Mills construction: Assessment 1 (Yang–Mills group, 10 November 2026, 35 pages), checked 17 November 2026 a b c d

Share

Keep reading

All posts