AlphaEvolve: pointing a model at problems that come with an objective scoring function
alphaevolve
Google DeepMind's coding-agent-driven evolutionary search: Gemini Flash proposes candidates in volume and Gemini Pro in quality, automatic evaluators score them, and high scorers stay in the population as next-round context, i.e. evolutionary search with an LLM as the mutation operator; it applies only where an automatic evaluator exists, hence our math and agents filing. An evolved scheduling heuristic has run in production in Google data centres (Borg) for over a year, recovering 0.7% of Google's global compute (Google's compute, not all compute on earth), our card reading and the only result validated by long-running production. An evolved matmul kernel is 23% faster at specific sizes, and about 20% of 50+ open maths problems improved, including 4x4 complex matrix multiplication in 48 multiplications against Strassen's 1969 record of 49. Verification differs from the prover line: a Lean check is mathematical correctness, "23% faster" is an empirical reading on specific hardware. No public weights, Early Access is a waitlist, and the precondition is an evaluator you write yourself. Graded B (contested): a year of production behind the Borg result, nothing reproduced by us.
- CONFIDENCE
- Contested
- Sources disagree (data leakage, protocol mismatch)
- KEY METRIC
- Google 全球算力回收(生产验证)
- Contested · 2025-05
- MATURITY
- Production
- research → demo → product → production
Our takeWe grade it B (contested / partly independently verifiable), and the grade needs explaining. B is not because the results are doubtful - quite the opposite. Some of them carry the highest evidence grade of anything we catalogue: the 48-scalar-multiplication construction for 4x4 complex matrix multiplication can be expanded and checked without trusting anyone's evaluation, and the Borg scheduling heuristic has run in production for over a year while recovering 0.7% of Google's worldwide compute, which is long-term production validation rather than a single benchmark. What keeps it at B instead of A is the other side: the system is entirely closed, so nobody outside can test how generally the method applies, Early Access is a sign-up form, and outside teams have not had one chance to try it. "Results credible, method unverified" is exactly what B means here.
Its place in the AGI story matters more than its scores. Most claims that "AI accelerates research" stop at laboratory readings; AlphaEvolve is a working loop - model proposes, evaluator scores, evolution keeps - and what comes out went back and optimised the infrastructure that trains those models. A 23% faster matmul kernel buying 1% off Gemini training time is real money at flagship scale, and the direction is self-reinforcing. This loop is the most concrete instance of self-improvement available today, and it deserves more serious study than any speculation about recursive self-improvement.
The methodological limits have to be read clearly, or the results get over-claimed. First, it only works where you can write an automatic evaluator, and writing a cheap, fast, trustworthy scoring function is often harder than solving the problem; anything needing human quality judgement is out of scope. Second, evolutionary search exploits evaluator loopholes without mercy, so a scoring function measured on one distribution can easily be gamed into something useless elsewhere - the Borg result is strong precisely because its score comes from production. Third, the gains are single-digit percentages (0.7% / 1% / 23% / 32.5%); each holds in its own context, but together they do not support "AI made computing 30% faster". Finally, the community's inspired open-source reproductions are not AlphaEvolve, and attaching their level to this asset would be wrong.
The problem it solves: pointing a model at things that have an objective scoring function
Most generative tasks are open-ended: whether a piece of code is good or an image looks right still needs a human to judge. AlphaEvolve deliberately restricts itself to the other class - optimisation problems with an automatic evaluator. How much compute a scheduling heuristic saves, how many scalar multiplications a matrix-multiplication algorithm uses, what the lower bound of a mathematical construction is: all of these can be scored by a machine. Once there is a score, evolution has a direction.
It closes the loop out of three parts: Gemini Flash and Gemini Pro propose candidates (Flash for volume, Pro for quality), an automatic evaluator scores them, and an evolutionary framework keeps the high scorers in the population and feeds them back as context for the next round. It is essentially evolutionary search with an LLM as the mutation operator - the model is not expected to answer correctly once, but to be filtered by the scoring function across thousands of rounds.
This is why we file it under both `math` and `agents` rather than under a model benchmark. What it measures is not "how smart is the model" but "how much verifiable improvement can the system of model plus evaluator plus search produce on real engineering problems". Its most direct link to AGI is self-improvement: a system that can optimise the infrastructure it runs on has a path to progress that does not depend entirely on human researchers.
The heaviest piece of evidence: it has been in production for over a year
Most AlphaEvolve results are laboratory readings. One is not:
- A scheduling heuristic it evolved has run in production inside Google's data centres (Borg) for more than a year, continuously recovering 0.7% of Google's worldwide compute.
Read that 0.7% precisely. It is not a vague "0.7% efficiency gain"; it is 0.7% of machine time continuously recovered on one of the largest, most heavily hand-tuned scheduling systems in the world, over a period of years of prior human optimisation. We chose it as this asset's card reading because it is the only result in the whole set that has been validated by long-running production rather than by a single benchmark run.
Two further engineering readings: an evolved matmul kernel is 23% faster at certain sizes, cutting Gemini training time by about 1%; improvements to FlashAttention reach up to 32.5%. The first matters because it feeds back into training the very class of model AlphaEvolve is built from - the most concrete form infrastructure-level self-improvement takes. One percent sounds small, but on flagship training cost it is real and material.
The mathematics side: one result improved for the first time since 1969
AlphaEvolve was run on more than 50 open mathematical problems: it reproduced the known optimum on roughly 75% and improved on about 20%. The hardest single result is that 4x4 complex matrix multiplication can be done in 48 scalar multiplications, improving on the 49 that Strassen's algorithm has given since 1969.
Why this carries more weight than "improved 20% of problems": Strassen's two-level algorithm is a classical result in numerical linear algebra, the 4x4 complex case has been studied intensely and is hit constantly by real implementations (GPU kernel tile sizes land exactly there), and pushing a 56-year-old record at that size is independently and mechanically checkable - anyone can expand the 48-multiplication construction and verify it without trusting anybody's evaluation. The other frequently cited result is raising the lower bound for the kissing number in 11 dimensions to 593; constructions of that combinatorial-geometric kind are likewise directly checkable.
Why it differs from the prover line
This site also catalogues DeepSeek-Prover-V2 and Kimina-Prover, which produce formal proofs: output checked by the Lean kernel, with strict binary truth. AlphaEvolve takes a different route - the output is a program or a construction, and truth comes from what the evaluator measures. The two are not equally verifiable. A Lean kernel acceptance is correct in the mathematical sense, whereas "the kernel is 23% faster" is an empirical reading tied to specific hardware, sizes and measurement methodology; change the environment and the number moves.
That distinction drives our confidence grade: AlphaEvolve is B (contested / partly independently verifiable) - one step above the provers' C, because the Borg result has a year of production behind it, but short of A, because most readings cannot be reproduced outside Google.
Availability: currently close to zero
- No public model weights: AlphaEvolve depends on internal API access to Gemini Flash and Gemini Pro; it is not a downloadable artefact.
- Early Access is not open: the official page carries a sign-up form only, and at the time of our check there was no publicly available access channel.
- No official open-source reproduction: the community has several inspired implementations (all "LLM plus evaluator plus evolution loop"), but they are not AlphaEvolve; their capability level bears no established relation to it, and we do not equate them with this asset.
- The practical precondition is that you can write the evaluator: this is the real gate. To reproduce the method on your own problem you first need a cheap, fast, trustworthy automatic scoring function; problems with no such function are simply out of scope.
Boundaries
- Only works where quality can be scored automatically: domains needing human judgement (product design, code maintainability, research taste) are outside the capability surface. The evaluator's quality is the ceiling of the result.
- Reward hacking is the evaluator's problem to bear: evolutionary search exploits any loophole in the scoring function without mercy. An evaluator that measures only one input distribution can easily be gamed into a solution that scores well there and is useless elsewhere. The Borg result is credible precisely because its score comes from real production.
- Search cost is very high: thousands of rounds of generation plus evaluation, each round running real benchmark measurements. That cost structure suits "optimise once, reuse for years" infrastructure problems, not per-request work.
- Improvements are mostly single-digit percentages: 0.7%, 1%, 23% (one kernel), 32.5% (one operator). Each number is solid in its own context; do not read them as "AI made computing 30% faster".
- Mathematical gains concentrate on constructions with an explicit objective: matrix multiplication, kissing numbers, that kind of exhaustively checkable combinatorial construction. Theorem proving and conjecture generation are not its arena.
Our verification status
Facts come from Google DeepMind's official AlphaEvolve announcement page and technical blog, read in full (covering Borg production operation, the 0.7% compute recovery, matmul 23% / training time 1%, FlashAttention 32.5%, the 48 scalar multiplications, 50+ open problems at 75%/20%, the 593 kissing-number bound in 11 dimensions, and the Early Access sign-up-only status).
Confidence is graded B (contested), on three layers. First, some results are independently and mechanically verifiable: the 48-multiplication construction and the 593 kissing-number construction can be expanded and checked by anyone without trusting DeepMind. Second, the Borg result has long-running production behind it, not a single benchmark - the highest evidence grade in the whole set. Third, we have reproduced none of it: we did not re-run the 23% matmul kernel on our own hardware and have no access to Google's internal Gemini or evaluator pipeline. The system is also entirely closed, so outside researchers cannot assess how generally the method applies. "Contested" here means precisely "the results are credible; the generality and reproducibility are unverified".
Reaching grade A would require DeepMind to open the system or publish enough detail for outside teams to reproduce comparable gains on their own problems. What we can genuinely do ourselves is independently verify the two mathematical constructions - expand the 48-multiplication construction in a symbolic algebra system and check the 593 construction entry by entry - and that is our next concrete step.