Google DeepMind's Benoit Schillings argues that superhuman syntax generation has arrived — "code is over, but there's plenty to do." With roughly 80% of new GitHub code now machine-generated, human training data is running out, so DeepMind turns to AlphaZero-style self-play to keep pushing. As writing code becomes essentially free, the real work moves to design, security guardrails, correct-by-construction code, richer evals, and languages built for models rather than humans.
Schillings opened by calling himself "a bit of a noob when it comes to machine learning" — until a year and a half ago he was at Google X, working on projects like Waymo and Glass. His team's mandate at DeepMind is deliberately scoped: "develop whatever technology will be needed to make Gemini incredible between one month and one year from now" — near enough to matter, far enough that no one can really predict it. Much of that work centers on code, but also spans the evolution of reasoning, network topology research, and the fundamental science of reinforcement learning.
The origin story goes back to a 2018 Google X project named Pitchfork, aimed at using ML to improve how code is written. In 2018 "nobody would give us the time of day" — the prevailing question was "why would you ever need ML to write code?" The team both underestimated how fast it would move and dismissed early signals: when people talked about vibe coding, Schillings' reaction was "that's why we have programming languages, English is not a programming language." He now concedes he was "pretty wrong on that front."
Drawing on 45 years of writing code — starting in assembly on the Apple II and Commodore 64 — Schillings mapped three eras. First, the machine era, where the limit was hardware and you fought "to extract the last ounce of power." Second, the cloud/modularity era, where compute got cheap and the real constraint became the human brain: a person holds only seven-to-nine rich tokens of context, and that limit shaped how libraries, functions, and modular decomposition were designed. Modern ML context is "basically going to be infinite pretty soon," so that constraint is dissolving.
That brings the AI frontier, where "writing the code is not the challenge anymore." He declared superhuman syntax generation already done — he no longer looks at a Gemini-written function and thinks he could do better. But he was careful to distinguish syntax from engineering: "software engineering is not about writing code." The hard part is multi-step work across huge codebases (his example: joining a company and finding 35 million lines of PHP you must change) and architecture — the Jeff Dean-level thinking whose implications reach into hardware, security, and living with a system ten years later. "So code is over, but there's plenty to do."
Code was a uniquely tractable ML target for two reasons: an enormous supply of training data (you could scrape GitHub) and cheap verification (you can compile it, run it, write unit tests). That combination brought models to where they are today — but the data well is drying up. Schillings estimates that ~80% of new code added to GitHub today is machine-generated, so "human bringing some knowledge that can be used for mining" is reaching an end.
The answer is self-play, long a DeepMind favorite. He invoked AlphaZero, which became a superhuman Go and chess player "without any human knowledge, just by playing against itself," and argued frontier code models are reaching the same stage: they can create their own challenges, judge the validity of answers, and even judge architecture. His thought experiment: lock a brilliant engineer in a room for two years with pizza and the mission to get better — they invent verifiable challenges and grind on them. Compute and self-play time now set the horizon for how far superhuman coding can go.
"We're now in a world where writing code is free or nearly free," which means the volume of code produced will explode. That creates hard implications: how do you keep sprawling, sometimes dynamically-written systems reliable at the microscopic level, especially when — he predicts within a year — models generate code that "nobody will actually look at," much as few people inspect a compiler's assembly output. His priority list starts with active guardrails and security: vulnerability discovery is a never-ending cycle (models get smarter, find subtler bugs), so the grail his team is chasing is teaching models to write correct code from the start rather than patching after the fact.
He also called for inductive architecture — models are still weak at transferring knowledge across domains and at correct problem decomposition — and for better evaluation, singling out benchmarks like SWE-bench that only verify whether code runs and produces the right output. He favors open-ended evals such as lossless text compression (loss = compressed size plus source size), which force novel algorithmic invention. Longer term, he questioned thinking as a chain of tokens, arguing coding is a visual, spatial activity that a multimodal model like Gemini should represent natively, and floated designing a new language for models — strongly typed, Lean-inspired, correct by design, and not necessarily human-readable. Finally, he pointed "beyond code" to science, chemistry, and biology, where fast, free experimentation could surface "the gold we cannot see."
So code is over, but there's plenty to do. There's plenty of progress to be made.
This is over and we're switching now to that AI frontier where really writing the code is not the challenge anymore.
I think that 80% of the new code added to GitHub today is machine generated.
We're now in a world where writing code is free or nearly free.
| Time | Topic |
|---|---|
| 00:00 | From Google X to DeepMind; the one-month-to-one-year Gemini mandate |
| 01:34 | Origin story: Project Pitchfork (2018) and resistance to ML-written code |
| 04:10 | Three eras of software and the human-brain context bottleneck |
| 06:18 | Where we are: superhuman syntax, multi-step engineering, architecture |
| 08:23 | Code's uniqueness, running out of data, and AlphaZero-style self-play |
| 11:01 | Economics of near-free code: guardrails, security, correct-by-construction |
| 13:37 | Better evals, thinking beyond tokens, new languages, and beyond code into science |