Maxime Rivest and Isaac Miller make the case for the unreasonable effectiveness of separating the task from the model, its harness, and implementation details — treating repeated AI tasks like reusable, composable, optimizable functions. A fully specified task needs three things: specs (what should happen), code (what must happen), and evals (what good looks like), after which DSPy can automatically optimize and swap models, prompts, and harnesses underneath a fixed interface.
Maxime Rivest opens on behalf of the DSPy community with what he calls the unreasonable effectiveness of separating the task from the model, its harness, and all of the implementation details. His analogy: in programming, when you want to repeat a task often you make it a function — and functions are reusable, composable, testable, and optimizable. You give a function a name, define inputs and outputs, and put implementation logic inside; anyone can then use it as a black box knowing only its contract.
DSPy, an open-source Python framework, brings these properties to AI workflows and AI programs. The motivation is agility: the field invents new models, techniques, and strategies every other week, and most of them are just implementation tactics. If you wrap a repeated AI task in a clear input/output contract, you get to freely play with the internals while everything outside the boundary stays stable.
Rivest grounds it in personal examples: extracting tax values from farm invoices for his taxes, and a keyboard command that reads his clipboard to correct grammar or rewrite text for clarity. Because the interface is fixed, when a new model comes out he can swap it in easily.
The same boundary scales to ambitious programs — drafting replies over an entire inbox using recursive language models, or agentic engineering and vibe coding where you give a spec and a repository and get a PR. Once the boundary is fixed you focus on the how at the top: iterate on a prompt, turn it into an agent, add tools as they are invented, and add loop engineering inside — all without changing your integration.
Rivest recounts that even before ChatGPT, DSPy's creator landed on the idea that you need three things to fully specify a task. The first is what should happen — instructions, expressed as natural-language signatures independent of the model you set at the top of a script (e.g., extract all taxes, output zero if illegible). The second is what must happen — hard constraints best enforced with code, such as re-running with chain-of-thought reasoning if a vanilla predictor fails, or throwing to a human if a value is below zero. Even with AGI, he wants those requirements to hold.
The third is what good looks like. Rivest tells how his father couldn't give him instructions or code to recognize a maple tree, yet he learned it over time from examples — the long tail of latent, hard-to-articulate behaviors you learn the way a mentee learns from a mentor. With specs, code, and evals together the goal is fully specified, so you can optimize with tools like GEPA. DSPy's optimization evolved from finding few-shot examples, to optimizing instructions, and increasingly to delegating implementation details away.
Isaac Miller takes over to argue these ideas aren't only academic — they're used in production by large enterprises for massive gains. Flexibility on implementation lets you apply the bitter lesson, searching over solutions to find something that solves your problem cheaply and scaling to data sizes an expensive implementation couldn't reach. He cites Shopify running 550 times cheaper by moving from an expensive to a cheap model while keeping the same emails and iterating on business logic, and points to three case studies to review after the talk.
DSPy constantly adds new techniques, but none will definitely solve your problem — that's your job; what DSPy does is solve sub-problems that make implementation easier. Miller cites Alex Zhang's recursive language models paper from MIT, which can be brought into DSPy to try on long-context tasks in one line with the signature unchanged. Other community innovations include GEPA (a prompt optimizer out of Berkeley) and better-together multi-module GRPO — all tryable just by building in the ecosystem.
Miller previews two DSPy 4 features. DSPy.flex is a new kind of module: where optimization started with few-shot examples, then prompts, and now code, DSPy.flex lets you learn a completely custom harness over time to solve any function you define, as long as it solves your business problem and you've defined ways to measure it via specs, code, and evals. Qualitative learning attacks the hard problem of building evals — defining what good looks like is hard, reducing quality to good/bad loses detail, and hand-built datasets are only proxies for reality.
The bet is that models are now good enough to interpret textual feedback in the environment — traces, user actions, product analytics — and convert it into evals and a hill the model can climb and iteratively refine. Miller closes on last-mile learning: even with AGI, a model won't know your context, tasks, or relationships. As he puts it, ask Einstein to help with your emails and he'd ask what an email is; being all-knowing is different from having learned your specific problem. Since 2022 DSPy has focused on specs, code, and evals — and he invites the audience to build reliably, join the Discord, and contribute the next technique back to the open-source, open-research community.
the unreasonable effectiveness of separating the tasks from the model, its harness, and all of the implementation details
in programming, if we want to repeat a tasks often, we make it a function. We believe the same should be true for AI programs.
you need three things to specify your task
even when we have an incredibly smart model, the model won't know how to solve your problems
| Time | Topic |
|---|---|
| 00:54 | AI programs as functions: reusable, composable, optimizable |
| 02:26 | Why fix the boundary — agility against new models and techniques |
| 02:56 | Concrete examples: tax extraction, grammar rewriting, drafting emails |
| 05:30 | Three things to specify a task: specs, code, and evals |
| 09:38 | Isaac Miller: enterprise gains and the Shopify 550x example |
| 12:15 | DSPy 4: DSPy.flex and qualitative learning |
| 14:52 | Last-mile learning, AGI, and open-source contribution |