This talk is about separating the task, the job to be done, from the implementation details: the model, harness, tactics, and other elements that are constantly changing.
This separation, the task boundary, is the difference between a brittle program locked to an aging model and a sustainable system that can adopt better innovations when they arrive. It's the key to shipping AI systems that are easier to build, maintain, and improve.
In programming, when you want to create something that will be repeated and reused, you use a function. We believe the same should be true for AI programs.
Functions are awesome. They're reusable, composable, testable, and optimizable. You can reuse them dozens of times. You can improve them without affecting the rest of your program. And you can also use them as building blocks, composing them into bigger programs.
And you can also share functions. They can be packaged up and used by someone else, who only needs to understand the contract, the signature specifying the inputs and outputs. Everything inside can be treated as a black box.
DSPy is an open-source Python framework that brings all of these properties to AI programs.
We're living in an exhilarating era. A new model every other week, new tools, and new strategies: smarter prompting, chain-of-thought, tool use, agents, context engineering, environments, and lately loop engineering.
If you're like us, you want to try all of them. But it's worth pausing occasionally, and asking: will any of these new techniques help you implement your task? These are implementation tactics, tools, not answers. And none of them is guaranteed to help you.
How can we build systems that can quickly try these new ideas and determine if they work?
A very important part of a function is its signature, the well-defined interface.
Perhaps I have an invoice and want to extract its tax values. Or I could have text and want to correct its grammar. Or rewrite a given text for clarity. Or a more ambitious repeated task: given an entire email inbox and a new email, draft a reply.
Anything that is repeated benefits from a clear signature boundary. Notice how large these inputs are (an inbox, an entire repo), which implies retrieval and long-horizon reasoning underneath. Signatures are not just short-to-short transformations.
Here's a signature for my tax task:
def extract_taxes(invoice) -> taxes:This captures what the function does. This is fixed, the task boundary.
Everything below this line, how it gets done, is left completely free to evolve.
Everything below the boundary is replaceable. Prompts, agents, tools, loops, models, and optimizers all can change. 18 months ago we didn't know what GEPA was. Loops only started emerging last Autumn.
But everything above the boundary stays put. Your evals, your integrations, your observability, your software ecosystem, your business...
A signature tells you where a task begins and ends, but it doesn't fully specify the task. Throughout the years, the DSPy community has converged on a simple answer: every task is completely specified by answering three questions:
- What should happen?
- What must happen?
- What does good look like?
The first question is what should happen. This is the signature's job.
Natural language is a remarkably efficient way to communicate intent. It's easier to explain the rules of a game to someone than to have them learn through trial and error. With AI it's no different: it's easier to describe the behavior we expect from an AI system.
The second question is what must happen. The constraints that have to be enforced, not merely requested. For this, we enforce constraints using deterministic code, not stochastic LLMs.
In the example, self.extract runs a plain Predict while self.recheck runs ChainOfThought over the same signature: one vanilla pass, one with more reasoning. Both LLM calls.
The forward method, however, encodes hard requirements: if the first pass returns no taxes, rerun with the reasoning version (you have to get taxes right); if any value comes back negative, raise an error and route it to a human.
These guarantees don't change even if the model gets dramatically better (we just might make fewer 2nd passes). Business logic, safety checks, routing, tool use, and output schemas belong in code, because code is where "must" is actually enforced.
The third question is what does good look like. Some knowledge is easier to demonstrate than to explain.
My dad can identify every tree species in a forest, but he can't fully explain how. He learned from examples, and that's how he taught me. Examples express what words cannot. Evals capture the long tail of behavior, through examples and metrics.
With specs, code, and evals in hand, our task is fully specified, ready to hand it to an optimizer, like GEPA, instead of hand-engineering.
Specs, code, and evals aren't just an abstract academic theory. Some of the largest enterprises use them in production for major gains.
Two benefits show up again and again. First, the implementation gets cheaper: when you stay flexible about how the task is solved, you can try many implementations underneath to search across solutions and find a cheap one that works.
That flexibility also lets you scale to data sizes an expensive implementation could never touch. Shopify made a task 550× cheaper simply by moving from an expensive model to a cheap one, while keeping the same evals and continuing to iterate on the implementation inside.
The case studies on this slide are worth reading in full:
- Dropbox: Optimized Dash's relevance judge using DSPy across ranking, training data generation, and offline evaluation.
- Shopify: Structured metadata extraction across all Shopify shops with DSPy and GEPA; reduced yearly costs ~550×.
- Databricks: Research, products, and customer solutions around LM judges, RAG, classification, and other applications.
Searching through different implementations is one reason to build with DSPy and define clean task boundaries. Another benefit is that you can quickly try new techniques.
When a new model comes out, how hard is it to convince yourself and your colleagues that it's better, let alone to invest the cycles in trying it? What about new techniques, like Recursive Language Models (RLMs)?
Alex Zhang first wrote about RLMs in October, and by January DSPy had shipped a Module implementing the technique. With a single line change (tweaking dspy.ReAct(...) to dspy.RLM(...)), you could try the RLMs with your task.
When teams want to try new models, it often requires them tweaking their prompts to get a true comparison. After adopting DSPy, Dropbox found they could trial new models in a day, instead of weeks, allowing them to iterate faster towards an optimal implementation.
Just in the last year the community has contributed RLMs, GEPA, BetterTogether, and multi-module GRPO...and we aren't slowing down.
For example: this month we'll be releasing Flex, a new module that applies some of the techniques detailed in Yoonho Lee's MetaHarness paper. With dspy.Flex, you write only the signature; the model writes the code underneath. This could involve combining multiple sequential RLM calls, running regex, or doing validation and cleaning. The code of your module, along with the prompt, are exposed to GEPA during optimization.
Because the model can effectively write DSPy, Flex benefits from DSPy's abstractions and remains extensible.
One of the hardest problems in AI engineering is building evals. Converting reality to a dataset and metrics is lossy and slow.
Defining "good" is hard for any real-world problem. Reducing a complex environment to binary evaluations, categories, or even rubrics throws away detail from the real world. Any offline dataset is a proxy for reality. But, as the capabilities of models improve, their ability to discover what matters for your problem becomes sufficient. In other words, without an explicit reward function, online feedback (traces, user corrections, comments, outcomes, product analytics, conversations) can be used by a model to find the hill to climb, then climb it.
Learning without a specified hill is an active research area. We are working to deliver this pattern as a new kind of optimizer in DSPy, one that does not require an explicit metric, learns from unstructured natural language feedback, and discovers better solutions. We're excited to share more as we progress.
Driving this work are several fronts of research, all focusing on the "last-mile" problems in AI engineering, including:
- Machine Studying, Li et al.
- OBLIQ-Bench, Tchuindjo et al.
- Recursive Language Models, Zhang et al.
- Pedagogical RL, Ziems et al.
Even as smarter and smarter models arrive, the last-mile challenges don't disappear. The smartest foundation model still won't know your business problems and your context. Intelligence isn't the same as all-knowing.
The beauty of DSPy is that these researchers are a part of our community, and the problems that we learn from DSPy users in production directly inform tomorrow's research areas.
Since 2022, DSPy has defined AI functions with specs, code, and evals. The framework has evolved with the field, from evolving few-shots to rewriting prompts to evolving harnesses and, soon, to evolving evals themselves. Throughout this evolution, the interface has stayed stable throughout.
And there will be more innovations this year, or even this week. With stable system definitions you can try out new innovations and iterate faster. Specs, code, and evals hold new implementations accountable. So your task grows with the ecosystem, leveraging the intelligence of the community.
If you want to build reliable AI software, join us! DSPy is completely open source and fueled by open research. Visit dspy.ai today and try the library. Join the community by joining the Discord. And come contribute on Github.
Or, innovate in the open! As you develop new models and techniques, we'll help bring them to DSPy, so people can try them with their tasks.