What do you like best about Devstral Medium?
The reason Devstral Medium ended up wired into our pipelines is simple: it was the first model where running a full agent loop on every pull request stopped being a budget conversation. At $0.40 per million input tokens and $2.00 per million output, an agent pass that explores the repo, reads the failing tests, and proposes a fix costs cents. We went from running agentic checks on a handful of flagged PRs to running them on every single one, because the arithmetic finally allowed it. That shift, from rationed to routine, is the single biggest thing this model changed for us.
What the model is actually good at is the agent loop itself, and that is a different skill from writing a clean standalone function. Devstral Medium was built by Mistral together with All Hands AI specifically for software engineering agents, and it shows in the behaviors that matter when nobody is watching the run. It navigates a codebase through tool calls instead of hallucinating file contents. It reads a traceback and goes looking for the actual origin of the failure rather than patching the symptom at the surface. It keeps iterating against the test suite until the failure clears or it runs out of turn budget. On SWE-bench Verified it landed at 61.6 percent at launch, which at the time put it ahead of models costing several times more, and in our own harness the fix rate on real tickets tracked that number closely enough that I stopped double-checking the benchmark claim.
Function calling is dependable in the way you need it to be for unattended work. The model emits well-formed tool calls at a rate high enough that our retry logic almost never fires, and it uses the editing tools we expose rather than dumping a diff into prose and hoping someone applies it. This sounds like a small property. For automation that runs at 3am with no human in the loop, it is the property that decides whether the model is usable at all.
Scaffold portability was the pleasant surprise. Mistral tuned this generation to generalize across different prompts and agent frameworks rather than to one blessed harness, and that claim held up in practice. We ran it inside OpenHands first, then moved it to our own thinner scaffold months later, and the behavior carried over without a prompt rewrite. When a model only performs inside the exact harness it was tuned on, you inherit that harness as a permanent dependency. Devstral Medium did not force that trade on us, and it is the reason the later scaffold migration took an afternoon instead of a sprint.
The 128K context window covers most of what a single-service task actually needs in one prompt:
- the service source plus its test files, together
- the relevant migration history when the task touches the schema
- the full CI log for the failing run, untruncated
- enough of the dependency interfaces that the model stops guessing function signatures
Having all of that visible at once is what makes multi-file edits coherent. The model sees how the pieces connect instead of reasoning about each file in isolation, and the patches reflect it.
Latency deserves a mention because it compounds. An agent run is not one call, it is dozens of them chained together, so per-call speed multiplies across the whole loop. Devstral Medium responds fast enough that a typical repair loop over a real ticket finishes in minutes rather than the better part of an hour, and when you are running hundreds of these a week the difference shows up in queue depth, not just in patience. Throughput on the API has been steady too. We batch the overnight jobs without elaborate backoff logic because the service simply keeps up.
There is also a customization path that most models at this price point do not offer. Mistral makes fine-tuning available for Devstral Medium, so a team with a large private codebase and specific conventions can adapt the model rather than fighting it in the prompt. We evaluated it, ran the numbers, and ended up staying on the base model because prompt-level convention notes got us close enough, but having the option on the table shaped that decision instead of the decision being made for us.
Model pinning has been boring in the best sense. We pinned the versioned identifier at integration time and the behavior has stayed put for months, which for a model embedded in CI matters more than a few extra benchmark points would. A model that quietly shifts under a pinned version string breaks pipelines in ways that take days to notice and longer to diagnose. This one has simply kept doing what it did on day one.
It also stays within its lane, and I mean that as praise. It is a text-only coding model, it does not try to be a general assistant, and when the task is code the focus shows in how rarely the output wanders into commentary nobody asked for. Terse, on-task, done. Review collected by and hosted on G2.com.
What do you dislike about Devstral Medium?
The transparent framing first, because it is the most important thing a prospective buyer should know. This is now a previous-generation model twice over. Mistral shipped Devstral 2 in December 2025 with a 256K context window and open weights, and Mistral Medium 3.5 has since taken over as the flagship coding model in the lineup. Devstral Medium remains available over the API and our pinned pipelines keep running on it without issue, but the roadmap attention has clearly moved on, and anyone evaluating today should benchmark the successors before settling here. My own reason for staying is a pinned model with months of stable, characterized behavior in CI, which is worth real money to me. It is not a claim that this is still the best pick on raw merit, because it is not.
Closed weights are the structural limitation of the Medium tier. Devstral Small 1.1 ships under Apache 2.0 and runs on your own hardware, but Medium is API-only, so there is no self-hosted path and no offline option for restricted environments. Our workaround is a split deployment: Small runs locally for development and quick iteration, Medium handles the heavy passes over the API. It works, but it is two configurations to test and maintain where one would do, and the successor generation shipping its large model open-weight rather underlines that this gate was a choice, not a necessity.
Instruction adherence on strict output contracts drifts occasionally. Ask for a rigid JSON envelope around a result, or an exact commit message format, and every so often the model decorates the output with prose or reorders fields. Independent instruction-following scores for this model sit below what its price tier suggests, and that matches what I see in the logs. A validation layer catches everything, and you want one anyway for unattended runs, but this model needed it more than others we have used, and the retry it triggers is wasted spend even at these prices.
Outside of code the ceiling is low. Mathematical reasoning is weak, and general analytical tasks that a frontier generalist handles casually come back shallow. This is by design, it is a specialist, but it means it cannot be the single model behind a mixed workload. We route non-code tasks to a different model and treat Devstral Medium strictly as a dedicated tool, which is one more routing rule in the stack.
One smaller thing: the documentation around the model was written for launch and has not kept pace with the lineup shuffles since. Figuring out where Devstral Medium now sits relative to Devstral 2 and Mistral Medium 3.5, and what the deprecation posture actually is, took me a reading of release announcements rather than a clear lifecycle page. For a model people embed in CI, a stated support window would remove real uncertainty. I settled it by testing that the pinned version still behaves, but I would rather have read it than proven it.
The 128K context, generous when it launched in mid-2025, is now the tier where monorepo work needs a retrieval step in front of the model. For a task scoped to one service it is fine and we never think about it. For a change that spans several packages we pre-select files with a small indexing step before the prompt is assembled, which is extra machinery the 256K successors would let us delete outright. Review collected by and hosted on G2.com.
