Ollama: Weekly Recap - Speed, Speculation, and Scheduler Reliability
This week's work centered on speeding up the MLX engine through speculative decoding and kernel fusion, hardening the scheduler against stale runners, and cleaning up rough edges across the CLI and streaming APIs. Fifty pull request items and eighteen additional commits moved through the repository, with performance and reliability as the clear throughlines.
Duration: PT3M15S
Episode overview
This episode is a short developer briefing from Ollama.
It explains recent repository work in plain language.
- Show: Ollama
- Published: 2026-08-03T09:08:24Z
- Audio duration: PT3M15S
Transcript excerpt
This excerpt keeps the crawler page concise. Listen to the episode or use the RSS feed for the full update.
Good morning. It's the week of July 27th through August 3rd, and here's your Ollama development recap. Fifty pull request activity items and eighteen additional commits landed this week.
The lead story is performance on the MLX engine. Jesse Gross shipped speculative decoding for Qwen 3.5 in PR 17454, using the model's built-in draft head to speed up generation — a 24 percent boost on the 35 billion parameter model and up to 70 percent on the dense 27 billion model, measured on Apple silicon. That…
The second theme is scheduler reliability. Two related pull requests, 17515 and 17516, target the same class of bug: runners that look loaded but are actually dead. One fixes a deadlock where a concurrent request could resurrect a runner that was already being evicted; the other makes the scheduler actively check…
Third, there's a consistent push on developer-facing API correctness. PR 17485 reworked streaming chat completions to match OpenAI's actual wire format, chunk for chunk. PR 17478 added input token counting routes. And two separate pull requests, 17505 and 17526, finalized incomplete tool calls for the Cogito and…
Smaller but notable: PR 17483 and its follow-ons add a prompt…
Next…
Nearby episodes from Ollama
- Scheduler Reliability Overhaul
- Speculative Decoding and the Cloud Model Nudge
- API Compatibility and Model Correctness Push
- Speculative Decoding Gains and a Lint Lockdown
- Cleaning Up Concurrency and Cutting Experimental Code
- Trust Your Cache, Fix Your Logs
- Parsing Bugs and Process Hardening
- Weekly Recap - New Model Support & Concurrency Hardening