Ollama: Taming Memory in the MLX Runner
Jesse Gross landed a major overhaul of MLX array lifetimes and prefix cache eviction that cuts peak memory on long conversations nearly in half, while separate fixes tightened GGUF metadata handling and cloud proxy timeouts.
Duration: PT2M43S
Episode overview
This episode is a short developer briefing from Ollama.
It explains recent repository work in plain language.
- Show: Ollama
- Published: 2026-09-11T13:01:06Z
- Audio duration: PT2M43S
Transcript excerpt
This excerpt keeps the crawler page concise. Listen to the episode or use the RSS feed for the full update.
Good day. It's September 11th, 2026, and this is Ollama's developer briefing.
The dominant story today is memory discipline in the MLX runner, spread across one PR and several follow-up commits from Jesse Gross. The core fix, PR 18327, replaces a pin-and-sweep memory model, where freeing anything required knowing what every other caller still held, with scoped array lifetimes: memory is freed…
That change enabled a companion fix in PR 18353: the prefix cache used to protect every node on a conversation's own active path from eviction, so a long chat's snapshots never got reclaimed. On models with sliding-window or recurrent layers, that's roughly 800 megabytes per turn, growing without bound. Now only the…
Related commits round this out: one waits for a killed runner to fully exit before the scheduler loads the next model, another bounds new MLX loads by actual system free memory rather than just the Metal working set, echoing a rule llama-server already follows in PR 18345. Together, these close a set of paths that…
A second theme: correctness in shared state. PR 18367 fixed template rendering that mutated caller messages, causing different output on a second render of the same…
Small…
Nearby episodes from Ollama
- Tool-Call Parsing Gets a Hardening Pass
- Codex Integration Hardening and Streaming Reliability
- Tightening Up the OpenAI Compatibility Layer
- Closing the Gaps Between What You Configure and What Actually Runs
- Weekly Recap - MLX Hardens as a First-Class Engine
- Context Handling Gets Serious
- Codex Integration Push and Schema Hardening
- Structured Output Meets Speculative Decoding