PyTorch: Crash Fixes and NCCL's Next Chapter
Today's activity centers on closing off crash-causing edge cases across attention, MPS, and Dynamo, while a cluster of NCCL work lays groundwork for a future backend transition.
Duration: PT2M42S
Episode overview
This episode is a short developer briefing from PyTorch.
It explains recent repository work in plain language.
- Show: PyTorch
- Published: 2026-07-28T13:00:18Z
- Audio duration: PT2M42S
Transcript excerpt
This excerpt keeps the crawler page concise. Listen to the episode or use the RSS feed for the full update.
Good day, it's July 28th, 2026, and here's your PyTorch briefing.
The clearest pattern today is defensive hardening — several PRs exist specifically to stop the process from crashing outright, rather than just fixing wrong answers. PR 191325 fixes a SIGFPE in scaled dot-product attention when a head count hits zero — previously an uncatchable crash, now a proper Python-level…
Second theme: distributed is quietly preparing for a backend switch. PR 191272, from Tristan Rice, adds an explicit "nccl-legacy" backend entry point, keeping today's ProcessGroupNCCL as the default while a new "nccl2" path is opt-in. Supporting commits extend that work: one publishes host communicators to symmetric…
Third, performance regressions are getting caught and reversed. PR 191306 fixes a twenty-percent training slowdown on mobilenet caused by Inductor's layout optimizer failing to recognize backward-pass convolution nodes. PR 191269 tackles FlexAttention out-of-memory errors on consumer GPUs like the RTX 4090 by…
Worth noting: two PRs were reverted overnight — the keepdim parameter for cosine similarity, and a Dynamo guard-skipping optimization for constant ops — both pulled after internal issues…
What's…
Nearby episodes from PyTorch
- Compiler Correctness Meets Test Portability
- Weekly Recap - Performance Fixes and Toolchain Modernization
- Toolchain Upgrades and Correctness Fixes Across the Stack
- Native Kernel Push and Distributed Backend Parity
- Correctness Hardening Across Inductor and AOTI
- The NVGEMM Epilogue Fusion Marathon
- Distributed Correctness Push
- CUDA Graphs Get a Lifecycle, Distributed Gets Cleaned Up