PyTorch: The NVGEMM Epilogue Fusion Marathon

Michael Lazos landed a fourteen-plus PR ghstack building fused epilogue support into the vendored NVGEMM kernel, while separate efforts pushed hardware-agnostic testing and profiler cleanups across the codebase. The common thread: reducing kernel launches and making device-specific code paths safer and more portable.

Duration: PT2M33S

Episode overview

This episode is a short developer briefing from PyTorch.

It explains recent repository work in plain language.

  • Show: PyTorch
  • Published: 2026-07-23T13:01:30Z
  • Audio duration: PT2M33S

Transcript excerpt

This excerpt keeps the crawler page concise. Listen to the episode or use the RSS feed for the full update.

Good morning. It's July 23rd, and today's codebase activity is dominated by one massive stacked effort, with two smaller but related threads running alongside it.

The headline is Michael Lazos's NVGEMM epilogue project — a stack of more than a dozen PRs, from 190808 through 190825. The goal: fuse pointwise operations, grouped sums, means, and square reductions directly into the vendored scaled GEMM kernel instead of running them as separate passes. PR 190819 vendors the…

Second theme: hardware portability. Commit e8f97c1 refactors FSDP tests to use capability checks instead of hardcoded CUDA or HPU branches, PR 190833 generalizes sixty-two Dynamo tests the same way, and PR 190846 introduces a capability-based test-skip mechanism for things like FP8 and Flash Attention support.…

Third, profiler and CUPTI work from dolpm and ryanzhang22: PR 190850 adds CUDA-graph dependency arrows to trace exports, PR 190851 cleans up graph-node registries on destruction to avoid leaks, and PR 190787 pulls inductor-specific trace logic out of core profiler code into its own handler — a maintainability win.

Two autorevert entries appeared overnight, on PRs 183731 and 181780, both flagged by PyTorch's…

Wha…

Nearby episodes from PyTorch

  1. Distributed Correctness Push
  2. CUDA Graphs Get a Lifecycle, Distributed Gets Cleaned Up
  3. One Engineer, Many Kernels
  4. Weekly Recap - Cleanup, Correctness, and a Random Number Overhaul
  5. Cleanup Sweeps and a New RNG Foundation
  6. Hardening Memory and Lifetime Management Across GPU Backends
  7. Correctness Fixes and Reverts Take Center Stage
  8. Distributed Backends and Overflow Fixes Take Center Stage