FA3: deterministic SWA backward scheduling
Shorten the dQ semaphore dependency chain inside the fused kernel.
Problem & contribution
Long-sequence sliding-window attention regressed in deterministic backward mode. The goal was lower backward latency without losing bitwise determinism.
Owned diagnosis, scheduling changes and validation. Nsys / NCU isolated the dQ semaphore dependency chain; contributor-relative tickets were implemented in the existing fused kernel and aligned with the reverse scheduler.
Validation
| Condition / metric | Result |
|---|---|
| Full backward, representative packed shape | 6.755 ms → 1.699 ms · 3.98× |
| Tested 2K–12K sequence range | 1.59–5.37× |
| Regression coverage | 3,601 |
| Repeated bitwise-determinism checks | 1,000 |
Delivered as a wheel.
Source: the author’s September 2026 resume and engineering summary; results apply to the stated test conditions.
Speedups describe the stated SWA backward workloads, not a full training step or the final IQuest-Q1 model end to end. This is internship engineering work, not a separately counted upstream PR.