OE: asynchronous GPU state operators
Keep token history on the GPU and reduce synchronization with Triton fused hashing.
Problem & contribution
Token-history construction added synchronization and CPU round trips. Asynchronous execution also required correct state across recovery, slot reuse and decode steps.
Kept history state on the GPU, built inputs with Triton fused hashing, and fixed cross-step recovery and slot-reuse behavior. Validated the TP1 fused path separately from TP2 / TP4 safe paths.
Validation
| Condition / metric | Result |
|---|---|
| TP1 validation and throughput | 28 / 28 · +4.99% |
| TP2 safe path | 84 / 84 · 0 mismatch · +4.6% |
| TP4 safe path | 84 / 84 · 0 mismatch · +3.0% |
Established correctness and performance acceptance by TP configuration.
Source: the author’s September 2026 resume and engineering summary; results apply to the stated test conditions.
TP1 and TP2 / TP4 do not use the same fused path. Multi-GPU results describe the validated safe path, not universal fused-hash support.