TensorFlow MUSA Extension
Moore Threads internship: operators, optimizers, GELU fusion, profiling and long-run stability.
Problem & contribution
The GPU backend needed consistent TensorFlow operator semantics, HostMemory placement, resource variables and device-kernel lifetimes.
January–April 2026, Operator and Compiler Optimization Intern. Integrated muDNN, repaired GELU fusion, optimized Logical_Or, and implemented or fixed Adam / AdaMax / GradientDescent paths and tests. Reworked HostMemory handling and fixed resource updates and session shutdown.
Validation
| Condition / metric | Result |
|---|---|
| GELU fusion coverage | 11 / 11 |
| GELU latency on workload-shaped tests | −36.6% |
| Logical_Or latency | 21.2 μs → 10.7 μs |
| Staged stability validation | 500 rounds ≈30% success → 1,000 rounds 100% |
| Long-run validation rounds | 40,000 / 400,000 / 800,000 |
Public PRs provide verifiable internship outputs. They are included once in the internship-inclusive contribution total.
Source: the author’s September 2026 resume and engineering summary; results apply to the stated test conditions.
GELU and Logical_Or numbers are operator-level results, not whole-model throughput gains. Long-run results describe the tested environment and duration, not indefinite stability.
Public internship PRs
2026-09-29| PR | Merged change | Status |
|---|---|---|
| #163 ↗ | AdaMax operators, resource-variable support and tests. | Merged |
| #144 ↗ | Long-run stability, OOM and GradientDescent locking fixes. | Merged |
| #138 ↗ | GradientDescent operators for reference and resource variables. | Merged |
| #135 ↗ | Debug timing and shape synchronization stability. | Merged |
| #129 ↗ | Logical_Or scalar-broadcast optimization and logical-operator tests. | Merged |
| #113 ↗ | AddV2 hot-path optimization and profiling-noise reduction. | Merged |
| #101 ↗ | muDNN GELU, fusion fixes and workload-shaped benchmarks. | Merged |
| #78 ↗ | ResourceApplyAdam updates and session-close stability. | Merged |
| #72 ↗ | GELU graph fusion and correctness validation. | Merged |
| #57 ↗ | Kernel timing instrumentation. | Merged |