PyTorch MLP Profiling: How nn.Linear Transposes Weights Before Matrix Multiplication
Hugging Face's second profiling guide reveals the hidden transpose operation in PyTorch's Linear layer and demonstrates kernel fusion techniques for production MLPs.
AutoMegaKernel: RightNow AI's LLM-to-CUDA Compiler Aims for Provably Correct Inference Kernels
A GitHub research project claims to compile LLM computation graphs into single CUDA kernels with formal correctness guarantees, but lacks published benchmarks or third-party validation.