CoursesInteractive teaching materials by Han Tian. Software Optimization for Intelligent ComputingChapter 5 · GPU Architecture and Computational Optimization 01
Interactive demo · Matrix multiplication GPU Matrix Multiplication: From Naive Computation to Tile ReuseCompare naive multiplication, register tiling, and shared-memory plus register tiling. Follow matrix accesses, data reuse, and output accumulation step by step.
02
Interactive demo · Attention FlashAttention and FlashAttention-2: Matrix Tile TransfersSelect actual tiles within the full Q, Kᵀ, and V matrices. Follow their elements from HBM to on-chip GPU storage, through local computation and online Softmax, and back to the matching output tiles in O.
|