Home for "How To Scale Your Model", a short blog-style textbook about scaling LLMs on TPUs
-
Updated
Oct 19, 2026 - SCSS
Home for "How To Scale Your Model", a short blog-style textbook about scaling LLMs on TPUs
Modular C++ Toolkit for Performance Analysis and Logging. Profiling API and Tools for C, C++, CUDA, Fortran, and Python. The C++ template API is essentially a framework to creating tools: it is designed to provide a unifying interface for recording various performance measurements alongside data logging and interfaces to other tools.
Why is LLM inference slow — and how do you make it fast? A hands-on, first-principles course: roofline → KV cache → quantization → parallelism → vLLM/SGLang, with GPU labs on open models.
Analyze LLM inference: FLOPs, memory, Roofline model. Supports GQA, MoE, MLA, RoPE, SwiGLU. 19 models × 20+ hardware platforms.
Closed-loop CUDA kernel discovery: profile, diagnose, optimize, and machine-check every variant against a Lean 4 model of PTX
Hand-written CUDA vs Mojo GPU kernels benchmarked on consumer Ampere (RTX 3090, sm_86), with roofline analysis
Interactive 3D visualization of dense decoder-only LLM inference. Companion to the AI Inference Engineer 2026 course.
Reproducible GPU performance workflow with Nsight CSV parsing, Roofline analysis and portable backend seams.
Memory-bound Jacobi stencil on SuperMUC-NG, optimized from AVX-512 and temporal blocking to NUMA-aware OpenMP and hybrid MPI. Explained by LIKWID hardware counters.
From-scratch IO-aware attention: online-softmax proof, 500-test harness written before the kernel, and a measured roofline. Triton/CUDA waves pending a GPU.
《大模型推理系统工程:性能、诊断与优化》—— 从一次请求的完整路径讲到容量与成本
Interactive theoretical Kimi-K3 inference roofline calculator for H200, B300, and GB300
JAX benchmarking, profiling and evaluation metrics for Flax NNX: a registry of pure-function metrics (regression, classification, calibration, uncertainty, forecasting, generative, image, text, audio, graph, fairness), XLA FLOP counting, roofline analysis, GPU and energy monitoring, regression detection, publication exports, W&B and MLflow.
Dense matrix multiply in four parallel models — hand-tiled OpenMP with roofline, Cannon's algorithm, hybrid MPI+OpenMP reaching 45 TFLOP/s on 16 nodes, and multi-GPU cuBLAS
How LLMs actually run on real hardware, from first principles: roofline, KV cache, quantization, batching. Runnable labs, tested numbers, MIT licensed.
Why code is slow, at four levels — an out-of-tree LLVM pass that predicts vectorisability from IR, cache-blocked GEMM against a roofline, and a CUDA FFT port
The physics before the engines: measured arithmetic intensity, prefill compute-bound vs decode memory-bandwidth-bound, with the roofline plot.
Running & optimizing open-weight LLMs on a single Tenstorrent Blackhole p150a: decode roofline, BFP8/BFP4 emulation, TT-Metalium kernels (KEYEAST, Changwon Science HS)
To associate your repository with the roofline topic, visit your repo's landing page and select "manage topics."