Monsoon

LLM engineer's handbook (30 minutes a day, 10 weeks, 50 lessons) a roadmap for llm inference serving where everything points at one service instead of scattering across demos. you get the mental model first, then serve a model, instrument it, load test it past 1000 concurrent requests, and tune it. you finish with a stack you configured yourself and a benchmark worth publishing. here is what it covers: → the roofline model, and why decode waits on memory while prefill waits on compute → vLLM internals, PagedAttention and the scheduler, read from the code → Prometheus and Grafana for TTFT, inter-token latency, and queue depth → SGLang and RadixAttention prefix reuse, benchmarked against vLLM → load testing past 1000 concurrent requests → quantization across FP16, FP8, and INT4, on quality as well as speed → speculative decoding and KV eviction, including where the gains disappear → disaggregated prefill and decode, deployed on Kubernetes → a cost, latency, and quality router with per-request token budgeting → publishing a reproducible benchmark the roadmap on GitHub: github.com/patchy631/time-to… (don't forget to star 🌟) i am also writing an article for each major topic. the first one is out, on how a GPU actually works. the article is quoted below.