#inference, #llm Everyone can call an inference API. Few can explain why vllm serve beats their PyTorch loop.
So I wrote the course: 22 chapters from a while-loop to PagedAttention, continuous batching and speculative decoding.
No GPU, no PyTorch. llminference.xinbetween.com
