Vizuara Books
How to Host Kimi K3
A measured field guide to serving a 2.8-trillion-parameter open model.
Kimi K3 is 1.561 TB of MXFP4 weights across 96 shards, and most of the GPU market cannot hold it. This book does the memory arithmetic from first principles, then walks the one deployment we ran end to end: 8x B300 on Modal with vLLM, every flag explained, 0.93 s to first token, 92 tok/s at steady state and $190.13 per million output tokens. Five failures that cost real time and money are documented with their exact errors. Every claim is labelled measured by us, reported by others, or not verified.
Read on your Kindle
We'll send this whole book straight to your Kindle — it opens natively, so you can resize the text, read fully offline, and it remembers where you left off. Nothing to download or manage.
Sending to Kindle is for subscribers — subscribe to read the whole library on your Kindle.
00Start Here3 capsules
What this book is, how to tell a measured number from a reported one, and what the licence lets you do.
01How to Read This Bookorientation10 min02What You Are Actually Hostingoverview10 min03The Licence and What It Permitslegal10 min01The Memory Math4 capsules
The arithmetic that decides, before you launch anything, which configurations are even possible.
04The Checkpoint on Diskmeasured10 min05The Architecture That Decides Your Billmeasured12 min06Weights, KV Cache, and Headroomfirst-principles12 min07Why 640 GB Can Never Hold 1.56 TBthe-wall18 min02The Deployment That Works11 capsules
The heart of the book: the 8x B300 vLLM deployment we ran end to end, reproducible from these chapters alone.
08The Node That Fitsthe-main-path11 min09Getting the Weights onto a Volumeoperations12 min10Verifying Before You Rent a GPUmeasured10 min11The Server in Fullthe-main-path16 min12Every Flag Explainedreference16 min13The fp8 KV Cache Trapgotcha16 min14What Cold Boot Actually Doesmeasured15 min15First Tokens, and the Reasoning Fieldgotcha11 min16The Measured Numbersmeasured15 min17Tool Calling That Worksmeasured11 min18Reproduce This in One Sittingrunbook12 min03The Hardware Map2 capsules
Supporting material: the GPUs on the menu, what distinguishes them, and how eight of them are wired together.
19The GPU Menu, A100 to B300hardware12 min20NVLink, Tensor Parallelism, and Topologyhardware11 min04The Quantized Path6 capsules
It works, and it is a trap. Kimi K3 served on 8x A100 at ~9 tok/s and about $620 per million tokens, which is 3.3x more per token than the B300 tier.
21MXFP4 vs GGUFconcept12 min22Choosing a Quantreported16 min23The Xet Trapgotcha9 min24Mainline llama.cpp Cannot Run K3gotcha11 min25Building the Unsloth Forkmeasured26 min26What the Quantized Path Actually Costsmeasured12 min05Routes We Have Not Verified3 capsules
Documented elsewhere, untested here. Presented as research, with every number attributed to whoever measured it.
27Multi-Node Servingnot-verified12 min28Other Enginesnot-verified14 min29Macs, CPU Offload, and Pruningnot-verified14 min06Making It Fast4 capsules
Four techniques that change throughput. All four are reported by vLLM or SGLang; we ran none of them.
30Speculative Decoding with DSparkreported14 min31Prefix Caching over a Recurrent Statereported12 min32Prefill and Decode Disaggregationreported11 min33Expert Parallelism and EPLBreported12 min07Making It Cheap3 capsules
The endpoint costs $1,363 a day if you leave it running. The arithmetic for not doing that.
34Scale to Zerocost17 min35Dollars per Million Tokenscost17 min36Host It, or Call the APIdecision12 min08Operating It4 capsules
What to watch, what the endpoint will not tell you, and what to do when something breaks.
37Vision and a Million Tokensoperations13 min38What to Monitoroperations14 min39Inspecting Weights Without a GPUtechnique19 min40When Things Breakreference22 minRatings & reviews
No ratings yet. Yours would be the first.