Vizuara Books
Build a Diffusion Language Model from Scratch
Generate text all-at-once, from an empty file.
Build a masked diffusion language model from scratch and see a sentence emerge from noise instead of one token at a time. From embeddings and the autoregressive baseline to masking, denoising, unmasking schedules, and a full Colab build.
Read on your Kindle
We'll send this whole book straight to your Kindle — it opens natively, so you can resize the text, read fully offline, and it remembers where you left off. Nothing to download or manage.
Sending to Kindle is for subscribers — subscribe to read the whole library on your Kindle.
00Foundations: Text as a Distribution6 capsules
Foundations: Text as a Distribution — 6 chapters.
01Why a Diffusion Model for Language?intuitionfree14 min02Discrete vs Continuous Dataconcept🔒13 min03Representing Text in a Vector Spaceconcept🔒12 min04Word Embeddings: Word2Vec, GloVe, and BERTconcept🔒16 min05Text Generation as Sampling from a Distributionmath🔒13 min06Language Models as Distribution Approximatorsconcept🔒14 min01The Diffusion Idea4 capsules
The Diffusion Idea — 4 chapters.
07Generative AI: One Probabilistic View of Everythingintuition🔒12 min08Diffusion for Image Generationconcept🔒14 min09The Noising and Denoising Loopintuition🔒14 min10From Noise in Pixels to Masks in Textintuition🔒13 min02The Autoregressive Baseline6 capsules
The Autoregressive Baseline — 6 chapters.
11How Autoregressive Models Generate Textconcept🔒14 min12Tokenization and Embeddingscode🔒14 min13The Attention Mechanismmath🔒14 min14The Causal Attention Maskmath🔒13 min15Inside the Transformer Blockcode🔒15 min16Build an Autoregressive LLM from Scratchproject🔒14 min03Diffusion Language Model Theory6 capsules
Diffusion Language Model Theory — 6 chapters.
17Masked Diffusion Language Modelsconcept🔒14 min18The Forward Masking Processmath🔒14 min19The Matrix and Vector Operations Under the Hoodmath🔒16 min20Training vs Generation Phasesconcept🔒14 min21Denoising as Progressive Demaskingintuition🔒15 min22Visualizing Diffusion Generationintuition🔒15 min04Inference and Unmasking Strategies5 capsules
Inference and Unmasking Strategies — 5 chapters.
23The Denoising Process in Detaildeep-dive🔒14 min24Token Unmasking Schedulesconcept🔒13 min25Why Diffusion Is Faster: Speed vs Autoregressivedeep-dive🔒13 min26The Coherence vs Speed Tradeoffconcept🔒15 min27Speculative Decoding and Advanced Accelerationdeep-dive🔒13 min05Build It: Code the Diffusion LM6 capsules
Build It: Code the Diffusion LM — 6 chapters.
28Data Preprocessing and Tokenizationcode🔒14 min29The Forward Pass: Code Walkthroughcode🔒15 min30Training the Diffusion LMcode🔒16 min31Evaluation and Running Generationcode🔒16 min32Running on Colab and RunPodproject🔒17 min33Putting It All Togetherproject🔒16 min06Frontiers and Research Directions2 capsules
Frontiers and Research Directions — 2 chapters.
34Diffusion vs Autoregressive: Full Recapconcept🔒14 min35Potential Areas of Researchdeep-dive🔒15 minRatings & reviews
No ratings yet. Yours would be the first.