Vizuara Books
Transformers for Vision & Multimodal LLMs
Build every vision and multimodal transformer by hand.
From patch embeddings to diffusion: ViT, DeiT, Swin, DETR, SAM, NanoVLM, Flamingo, LLaVA and DDPM, coded from scratch and illustrated. Start at the convolutional inductive bias and end with your own end-to-end multimodal model.
Read on your Kindle
We'll send this whole book straight to your Kindle — it opens natively, so you can resize the text, read fully offline, and it remembers where you left off. Nothing to download or manage.
Sending to Kindle is for subscribers — subscribe to read the whole library on your Kindle.
00Foundations — From CNNs to Transformers5 capsules
Foundations — From CNNs to Transformers — 5 chapters.
01Why Vision Needs Transformersintuitionfree14 min02The Limits of the Convolutional Inductive Biasconcept🔒13 min03How nn.Conv2d Actually Workscode🔒14 min04Journey of a Token Through a Transformerintuition🔒16 min05The Feedforward Block and the Residual Streamconcept🔒15 min01Attention From Scratch7 capsules
Attention From Scratch — 7 chapters.
06From RNNs to Attentionintuition🔒13 min07Simplified Self-Attention, Coded From Scratchcode🔒12 min08Queries, Keys and Valuesconcept🔒14 min09Self-Attention With Trainable Weightscode🔒12 min10Masked (Causal) Self-Attentioncode🔒12 min11Multi-Head Attentioncode🔒13 min12Counting GPT-3's 175 Billion Parametersmath🔒12 min02The Vision Transformer5 capsules
The Vision Transformer — 5 chapters.
13Patch Embeddings and the CLS Tokenconcept🔒16 min14Coding the Vision Transformer From Scratchcode🔒14 min15Training ViT and Its Data Hungerdeep-dive🔒12 min16DeiT — Data-Efficient Image Transformersintuition🔒13 min17Coding DeiT From Scratchcode🔒14 min03Hierarchical & Dense Vision Transformers6 capsules
Hierarchical & Dense Vision Transformers — 6 chapters.
18Swin Transformer — Windows and Hierarchyintuition🔒14 min19Reading the Swin Paperdeep-dive🔒14 min20Coding Swin From Scratchcode🔒15 min21DETR — Detection as Set Predictionconcept🔒15 min22Coding DETR From Scratchcode🔒14 min23The Segment Anything Model (SAM)project🔒14 min04Video & Multimodal Models6 capsules
Video & Multimodal Models — 6 chapters.
24TimeSformer — Attention Over Videoconcept🔒11 min25Running TimeSformer on Videocode🔒15 min26Coding NanoVLM — Your First Vision-Language Modelproject🔒13 min27Flamingo — Few-Shot Multimodal Learningconcept🔒13 min28Building Flamingo-Style Cross-Attentioncode🔒14 min29LLaVA — Visual Instruction Tuningdeep-dive🔒12 min05Generative Vision — Autoencoders to Diffusion6 capsules
Generative Vision — Autoencoders to Diffusion — 6 chapters.
30Autoencoders From Scratchcode🔒13 min31Variational Autoencodersmath🔒14 min32Coding a VAEcode🔒15 min33Introduction to Diffusion Modelsintuition🔒13 min34Coding DDPM From Scratchcode🔒13 min35From DDPM to Stable Diffusiondeep-dive🔒14 min06Putting It All Together2 capsules
Putting It All Together — 2 chapters.
36The Vision & Multimodal Transformer Family Treeconcept🔒13 min37Capstone — Build Your Own Multimodal Modelproject🔒13 minRatings & reviews
No ratings yet. Yours would be the first.