TA
Tomás Alvarez
2 weeks ago
Loved the from-first-principles approach. It rebuilt my intuition rather than just handing me formulas.
Free preview available. Sign in and subscribe to unlock the full book.
Vizuara AI Labs · advanced
Build a NanoVLM from Scratch
Fuse vision and language from an empty file.
Build a tiny CLIP-style vision-language model by hand: encode images and captions into a shared embedding space and align them with contrastive learning. From patchified images and cosine similarity to the symmetric contrastive loss and zero-shot classification, every piece is coded and illustrated.
36 capsules125 figures~7 hoursby Dr. Raj Dandekar
Read on your Kindle or e-reader
Download the EPUB and read offline — perfect for the train. Works on Kindle, Kobo, Apple Books & more.
00Vision-Language Foundations6 capsules
Vision-Language Foundations — 6 chapters.
01What Is a Vision-Language Model?conceptfree11 min02Why Multimodal — What VLMs Unlockintuitionfree11 min03One Idea: Vision and Language Are Both Vectorsintuition🔒10 min04From LLMs to VLMs: What Carries Overconcept🔒11 min05The Landscape: CLIP and Friendsconcept🔒13 min06What We Will Build: The NanoVLM Blueprintproject🔒13 min01Embeddings & Similarity6 capsules
Embeddings & Similarity — 6 chapters.
07Vectors as Meaningconcept🔒10 min08Text Embeddings from Scratchcode🔒13 min09Image Embeddings from Scratchcode🔒12 min10Cosine Similarity, Explainedmath🔒10 min11Dot Products, Norms, and Normalizationmath🔒12 min12The Shared Embedding Spaceintuition🔒10 min02Encoding Images into Vectors6 capsules
Encoding Images into Vectors — 6 chapters.
13Two Ways to Vectorize an Imageconcept🔒11 min14Patchify an Imagecode🔒13 min15Patch Embeddings and Positionscode🔒12 min16The CNN Route to an Image Vectorcode🔒12 min17Pooling to a Single Image Vectorcode🔒11 min18Matching Image and Text Dimensionscode🔒10 min03Contrastive Learning6 capsules
Contrastive Learning — 6 chapters.
19The Contrastive Ideaintuition🔒11 min20Positives, Negatives, and the Batchconcept🔒11 min21The Similarity Matrixmath🔒11 min22The CLIP Contrastive Lossmath🔒13 min23Temperature and Logit Scalingdeep-dive🔒12 min24Coding the Contrastive Losscode🔒10 min04Building the NanoVLM6 capsules
Building the NanoVLM — 6 chapters.
25NanoVLM Architecture Overviewconcept🔒13 min26The Image Encoder Modulecode🔒12 min27The Text Encoder Modulecode🔒12 min28The Projection Headscode🔒12 min29Synthetic Data for Trainingcode🔒13 min30Assembling the Full Forward Passcode🔒12 min05Training, Evaluation & Beyond6 capsules
Training, Evaluation & Beyond — 6 chapters.
31The Training Loopcode🔒14 min32Watching Alignment Emergeintuition🔒13 min33Zero-Shot Classificationproject🔒10 min34Image-Text Retrievalproject🔒11 min35Debugging and Scaling Tipsdeep-dive🔒14 min36From NanoVLM to Real VLMsconcept🔒10 minRatings & reviews
411 readersRate this book
JB
Jonas Bergström
4 months ago
Beautifully produced and genuinely deep. The reader experience makes it easy to keep going for hours.
DO
Daniel Okafor
5 months ago
Concise capsules I can finish over coffee, yet each one taught me something I actually use.
LC
Liam Chen
8 months ago
Rare mix of depth and readability. The worked examples are the clearest I have seen on this subject.