MB
Marcus Bell
3 weeks ago
The hand-drawn diagrams make abstract ideas click instantly. Wish every technical book was written like this.
Free preview available. Sign in and subscribe to unlock the full book.
Vizuara AI Labs · intermediate
Build a Vision Transformer (ViT) from Scratch
Build the Vision Transformer by hand in PyTorch.
An image becomes a sequence of 16x16 patch tokens, and a plain transformer classifies it. Build every piece from scratch — patch embedding, class token, multi-head attention, encoder blocks, and the training loop.
33 capsules115 figures~6 hoursby Dr. Raj Dandekar
Read on your Kindle or e-reader
Download the EPUB and read offline — perfect for the train. Works on Kindle, Kobo, Apple Books & more.
00Why Vision Transformers5 capsules
Why Vision Transformers — 5 chapters.
01Course Introductionconceptfree12 min02From CNNs to Transformersconceptfree11 min03An Image is Worth 16x16 Wordsconcept🔒11 min04The ViT Architecture at a Glanceintuition🔒15 min05Setup and Tensor Conventionscode🔒12 min01Turning Images into Tokens6 capsules
Turning Images into Tokens — 6 chapters.
06Splitting an Image into Patchesintuition🔒13 min07Flattening and Linear Projectionmath🔒11 min08Patch Embedding with a Conv2d Trickcode🔒12 min09Coding the PatchEmbedding Classcode🔒10 min10The [CLS] Class Tokenconcept🔒11 min11Positional Embeddingsmath🔒10 min02Attention Inside the ViT6 capsules
Attention Inside the ViT — 6 chapters.
12Self-Attention Intuitionintuition🔒10 min13Queries, Keys, and Valuesmath🔒11 min14Coding Scaled Dot-Product Attentioncode🔒11 min15Multi-Head Attentionconcept🔒11 min16Coding Multi-Head Attentioncode🔒12 min17Why ViT Attention is Bidirectionaldeep-dive🔒11 min03The Transformer Encoder Block5 capsules
The Transformer Encoder Block — 5 chapters.
18Layer Normalizationmath🔒11 min19The MLP Feed-Forward Blockconcept🔒11 min20Residual Connectionsintuition🔒11 min21Coding One Encoder Blockcode🔒9 min22Stacking the Encodercode🔒11 min04Assembling the Full ViT5 capsules
Assembling the Full ViT — 5 chapters.
23The Classification Headconcept🔒9 min24Wiring the Full ViT Modelcode🔒11 min25Parameter Initializationcode🔒11 min26Counting Parameters and Computedeep-dive🔒10 min27End-to-End Forward Passcode🔒10 min05Training, Data, and Beyond6 capsules
Training, Data, and Beyond — 6 chapters.
28Why ViTs Are Data-Hungrydeep-dive🔒10 min29Loss, Optimizer, and the Training Loopcode🔒10 min30Training ViT on a Small Datasetproject🔒10 min31Visualizing Attention Mapsproject🔒11 min32Fine-Tuning Pretrained ViTsconcept🔒10 min33Putting It All Togetherproject🔒10 minRatings & reviews
1,164 readersRate this book
NK
Noah Kim
last month
Practical and rigorous at the same time. I went straight from reading a capsule to shipping it at work.
TA
Tomás Alvarez
4 months ago
A few sections moved fast for me, but re-reading with the figures open made everything land. Highly recommend.