HW
Hannah Weiss
6 days ago
Dense but never confusing. Every chapter builds cleanly on the last — you can feel the care in the structure.
Free preview available. Sign in and subscribe to unlock the full book.
Vizuara AI Labs · intermediate
Reinforcement Learning
Build every RL algorithm by hand.
From the agent-environment loop to DQN, policy gradients, RLHF and GRPO, every reinforcement learning algorithm is derived and coded from scratch. Watch agents learn from experience — landing a lunar lander, playing Atari, aligning a language model, and reasoning like DeepSeek-R1.
38 capsules140 figures~7 hoursby Dr. Raj Dandekar
Read on your Kindle or e-reader
Download the EPUB and read offline — perfect for the train. Works on Kindle, Kobo, Apple Books & more.
00Foundations of Reinforcement Learning5 capsules
Foundations of Reinforcement Learning — 5 chapters.
01What Is Reinforcement Learning?conceptfree10 min02The Agent-Environment Interfaceconceptfree10 min03The Markov Property and MDPsmath🔒11 min04Rewards, Returns and Discountingmath🔒12 min05OpenAI Gymnasium: Your First Environmentcode🔒12 min01The Three Pillars of Classical RL6 capsules
The Three Pillars of Classical RL — 6 chapters.
06Value Functions and the Bellman Equationsmath🔒10 min07Dynamic Programming: Policy and Value Iterationconcept🔒11 min08Monte Carlo Methods: Learning from Episodesconcept🔒9 min09Temporal Difference Learningconcept🔒10 min10Q-Learning and SARSAmath🔒12 min11Project: Landing a Lunar Landerproject🔒12 min02Deep Q-Networks5 capsules
Deep Q-Networks — 5 chapters.
12The Birth of Deep RL: From Pong to Atariconcept🔒10 min13From Q-Tables to Q-Networksconcept🔒11 min14Experience Replay and Target Networksdeep-dive🔒12 min15The DQN Training Loopmath🔒12 min16Project: Build a DQN from Scratchproject🔒12 min03Policy Gradient Methods from Scratch6 capsules
Policy Gradient Methods from Scratch — 6 chapters.
17Why Optimize the Policy Directly?concept🔒11 min18The Policy Gradient Theoremmath🔒10 min19The REINFORCE Algorithmcode🔒11 min20REINFORCE with a Baselinemath🔒12 min21Advantage Functions and Actor-Criticconcept🔒11 min22Generalized Advantage Estimationdeep-dive🔒12 min04RLHF from Scratch7 capsules
RLHF from Scratch — 7 chapters.
23Why Language Models Need Alignmentconcept🔒10 min24The Three-Stage RLHF Pipelineconcept🔒12 min25Training a Reward Model from Preferencesmath🔒11 min26Project: An SLM That Writes Positive Storiesproject🔒11 min27The PPO Algorithm Explainedmath🔒10 min28The PPO Training Loop for Language Modelsdeep-dive🔒10 min29Project: A Reddit Post Summarizerproject🔒12 min05Build a Reasoning Model with GRPO4 capsules
Build a Reasoning Model with GRPO — 4 chapters.
30From PPO to GRPOconcept🔒10 min31How GRPO Worksmath🔒10 min32GRPO and the DeepSeek-R1 Revolutiondeep-dive🔒11 min33Project: Turn an SLM into a Reasoning Modelproject🔒12 min06Agentic Reinforcement Learning3 capsules
Agentic Reinforcement Learning — 3 chapters.
34What Is Agentic RL?concept🔒11 min35The Agentic RL Landscapedeep-dive🔒11 min36Project: An Agentic RAG App Trained with RLproject🔒10 min07From Learner to Researcher2 capsules
From Learner to Researcher — 2 chapters.
37Putting It All Togetherconcept🔒12 min38Doing Impactful RL Researchdeep-dive🔒11 minRatings & reviews
1,304 readersRate this book
EP
Elena Petrova
2 months ago
Practical and rigorous at the same time. I went straight from reading a capsule to shipping it at work.
ML
Mei Lin
3 months ago
The hand-drawn diagrams make abstract ideas click instantly. Wish every technical book was written like this.
FZ
Fatima Zahra
4 months ago
A few sections moved fast for me, but re-reading with the figures open made everything land. Highly recommend.
SR
Sofia Reyes
8 months ago
Rare mix of depth and readability. The worked examples are the clearest I have seen on this subject.