Non Playmore: Executable Transcript of Andrej Karpathy’s “Neural Networks: Zero to Hero” Series
Today I’m releasing part I of Playmore: Executable Transcript of Andrej Karpathy’s “Neural Networks: Zero to Hero” Series. It’s a Jupyter-based, hands-on, code-along lecture companion, complete with timecodes.
Part I covers building micrograd, a minimal neural net training library. Along the way you develop an intuition of how neural networks learn and work. It shows how the code in Andrej’s lecture evolves, one idea at a time, following the linear order of the video. There’s deliberately almost no code encapsulation and DRY principle is blatantly violated. This trade-off escapes the need to scroll up and down: all the relevant code is always in your sight.
Table of contents
| Part | Notebook | Launch | Lecture |
|---|---|---|---|
| 1 | Building micrograd: the spelled-out intro to neural networks and backpropagation |  |
|
| 2 | Building makemore: intro to language modeling | Coming soon | |
| 3 | Building makemore: MLP | Coming soon | |
| 4 | Building makemore: Activations & Gradients, BatchNorm | Coming soon | |
| 5 | Building makemore: Becoming a Backprop Ninja | Coming soon | |
| 6 | Building makemore: WaveNet | Coming soon | |
| 7 | Building GPT: from scratch, in code, spelled out | Coming soon |
But why?
I started making these transcripts as a way to understand and review the material later. Then I realized this could be useful to others:
- There’s value in being able to follow the development of an idea through intuitions, trials and errors. It’s a way of remembering the why of how something works. Neat final code doesn’t do that.
- I find that reading commentary and executing code is way more efficient to revisit ideas after a while than going through the video again.
- Some people like text more than video. Or want to supplement one with the other for a multimodal learning experience.
If you prefer the final code, Andrej published it in this Github repo.
Does this map 1:1 to the video?
Not word-by-word. Occasionally there’s some extra code or metaphor I added to illustrate a point. Some parts are summarized and you should refer to the corresponding video section for details.
How to take the most out of this?
What I cannot create, I do not understand.
This is a play-as-you-watch lecture supplement helping you grasp material more fully and revise it after a while.
For maximum benefit, try to recreate the code for key ideas and write lecture notes of your own, aiming to express the underlying ideas precisely and poking holes in your reasoning.
Why Playmore?
It’s a reference to the play button used for executing a Jupyter notebook cell. And a subtle invitation to learn by doing and playing, which is the most effective approach to master a subject. You don’t learn to play tennis just by watching someone else do it - gotta do it yourself!
Acknowledgements
99% of the content comes from Andrej. Any mistakes or inaccuracies are all mine (please let me know!).
License
MIT License