Arham Amin

Day 26 of learning AI bs from scratch > reading around Transformer training > went through more of nanoGPT + the GPT walkthrough a bit > looked deeper into tokenization, context windows and next-token prediction > attention + masking with the full training loop in mind > read through a few sections on scaling and why bigger models behave differently > getting down a bunch of gaps before moving further Mostly a reading-heavy day again, actually understand the training side before getting into a bigger model. good references: github.com/karpathy/nanoGPT youtube.com/watch?v=kCc8Fm gpt otw>

Day 25 of learning AI bs from scratch > spent time reading through Attention Is All You Need > went back over attention, masking and positional encoding > over laying code i had already written before > kept nanoGPT open alongside it > trained a small next-token prediction model > a bunch of things that finally made more sense now mostly reading and continuing the Transformer stuff from the last few days. references: arxiv.org/abs/1706.03762 github.com/karpathy/nanoGPT deploying a usable llm soon and wrapping this up, then let’s see what comes next >

Day 24 of learning AI bs from scratch >continued around Transformers > looked into tokenization + embeddings >understood causal masking a bit better > followed how tokens move through stacked Transformer blocks > played around with a small next-token prediction setup > went back through parts of Karpathy’s GPT walkthrough mostly around the Transformer stuff from the last couple days instead of trying to jump ahead. good reference: youtube.com/watch?v=kCc8FmEb… will make this GPT usable online. on.

Day 23 of learning AI bs from scratch > continued deeper into Transformers > touched multi-head attention > masking, residual connections and layer normalization > spent more time understanding how all the pieces interact inside a Transformer block > started looking at how multiple Transformer blocks get stacked together > stuff around Karpathy’s nanoGPT implementation A lot of different pieces today, less about finishing one thing and more about making the whole architecture click. good reference: github.com/karpathy/nanoGPT going deeper into training + implementation next.

Day 22 of learning AI bs from scratch > jumped properly into Transformers > implemented a basic self-attention block > worked through Q, K and V again while coding it > added positional encoding so the model knows token order > started putting the pieces together into a block Found some good resources, used Jay Alammar’s The Illustrated Transformer as a reference: jalammar.github.io/illustrat… catching up to the frontier.

Day 21 of learning AI bs from scratch > started the Transformer architecture > went through @karpathy breakdowns + a couple more resources around it (great video, linked below) > how things fit in transformers > understanding how self-attention, multi-head attention and positional information fit together > traced tokens → embeddings → attention → output > revisited Q, K and V + how everything connects from RNNs to Transformers updated the roadmap with the new resources + the models I’ve trained/hosted so far: arhamamin.com/pages/ai-roadm… Going deeper into the actual implementation next.

Day 20 of learning AI bs from scratch First models from this whole series are usable live! > trained a Neural Network for handwritten digit classification
> made a small interface where you can draw a number and let the model guess it
> deployed another model for tweet/headline sentiment analysis
> it predicts the sentiment and highlights the words carrying the most signal
> spent time testing both and making them usable live 
> getting closer to transformers entering the picture play with them: digit identifier → arhamamin.com/neural-network…
sentiment analysis → arhamamin.com/neural-network… going to make more stuff usable live. learning resources am going through
arhamamin.com/pages/ai-roadm…