Textbooks

• Wow, an excellent textbook: https://d2l.ai/

AI-ML Youtube Channels

• Three blue one brown
• two-minute papers
• StatQuest
• Prompt Muse

From the illustrated transformer

• Read the Attention Is All You Need paper, the Transformer blog post (Transformer: A Novel Neural Network Architecture for Language Understanding), and the Tensor2Tensor announcement.
• Watch Łukasz Kaiser’s talk walking through the model and its details
• Play with the Jupyter Notebook provided as part of the Tensor2Tensor repo
• Explore the Tensor2Tensor repo.

• Stable Diffusion (VAE) from scratch

• https://readmedium.com/building-stable-diffusion-from-scratch-using-python-f3ebc8c42da3
• https://github.com/FareedKhan-dev/create-stable-diffusion-from-scratch
• https://medium.com/@sayedebad.777/implementing-stable-diffusion-from-scratch-using-pytorch-f07d50efcd97

Bert

• https://huggingface.co/blog/bert-101

Pytorch

• https://pytorch.org/hub/huggingface_pytorch-transformers/
• https://pytorch.org/tutorials/intermediate/seq2seq_translation_tutorial.html

Huggingface

• https://huggingface.co/docs/transformers/main_classes/model
• https://huggingface.co/docs/diffusers/using-diffusers/loading

Andrej Karpathy

• https://github.com/karpathy
• https://cs.stanford.edu/people/karpathy/
• https://karpathy.ai/
• Deep Reinforcement Learning: Pong from Pixels: https://karpathy.github.io/2016/05/31/rl/

Datasets

Free Dolly: Introducing the World's First Truly Open Instruction-Tuned LLM

• https://github.com/databrickslabs/dolly/tree/master/data
• https://huggingface.co/datasets/databricks/databricks-dolly-15k/blob/main/databricks-dolly-15k.jsonl
• https://www.databricks.com/blog/2023/04/12/dolly-first-open-commercially-viable-instruction-tuned-llm