Andrej Karpathy

• https://github.com/karpathy
• https://cs.stanford.edu/people/karpathy/
• https://karpathy.ai/
• Deep Reinforcement Learning: Pong from Pixels: https://karpathy.github.io/2016/05/31/rl/
• _

Huggingface

• https://huggingface.co/docs/transformers/main_classes/model
• https://huggingface.co/docs/diffusers/using-diffusers/loading
•  

Fundamental Tutorials

• https://pytorch.org/tutorials/intermediate/seq2seq_translation_tutorial.html
• Book :: DAVID SILVER :: UCL Course on RL https://www.davidsilver.uk/teaching/
• _

Bert

• https://huggingface.co/blog/bert-101

Pytorch

• https://pytorch.org/hub/huggingface_pytorch-transformers/
•  

Self-instruct:

• https://github.com/yizhongw/self-instruct
• Self-Instruct: Aligning Language Models with Self-Generated Instructions https://arxiv.org/abs/2212.10560

• https://github.com/tatsu-lab/stanford_alpaca

  LLaMA: Open and Efficient Foundation Language Models (not actually open?)

  • https://arxiv.org/abs/2302.13971v1  (saved)
• Not really open then: https://crfm.stanford.edu/2023/03/13/alpaca.html
  • https://github.com/tatsu-lab/stanford_alpaca#data-generation-process
  •  
•  

EleutherAI

• https://en.wikipedia.org/wiki/EleutherAI

  https://www.eleuther.ai/releases

  • Pythia, A suite of models designed to enable controlled scientific research on transparently trained LLMs https://github.com/EleutherAI/pythia
  • _

Code

• https://github.com/NVIDIA/Megatron-LM
  • https://www.deepspeed.ai/
  • _
• _

Datasets

• Free Dolly: Introducing the World's First Truly Open Instruction-Tuned LLM
  • https://github.com/databrickslabs/dolly/tree/master/data
  • https://huggingface.co/datasets/databricks/databricks-dolly-15k/blob/main/databricks-dolly-15k.jsonl
  • https://www.databricks.com/blog/2023/04/12/dolly-first-open-commercially-viable-instruction-tuned-llm
• _