diy-llm
Covers pre-training data, Tokenizer, Transformer, MoE,distributed training, Scaling Laws, inference & alignment .6 progressive code assignments for full-stack LLM learning | 涵盖预训练数据、分词器、Transformer、MoE、分布式训练、缩放定律、推理与对齐，6 项渐进代码作业，掌握 LLM 全栈知识
- cs336
- datawhale
- gpu-programming
- grpo
- llm
- nlp
- rl
- scaling-laws
- sft
- transformer
- Stars
- 2,257
- Forks
- 218
- + today
- +6
- Created
- 11mo
Ranking data as of October 29, 2026 (UTC).
Star History
Today, hour by hour
Overview
Covers pre-training data, Tokenizer, Transformer, MoE,distributed training, Scaling Laws, inference & alignment .6 progressive code assignments for full-stack LLM learning | 涵盖预训练数据、分词器、Transformer、MoE、分布式训练、缩放定律、推理与对齐，6 项渐进代码作业，掌握 LLM 全栈知识 It ranks #64 on GitTiger, gaining +6 stars on October 29, 2026 (UTC).
The project is written in Jupyter Notebook and has 218 forks. It was created 11mo ago.
What it is
`diy-llm` is a Datawhale China learning project on building large language models from the ground up. The description (given in Chinese and English) says it covers pre-training data, tokenizers, Transformers, Mixture-of-Experts, distributed training, scaling laws, inference and alignment, organised around six progressive coding assignments aimed at full-stack LLM knowledge.
Capabilities described
The material is delivered as Jupyter notebooks. The README preview shows a chapter table — chapters, key content, companion assignment, status — plus a prerequisites section. Those prerequisites are:
- Working Python and software-engineering ability - Deep-learning fundamentals, including PyTorch and neural-network basics - Linear algebra, probability and statistics, calculus - Solid grounding in machine learning and deep learning - Optional CUDA/GPU-programming familiarity, with the note that beginners are not excluded because the project includes introductory material
Topic tags reference CS336, GRPO, SFT, scaling laws and GPU programming, which suggests the curriculum follows that Stanford course's themes.
Getting started
The evidence gives prerequisites but no install commands, notebook paths or environment setup, so start from the repository README. No licence is supplied in the metadata, which matters if you intend to reuse the material.
git clone https://github.com/datawhalechina/diy-llm.git
cd diy-llm
# see README for setupFrequently Asked Questions
Do I need a GPU to follow along?
The prerequisites list GPU programming as optional and say introductory tutorials are included, so prior CUDA knowledge is not required. No hardware requirement is stated.
What background should I have first?
Python and software engineering, PyTorch and neural-network basics, linear algebra/probability/calculus, and a solid grasp of machine learning fundamentals.
How is the course structured?
As chapters pairing key content with a companion assignment and a status column, across six progressive coding assignments, in Jupyter notebooks.
Is there a licence for reuse?
No licence appears in the repository metadata supplied here.