Andrej Karpathy explains how LLMs are pre-trained, fine-tuned and subjected to reinforcement learning.
June 14, 2026
Key Stages of LLM Development
Pre-training
Massive dataset training – models are trained on large corpora of internet text such as FineWeb.
Tokenisation – raw data is processed into tokens, the fundamental units the neural network manipulates.
Base model output – the result is a base model that functions as a stochastic token autocomplete system, predicting the next sequence of text based on learned statistical patterns.
Post-training
Fine-tuning – base models are fine-tuned using conversational datasets, often created by human labellers, to produce assistant models capable of following instructions, being helpful, and maintaining safety.
Reinforcement Learning
Reasoning refinement – models undergo further training to refine reasoning and problem-solving skills.
Chain-of-thought – recent breakthroughs such as DeepSeek-R1 demonstrate this through chain-of-thought reasoning, where the model works through problems step by step before producing an answer.