Fundamentals
Transformer
The neural network architecture, introduced in 2017, that made modern large language models possible.
Before Transformers, language models processed text mostly in order, one word after another, which made it hard to capture long-range relationships and slow to train. The Transformer architecture — from the 2017 paper “Attention Is All You Need” — introduced self-attention, letting the model weigh the relevance of every word in a passage against every other word at once.
That parallelism is what made today’s scale possible: Transformers can be trained efficiently across thousands of GPUs at the same time. Nearly every major LLM in production, from GPT to Claude to Gemini, is a Transformer variant.