Skip to content
LearnMLByDoing
Free
04

Transformers and LLMs

Attention as geometry: queries, keys and values moving through embedding space, scaled up into language models.

Lessons
8
Study time
5.1 h
Access
First lesson free

Syllabus

  1. 4.0

    Inside an LLM: from one neuron to the next word

    Follow a real, tiny GPT from a single neuron to an MLP, an attention head and a stacked decoder, until it turns a prompt into next-word probabilities and generates text one token at a time.

    45 minFreeLive 3D lab and Python challengeintermediate
    • Trace a token from a single neuron through an MLP to an attention head with queries, keys, values, a causal mask and a softmax
    • Follow the residual stream through multi-head blocks stacked into a decoder
    • Turn the final hidden state into next-token probabilities and run the generation loop
  2. 4.1

    Self-attention geometry and scaled dot-product in 3D

    Project tokens into query, key and value spaces, fire attention beams between them, and see exactly why we divide by √dₖ.

    42 minFree previewLive 3D lab and Python challengeintermediate
    • Compute Q, K, V projections and their tensor shapes
    • Explain why unscaled dot products saturate the softmax
    • Read attention weights as a soft lookup over keys
  3. 4.2

    Tokenisation and embedding spaces

    Byte-pair encoding from scratch, and how embedding tables turn token ids into directions with meaning.

    30 minProLive 3D lab and Python challengeintermediate
    • Train a BPE tokenizer on a small corpus
    • Measure similarity in embedding space
    • Add positional information with sinusoidal and rotary encodings
  4. 4.3

    Multi-head attention and the transformer block

    Heads as parallel subspaces, residual streams, pre-norm and the MLP that stores most of a model's facts.

    38 minProLive 3D lab and Python challengeadvanced
    • Split and merge heads with reshapes and transposes
    • Trace the residual stream through a block
    • Count parameters and FLOPs of a transformer layer
  5. 4.4

    Pretraining, sampling and fine-tuning LLMs

    Next-token loss at scale, temperature and nucleus sampling, and parameter-efficient fine-tuning with LoRA.

    40 minProLive 3D lab and Python challengeintermediate
    • Implement temperature, top-k and top-p sampling
    • Explain scaling laws at a high level
    • Fine-tune with low-rank adapters
  6. 4.5

    Embeddings, vector search and retrieval-augmented generation

    Embed a corpus, fire a query into it, and follow nearest-neighbour search through an HNSW graph before the results reach the prompt.

    38 minFree previewLive 3D lab and Python challengeintermediate
    • Retrieve with cosine similarity and explain the cost of exact search
    • Trace an approximate search through an HNSW index and its recall trade-off
    • Design a RAG pipeline: chunking, re-ranking with MMR, and retrieval evaluation
  7. 4.6

    Alignment: reward models, RLHF and DPO

    From instruction tuning to preference optimisation: reward models, the KL leash of RLHF, and the closed-form shortcut DPO takes.

    36 minProLive 3D lab and Python challengeadvanced
    • Train a reward model from pairwise preferences with the Bradley-Terry loss
    • Explain the KL-regularised RLHF objective and reward hacking
    • Derive the DPO loss and compare it with PPO-based RLHF
  8. 4.7

    LLM agents, tool use and evaluation

    Reason, act, observe: the agent loop, function calling, planning budgets, and how to evaluate a system that takes actions.

    34 minProLive 3D lab and Python challengeintermediate
    • Implement a ReAct-style loop with tool calls and observations
    • Bound cost and failure with step budgets, retries and guardrails
    • Evaluate agents with task success, trajectory checks and LLM-as-a-judge