Module 1 · What AI Is (Today)
A grounded mental model: LLMs vs classical ML vs rules, capabilities and limits, hype vs reality, when to use an LLM vs simpler tools.
- What AI Means for Developers Today: You are a developer and AI is now part of your toolkit. What AI actually is today, what changed when models became an API, and where this course takes you.
- Rules vs ML vs LLMs: Picking the Right Tool (with Code): Rules, classical ML, and LLMs side by side: real code for each, what each costs to run, and a comparison to reach for whenever someone says 'let's add AI.'
- What LLMs Are Good At: Six Capabilities That Ship in Real Products: The six things LLMs do well in production: summarize, extract, classify, rewrite, draft, and assist with code, each with a code example and where it breaks.
- What LLMs Fail At: Limits Every Developer Must Design Around: LLMs hallucinate, fail at reliable math, have no memory, and are open to prompt injection. Each failure mode with examples and how engineers handle it.
- Decoding the Hype: What AGI, Reasoning, and Agents Actually Mean: AI terms are overloaded. This concept translates the five most common hype words into engineering questions you can actually answer in a design review.
- Choosing the Right Tool: A Decision Framework You Can Use Monday Morning: Given a feature request, should you use rules, ML, an LLM, or nothing? A repeatable decision process with five worked examples to choose with confidence.
Module 2 · How LLMs Actually Work
Tokens, context windows, next-token prediction, sampling and temperature: the vocabulary used in every later module.
- How LLMs Actually Work: The 60-Second Version: Module 1 showed you what LLMs can do. This concept opens Module 2 by explaining how: the model is a function from tokens to tokens, and everything else (cost, latency, context limits, hallucinations) follows from that.
- Tokens Are Not Words: Tokenization, BPE, and Why Counting Can Fail: A beginner-friendly explanation of tokenization: how text becomes tokens, why tokens differ from words/characters, and how this affects cost, limits, truncation, and surprising failures.
- Next-Token Prediction: How Text Generation Actually Works: A truthful, non-magical mental model: generation is a loop of probabilistic next-token picks. Learn why models can be fluent without 'understanding' and why output can vary run to run.
- Transformer Mental Model: Attention and the Context Window (No Math Required): Understand attention in one diagram: every token can look back at previous tokens. Learn what the context window really is, and how window size changes product design (8k vs 128k vs 1M).
- Sampling and Temperature: Greedy vs Top-k vs Top-p (and when to use each): A beginner-friendly guide to generation controls: temperature, greedy decoding, top-k, and top-p. Learn why the same prompt can produce different answers and how to tune for extraction vs drafting vs brainstorming.
- What "Training" Means: Pre-training, Fine-tuning, and RLHF (High Level): A beginner-friendly overview of how models become useful chat assistants: pre-training, supervised fine-tuning (SFT), and RLHF / instruction tuning. Focuses on the product-visible effects, not math.
Module 3 · Prompting
Prompts as a real discipline: roles, few-shot, chain-of-thought, decomposition into prompt chains, testable constraints.
- System, User, Assistant: What Each Message Role Is For: Chat models expect a list of messages with roles. This lesson explains what each role is actually for, how the model treats them differently, and the common mistakes beginners make when they confuse one for another.
- Instruction Clarity: Turning a Vague Prompt Into a Specification: Vague prompts do not fail loudly. They fail silently with plausible wrong answers. This lesson teaches the six levers that take a prompt from 'it kind of works' to 'it produces the right output on every input you will throw at it'.
- Few-Shot and Chain-of-Thought: Teaching by Example, Reasoning on Purpose: Two of the most useful prompting techniques, demystified. When to show examples instead of writing rules, when to let the model think out loud, and when both are a waste of tokens.
- Prompt Chains: Breaking a Big Task Into Small, Testable Steps: One prompt can only carry so much. This lesson teaches how to decompose a complex task into a chain of smaller prompts, why per-step validation matters more than the final output, and the mistakes that turn chains into brittle pipelines.
- Prompting Antipatterns: The Habits That Quietly Ruin Reliability: A catalogue of the prompting habits that look harmless, spread through a codebase, and end up being the reason a feature is flaky. Written from production postmortems, not Twitter threads.
Module 4 · Calling LLMs
Request/response shapes, streaming, structured output, providers, tokens and cost: the bridge from Foundations to Production.
- The API Shape: What an LLM Call Actually Looks Like Over the Wire: A real HTTP request, a real response, and every field named. Once you know the shape, every provider's API becomes a variation on one theme instead of a stack of unfamiliar docs.
- Tokens and Cost: Budgeting Per Request, Per Day, Per Feature: Treat tokens the way you treat bytes: as a budget, not an afterthought. This lesson shows how to estimate a call's cost before you send it, the four cost traps that blow monthly bills, and how to design prompts with a budget in mind.
- Streaming Responses: SSE, Token-by-Token UX, and When Not to Bother: Streaming makes a five-second call feel like a one-second call. It does not actually make anything faster. This lesson explains how SSE works, how to consume a stream in TypeScript, and the edge cases that make streaming harder than it looks.
- Structured Output: Four Ways to Get JSON, and When Each One Earns Its Keep: Asking the model for JSON in the prompt is the weakest of four options you actually have. This lesson walks through plain prompting, JSON mode, JSON schema, and tool use, with real code and the failure mode of each.
- Provider Landscape: Picking a Model Provider Without the Brand Pull: OpenAI, Anthropic, Google, and the open-weight stack all solve the same problem differently. This lesson walks through what actually varies between providers and how to pick one on substance instead of which logo was on the last blog post you read.
Module 5 · LLM as Backend
High-latency, non-deterministic APIs: timeouts, retries, schema validation, streaming UX, cost and telemetry.
- The LLM Is a Slow, Flaky Microservice: The Mindset That Everything in M5 Builds On: Stop thinking of an LLM call as a function call. Start thinking of it as a request to a slow, occasionally-wrong external service that you do not control. Every reliability technique in Module 5 follows from this single mental shift.
- Timeouts and Deadlines: Deciding When an LLM Call Has Taken Too Long: An LLM call is slow and the tail latency is long. This lesson explains the difference between a per-attempt timeout and an end-to-end deadline, how to pick numbers for each, and why letting the provider decide when to give up is almost always wrong.
- Retries and Backoff: Which Failures to Retry, and How Not to Make Things Worse: Retries are the most common source of self-inflicted outages. This lesson sorts LLM failures into retryable and non-retryable buckets, explains exponential backoff with jitter, and shows the retry pattern that actually belongs in a production wrapper.
- Circuit Breakers and Fallbacks: Failing Fast When the Provider Is Down: Retries help with transient blips; they hurt during real outages. This lesson builds the circuit breaker that stops retries once failures pass a threshold, and pairs it with fallbacks so your feature degrades instead of hanging.
- Idempotency for LLM Calls: Making Retries Safe in the Presence of Side Effects: Retries duplicate work. For pure text that is harmless; for an LLM that sends email, charges cards, or files tickets, it is a production incident. This lesson shows how to attach idempotency keys, dedup at the receiver, and keep tool-calling agents safe.
- JSON Schema Enforcement: Turning LLM Output Into Data You Can Trust: Downstream code needs structured data, but LLMs return strings. This lesson builds the three-layer defense that gets you to a validated object every time: provider-side JSON mode, schema validation, and a bounded repair loop.
- Rate Limits and Queuing: Staying Under the Provider's Ceiling Without Stalling Users: Every LLM provider has a requests-per-minute and tokens-per-minute ceiling. This lesson explains how those limits work, why token buckets are the right client-side model, and when to queue requests instead of failing them.
- Streaming and UX: Making a Slow API Feel Fast: An LLM call that takes 8 seconds feels broken without streaming and tolerable with it. This lesson covers the SSE plumbing, the UX patterns that actually help, and the failure modes streaming introduces.
- Cost Observability: Knowing What Each Call, Feature, and User Is Spending: LLM bills surprise teams not because the unit cost is high but because the visibility is low. This lesson builds the tagging, rollup, and alert setup that turns a monthly bill into a daily dashboard.
- Latency Observability: Seeing Where an LLM Call Actually Spends Its Time: Average latency hides everything interesting about LLM calls. This lesson covers the percentiles that matter, the time-to-first-token metric that is specific to streaming, and the span layout that makes root cause obvious.
- Logging and Privacy: Keeping the Evidence You Need Without the Data You Shouldn't: LLM logs are the most useful debugging artifact you can keep and the most dangerous privacy liability. This lesson sets the ground rules: what to log, what to redact, and how to separate the debug trail from the content trail.
- Prompt Versioning and Rollouts: Treating Prompts Like Code You Ship Carefully: A prompt edit is a behavior change that goes to every user at once. This lesson shows how to version prompts, gate rollouts, compare versions, and roll back cleanly when a new version quietly makes things worse.
Module 6 · Embeddings & Vector Search
Vectors, cosine similarity, embedding choice, vector DB tradeoffs, chunking. This is where many RAG systems fail first.
- Vectors as Meaning: What an Embedding Is, How to Make One, and Why It Works: A deep, hands-on first pass at embeddings. What a vector actually looks like in memory, how a transformer produces it, how to generate one yourself in Python and TypeScript, and the math that turns meaning into geometry.
- Why Keyword Search Can't Find Meaning: The motivating failure for everything in this module. A working support search, a right-on-target help article, and a query that returns nothing because keyword search is matching spellings, not meaning.
- Embeddings: What They Are and Where They Come From: An embedding is a list of numbers that places a piece of text somewhere on a map of meaning. This concept builds the mental model with one real vector, a coordinates analogy, two runnable ways to produce your own (hosted API and local library), and the one compatibility rule that has burned every team at least once.
- Choosing an Embedding Model: Dimension, Quality, Domain, Cost: The embedding model you pick is the map your retrieval system lives on. This lesson walks through the real tradeoffs: dimension count, benchmark scores, domain fit, multilingual coverage, cost, and the migration problem nobody plans for.
- How Similarity Works, Visually: Every embedding-based search ends with one score: how aligned are two arrows in high-dimensional space. This concept builds the intuition with real scores from a real model, names the measurement (cosine similarity), and calls out the topic-versus-intent trap that bites everyone at least once.
- Chunking: Splitting Text Before You Embed: You can't embed a 200-page manual as one vector and expect search to work. This concept covers the three chunking strategies you'll actually use, the size trade-off between precision and context, why people add overlap, and a sensible default to start from.
- Vector Databases: Storing and Searching Meaning: A vector database stores embeddings and answers 'what is closest to this?' efficiently. This concept covers the three jobs a vector DB actually does, a runnable pgvector example, the landscape (pgvector, Pinecone, Qdrant, Weaviate, Chroma, FAISS), and what every query will need from you.
- Semantic Search in One Picture: The closing concept for the module. Chunks, embeddings, a vector database, and a similarity score all come together into a single request-time pipeline. One diagram, one end-to-end snippet, a note on score thresholds and the relevance cliff, and the handoff to RAG in Module 7.
Module 7 · Advanced RAG
Beyond naive top-k: query translation (HyDE), reranking, semantic chunking, multi-modal ingestion, RAG vs fine-tuning.
- What RAG Is, and Where Naive RAG Breaks: Module 6 ended with top-k chunks. This concept turns those chunks into an answer, walks the naive five-step RAG pipeline, and shows the six places it falls over in production. Each failure mode previews a later concept in this module.
- Query Translation: Reshape the Question Before You Retrieve: A short user query and a long source document are different genres of text, and embedding similarity feels that difference. Query translation closes the shape gap with one extra LLM call before retrieval. This concept covers the four techniques that actually move the needle: HyDE, query rewriting, decomposition, and multi-query expansion, with step-back prompting as a niche fifth.
- Reranking: From Similar Chunks to Actually Relevant Ones: Embedding similarity gets you in the right neighborhood, not always to the right address. A reranker scores each candidate chunk against the query directly and reorders the shortlist by genuine relevance. This concept covers cross-encoder rerankers, the retrieve-more-then-rerank pattern, hosted vs self-hosted options, and LLM-as-reranker for tiny lists.
- Prompt Assembly: Grounded Answers, Citations, and Knowing When to Say "I Don't Know": After retrieval and reranking you have three to five clean chunks. Turning them into an answer the user can trust is a separate problem. This concept covers the shape of a RAG prompt, how to order chunks so the model actually reads them, how to ask for citations, and how to handle the two opposite failure modes: the LLM hallucinating when context is thin, and the LLM refusing when context is implicit.
- Evaluating RAG: Measuring Retrieval and Answers Separately: Without evaluation, you cannot tell whether a wrong answer means retrieval is broken or that the corpus does not contain the answer. This concept covers the two-layer eval (retrieval recall plus answer quality), how to build a small gold set in an afternoon, recall@k as the workhorse retrieval metric, LLM-as-judge for answer quality, and how to use eval to choose between the upgrades from earlier concepts.
- RAG or Fine-Tuning: Picking the Right Lever: RAG and fine-tuning solve overlapping but different problems, and a lot of teams reach for the wrong one. This concept frames the decision: when fresh facts are the constraint (RAG), when style or format is the constraint (fine-tuning), when both apply, and the cost picture for each.
Module 8 · Agents & Tool Use
The agent loop, tool design, agent memory, orchestration patterns, and what it takes to run agents in production.
- What an Agent Actually Is: An LLM with a Way to Act: An agent is not magic. It is a regular LLM call wrapped in a loop, with a list of tools the model can call. This concept builds a working 30-line agent against a real API, walks through the trace the model and runtime exchange, and tells you when you actually need one (and when you do not).
- Tool Design: The Most Important Skill in Building Agents: An agent is only as good as the tools you hand it. This concept walks through how to turn a real backend API into a set of tools the model can actually use: naming, descriptions, argument schemas, return shapes, and the kinds of subtle mistakes that make a model hallucinate calls or get stuck.
- The Agent Loop: Step Budgets, Termination, and Error Recovery: An agent that can call tools needs a loop, and the loop is where most production agents go wrong. This concept shows how to bound the loop with step and token budgets, how to detect when the agent is stuck, how to recover from tool errors, and how to make the whole thing observable.
- Agent Memory: Scratchpad, Sessions, and Long-Term Recall: Agents need to remember things, but you cannot just keep stuffing the conversation back into the prompt. This concept walks through three layers of memory: the scratchpad inside one run, the session that carries across turns of a chat, and long-term memory that survives forever, with code for each.
- Agent Orchestration: Routers, Supervisors, and Sub-Agents: One agent with one tool list works for narrow tasks. Real systems often need more than one agent, with something deciding which one to invoke. This concept walks through the four orchestration patterns you actually see in production: routers, planner/executor, supervisor with sub-agents, and parallel fan-out, with code for each.
- Agents in Production: Sandboxing, Approval, Audit, and When Not to Ship One: An agent that works on the happy path is not a production agent. This concept covers what changes when real users hit the system: sandboxing tool execution, human approval for risky actions, audit logs that satisfy a compliance review, cost caps, and the cases where shipping an agent is the wrong choice.
Module 9 · Model Context Protocol
The N×M integration problem, MCP protocol basics, building a server, transports and auth, MCP in production.
- The Model Context Protocol: A Common Plug for Tools: MCP (Model Context Protocol) is an open standard for exposing tools to LLM clients. Once your tools speak MCP, any compatible client (Claude Desktop, IDEs, your own agents) can use them without bespoke integration code. This concept walks through what MCP is, why it exists, and the message shape you actually have to know.
- Building an MCP Server: From Empty File to Claude Desktop: We build a working MCP server end to end, exposing the same support tools from Module 8 (lookup_user, search_docs, create_ticket), and connect it to Claude Desktop so you can see the model call your tools through the protocol. TypeScript SDK, stdio transport, real handlers.
- MCP Transports and Auth: From Local stdio to a Multi-User HTTP Server: Stdio is fine for one user on one laptop. The moment a server runs on a different machine, or serves more than one user, you need HTTP and you need authentication. This concept walks through the two transports MCP supports, when to use each, and how to wire up auth that scales past a single developer.
- MCP in Production: Rate Limits, Observability, and Safe Deploys: An MCP server that works for one developer is not the same as an MCP server that runs for a team or for customers. This concept walks through the operational layer: per-tool rate limits, what to log, how to roll out tool changes without breaking running clients, and the security mistakes that show up in real deployments.
- MCP vs Direct Tools: When to Reach for Each: Both shapes work. In-process tools (Module 8) are simpler, faster, and easier to evolve. MCP servers (this module) are portable, multi-client, and operable as a service. This concept walks through the decision: which to choose, when, and how to migrate from one to the other when the need changes.
Module 10 · Guardrails & Safety
Defense in depth: prompt injection, PII, jailbreaks, input/output filtering, red-team and safety evals.
- Trust Boundaries: Where Untrusted Input Enters: Before you write a single defense, you have to know where untrusted input enters your system. Most LLM teams get this wrong by missing the second channel: text the tools fetch on the model's behalf. This concept maps the trust boundary, names the two channels, and gives you a threat-modeling lens you can apply to any LLM application.
- Input Filtering: The Cheap First Layer: Length caps, blocklists, and URL allowlists at the front door. Input filters are not your primary defense, but they catch the obvious bad stuff before you spend a single model call on it. This concept covers what input filters can and cannot do, and how to wire one in without making it the only thing standing between users and your tools.
- Tool-Side Authorization: The Layer That Does the Work: If only one defense survives review, this is the one. Tool-side authorization makes dangerous tools refuse to perform dangerous actions even when the model asks nicely. This concept covers parsed-URL allowlists, recipient pinning, scoped reads, and how to decline cleanly so the agent loop can recover.
- Output Filtering and PII: The Last Line, Honestly Named: Output filters scrub the model's reply on the way out: secret patterns redacted, fake protocol turns blocked, length capped. They are useful and they are a band-aid. This concept covers what to filter, why this is defense in depth rather than your primary control, and how to keep the filter from becoming a comfortable place to hide.
- Red Teaming as a Practice, Not an Event: An attack suite you run on your own system, kept fresh as the system changes. This concept covers what to put in the suite, who runs it, how often, and what residual risk to write down. Without this, your defenses degrade silently the first time someone changes a tool.
Module 11 · Evaluation
Offline and online metrics, LLM-as-judge, golden sets, regression testing, and what “good” means for your task.
- Why Evals, Not Vibes: Most teams ship LLM features by reading a few model outputs and saying "yeah, looks good." That works on day one and fails on day forty. This concept is the case for evals: what shipping-by-vibes gets wrong, what an eval gives you that ad-hoc inspection cannot, and what the cheap minimum looks like.
- Building a Fixture Set That Tells You Things: A fixture set is the inputs you measure your feature against. Bad fixtures give comforting numbers and miss real failures. Good fixtures bias toward where the system is likely to be wrong. This concept covers what goes in a fixture, where to find good inputs, what gold answers look like for different task shapes, and how to grow the set over time.
- Deterministic Metrics and LLM-as-Judge: Two scoring shapes, both useful, neither sufficient on its own. Deterministic metrics are free, fast, and brittle. LLM-as-judge metrics are slower, costlier, and capture quality the regex cannot. This concept covers when to reach for each, how to build a judge prompt that does not lie to you, and how to combine them in one report.
- Comparing Prompt Variants Without Lying to Yourself: Run multiple prompt variants in one harness, on the same fixtures, with the same metrics. Read the per-metric and per-category numbers, not the aggregates. This concept walks through how to set up the comparison, what regressions look like in the report, and how to tell a real win from noise.
- Evaluating Agents: Judging the Path, Not Just the Answer: Agents do not produce a single response; they produce a sequence of tool calls and intermediate decisions ending in a final answer. Outcome-only eval misses the lucky agent that stumbled into the right answer and the wasteful agent that took ten tool calls when three would have done. This concept covers trajectory eval, the metrics that matter for agents specifically, how to extend Module 8's audit log into an eval fixture set, and how to read trajectory and outcome scores together.
- Eval-Driven Iteration: Closing the Loop: An eval is only useful if it changes how you work. This concept covers gating merges on the eval, growing the fixture set from real failures, retiring stale fixtures, watching metric drift across model upgrades, and the cultural habits that keep the practice alive past the first month.
Module 12 · Open-Weight Models
Open vs closed, model families, quantization, local vs GPU serving, and when self-hosting pays off.
- When Self-Hosting an Open-Weight Model Pays Off: Open-weight models let you run inference on your own hardware. That sounds attractive until you do the math. This concept covers the workload shapes where self-hosting genuinely wins, the ones where it does not, and the specific numbers that flip the answer.
- Picking an Open-Weight Model: Size, Quantization, License: Open-weight model picking sounds technical and is mostly economic. This concept walks through the size classes (3B, 7B, 13B, 70B), the quantization choices (full precision, q8, q4) and what they actually do to memory and quality, and the license traps that catch teams shipping with the wrong weights.
- Serving Stack Options: Ollama, vLLM, TGI, llama.cpp: Once you have a model file, you need something to actually serve it. The good news is the well-known stacks all speak the same OpenAI-compatible API. The differences live in throughput, batching behavior, and where they run. This concept walks through what each one is for and which to pick.
- Cost and Throughput: The Numbers That Decide: Tokens per second, requests per second, and dollars per million tokens — measured on your actual hardware, not from vendor blog posts. This concept walks through how to measure throughput honestly, how to convert it into cost, and the surprises that bite teams who skipped this step.
- Hybrid: Local for the Easy 80%, Hosted for the Hard 20%: Most production teams that run open-weight models in serious volume end up hybrid: a small self-hosted model handles the easy bulk of requests, a hosted frontier model handles the hard tail. This concept covers the routing decision, how to pick the threshold, and the operational lessons from running both sides.
Module 13 · Inference Infrastructure
Serving at scale: inference engines, latency anatomy, capacity and rate limits, honest cost-per-token math.
- Serving Runtimes and Continuous Batching: What actually runs the model on the GPU when a request comes in. Continuous batching, paged KV cache, and speculative decoding are the three ideas that turn a 30-token-per-second stream into a 1000-token-per-second aggregate. This concept covers how they work, what each runtime does best, and what to test for in your specific workload.
- Latency Anatomy: TTFT, ITL, and What the User Feels: "Latency" for LLM endpoints is three numbers, not one. Time-to-first-token, inter-token latency, and total wall-clock each map to a different part of the user experience. This concept covers what each measures, which one to optimize when, and how to read p50/p95/p99 honestly.
- Serving Shapes: Sync, Pooled, Batch: How you submit work to an inference endpoint changes both your latency and your unit cost more than you would think. Three shapes are worth knowing in detail: synchronous one-at-a-time, client-pooled with bounded concurrency, and offline batch. This concept covers when each fits, how they look in code, and where the wrong choice silently triples your bill.
- Capacity, Rate Limits, and Headroom: Every inference endpoint has limits — requests per minute, tokens per minute, concurrent requests, daily spend caps. This concept covers how those limits actually work, how to size your headroom so you do not get throttled at peak, and how to build a backoff path that keeps the user experience stable when you do run hot.
- Cost Per Token, Honestly: The rate card is the smallest part of an AI bill. Retries, failed calls, idle GPU hours, embedding pipelines, observability traces, and the input tokens nobody counted all add up. This concept covers what to add to the per-token quote to get a number that matches the invoice — and how to forecast cost before you ship.
Module 14 · LLM Internals
Inside the model: the forward pass, attention mechanics, positional encodings, why context length is hard, tokenizer design tradeoffs.
- Inside a Forward Pass: From Token IDs to the Next Token: Trace one forward pass through a modern LLM: the embedding lookup, the residual stream, 32 identical blocks, and the unembedding that turns vectors back into a token. These are the numbers behind every latency cliff and capacity decision you make when serving.
- Attention Mechanics: Queries, Keys, Values, and the One Formula: Open up the attention sublayer: what Q, K, and V actually are, a worked numeric example you can check by hand, the causal mask, multi-head attention, FlashAttention, and the PyTorch implementation you will reuse in the capstone.
- Positional Encodings: Order-Blind Attention and the RoPE Era: Attention treats your prompt as a bag of tokens, so position has to be injected. Learned positions and their hard ceiling, why RoPE became the default in every modern open model, and what "extended to 128k" in a model card actually means.
- Why Context Length Is Hard: Quadratic Attention and the KV Cache: The two real costs behind every context window: prefill compute that grows quadratically and a KV cache that pins gigabytes of GPU memory per request. Worked math on Llama-3-8B, why GQA exists, sliding windows, and lost-in-the-middle.
- Tokenization Tradeoffs: Vocabulary Size, the Language Tax, and Lock-In: Vocabulary size is a design dial with no neutral setting: it shapes your bill, your effective context, how fairly the model treats non-English users, and why models are weird at arithmetic. What changed between Llama-2's 32k and GPT-4o's 200k, and what it means for prompts and pricing.
Module 15 · Fine-Tuning
What tuning actually changes: SFT, LoRA and QLoRA, preparing data, why fine-tuning sometimes makes models worse, evaluating a tune.
- What Fine-Tuning Actually Changes: Behavior, Not Knowledge: Supervised fine-tuning is the same next-token training as pretraining, run on a tiny curated dataset at a low learning rate. This concept covers what that reliably changes (style, format, task shape), what it does not (facts), and the training pipeline every production model has already been through.
- LoRA and QLoRA: Fine-Tuning Inside a Real Memory Budget: Full fine-tuning an 8B model needs around 128 GB of GPU memory; LoRA gets nearly the same behavioral result by training a low-rank delta with under 1% of the parameters, and QLoRA fits the whole job on a single 24 GB GPU. How both work, the defaults that matter, and where the training run actually happens.
- Preparing Tuning Data: The Dataset Is the Product: The training run is a commodity; the dataset decides whether the tune ships. Chat-format JSONL, the verbatim quality bar, where examples come from, dedup, label consistency, contamination, and how much data each task type actually needs.
- Why Fine-Tuning Makes Models Worse: Forgetting, Regression, and Narrowing: Nothing in the training loss says "keep your old skills." Catastrophic forgetting, safety regression on benign data, distribution narrowing, memorization, and format brittleness: how each failure works, and the mitigations that actually address the mechanism.
- Evaluating a Fine-Tune: Help the Target, Hurt Nothing Else: The question is never "did the tune help"; it is "did it help the target task and not hurt everything else." Target-task win rates, the regression suite as a ship gate, the honest economics against a prompted larger model, statistical sample sizes, and when to re-tune.
Module 16 · Capstone: Build a Mini LLM
Train your own GPT from scratch on your laptop: BPE tokenizer, attention, training loop, sampling, and a fine-tune that demonstrates catastrophic forgetting.
- Capstone Spec and Setup: The Mini LLM You Are About to Build: The capstone spec in full: a 1-4M parameter GPT you train on your own laptop, the five stages that build it, and how to set up and work the lab. Plus honest expectations about what a 2M-parameter base model can and cannot do.
- Reading a Training Run: What the Loss Is Telling You: Your training loop prints loss numbers and nothing tells you if they are good. This concept makes you fluent: what cross-entropy loss means, a calibration table mapping loss to sample quality for this lab, the four curve pathologies and their fixes, and when to stop.