AI glossary · 718 terms

Every AI word, explained simply

Plain-words meanings for every key word in the Learn AI course. Each one links to the lesson that teaches it with an animation and a mini tool.

718 terms

A

Ablation

Switching a part off to see what changes.

Learn: Interpretability ›
Absolute path

A path written in full from the top of the disk, such as /home/you/file.py or C:\Users\you\file.py.

Learn: Connect your MCP server to an AI app ›
Accelerator

A chip built to do a huge number of simple calculations at the same time, such as a GPU.

Learn: GPUs, TPUs and accelerators ›
Accuracy

Right answers out of all answers.

Learn: Accuracy, precision and recall ›
Action

A tool call the agent makes.

Learn: Reason + act (ReAct) ›also in 1 more lesson
Activate

Tell the terminal to use that private box.

Learn: Set up Python and a virtual environment ›
Activation

The neuron's output after the activation function.

Learn: The forward pass ›also in 1 more lesson
Activation function

A fixed bend applied to a neuron's sum.

Learn: Activation functions ›
Activations

The numbers a layer outputs.

Learn: Batch and layer normalisation ›
Active parameters

The weights actually used for one token, as opposed to the total.

Learn: Mixture of experts ›
Active recall

Trying to remember something instead of re-reading it.

Learn: Learn anything with an AI tutor ›
Adam

Momentum plus a separate adaptive step size for each parameter.

Learn: Optimisers: SGD, momentum, Adam ›
Adapter

The small saved set of LoRA weights; you load it on top of the base model.

Learn: Fine-tune a small model with LoRA ›
Adversarial

Working against each other, like players in a contest.

Learn: GANs: generator vs discriminator ›
Agent

The learner that chooses actions.

Learn: Reinforcement learning ›also in 2 more lessons
Agent loop

Think, act, observe, repeat, until the goal is met.

Learn: The agent harness ›also in 1 more lesson
AGI

Artificial general intelligence: AI that matches people across most tasks.

Learn: AGI and the road ahead ›
AI feedback

Preferences produced by a model instead of a person.

Learn: Constitutional AI and AI feedback ›
AI safety

Work to keep powerful systems reliable and under human control.

Learn: AGI and the road ahead ›
AI winter

A period when interest and funding for AI fall after promises are not met.

Learn: AI winters ›
AlexNet

The deep network that won in 2012 and started the boom.

Learn: ImageNet and AlexNet ›
Alignment

Which output word matches which input word.

Learn: Attention is born ›also in 1 more lesson
Amino acid

One of 20 kinds of building block in proteins.

Learn: AI for science: protein folding ›
Analogy

A is to B as C is to ?

Learn: Word vectors (word2vec) ›
Angle

How far apart two arrows point, in degrees.

Learn: Cosine similarity ›
API

A way for one program to send a request to another and get a reply.

Learn: Open-weight vs closed models ›also in 1 more lesson
API key

A secret string that proves a request comes from your account.

Learn: Use a hosted model API safely ›
Approximate

To match closely without being exactly equal.

Learn: Why networks can learn almost anything ›
Approximate search

Faster search that may skip a few true matches.

Learn: Vector databases ›
Arguments

The inputs given to a tool, such as the expression to calculate.

Learn: Tool use and function calling ›
Artificial intelligence

Machines doing tasks that normally need human intelligence.

Learn: What is AI? ›also in 1 more lesson
Attention

A way for the decoder to look at all the input words and focus on the useful ones.

Learn: Attention is born ›also in 2 more lessons
Attention map

A grid of all the weights: rows look, columns are looked at.

Learn: Self-attention ›
Attention weight

How much one token listens to another, from 0 to 1.

Learn: Self-attention ›
Attention weights

Shares, adding up to 100%, showing how much each input word counts right now.

Learn: Attention is born ›
Attribution

Saying where a claim came from.

Learn: Grounding and citations ›
Autoregressive

Generating one token at a time, each based on the ones before.

Learn: Causal masking ›also in 1 more lesson
Average pooling

Keeps the mean of each block.

Learn: Pooling ›

B

Backdoor

A hidden trigger that makes a model misbehave only on a special pattern.

Learn: Data poisoning and model theft ›
Backpropagation

Working out every weight's gradient by sending the error backwards.

Learn: Backpropagation ›also in 1 more lesson
Bag of words

Words with no order, like cards in a pile.

Learn: Position: how models know word order ›
Bagging

Training each model on its own bootstrap sample, then combining by vote.

Learn: Random forests and ensembles ›
Base model

A pretrained model that just continues text.

Learn: Base models vs chat models ›also in 1 more lesson
Batch

A small group of examples used for one update.

Learn: Epochs, batches and steps ›also in 1 more lesson
Batch norm

Normalises one feature across many examples in a batch (2015).

Learn: Batch and layer normalisation ›
Batch size

How many examples are in each batch.

Learn: Epochs, batches and steps ›
Benchmark

A shared test that lets everyone compare results fairly.

Learn: ImageNet and AlexNet ›also in 1 more lesson
BERT

Bidirectional Encoder Representations from Transformers: an encoder model from 2018.

Learn: BERT: reading both ways ›
Bias

An extra number that shifts the line without any input.

Learn: The perceptron ›also in 4 more lessons
Bidirectional

Looking both left and right of a word.

Learn: BERT: reading both ways ›
Bit

A unit for measuring surprise. Each extra bit of surprise doubles how unlikely the word was.

Learn: Perplexity ›also in 1 more lesson
Blank

A label meaning “nothing to add here”.

Learn: Speech recognition ›
Bootstrap sample

A random copy of the data drawn with repeats.

Learn: Random forests and ensembles ›
Bottleneck

The narrow middle layer with few numbers.

Learn: Autoencoders ›also in 1 more lesson
Bounding box

A rectangle that marks where an object is.

Learn: Object detection ›
BPE

Byte-pair encoding: a way to build a vocabulary by repeatedly merging the most frequent pair.

Learn: How tokenizers are built (BPE) ›
Brittle

Works well inside its rules, then fails suddenly outside them.

Learn: Expert systems ›
β (beta)

A setting that scales how strongly the gap counts.

Learn: Direct preference optimisation ›

C

Cache hit

A request whose start was found in the cache and reused.

Learn: Prompt caching ›
Cache miss

A request whose start had to be worked out again.

Learn: Prompt caching ›
Caching

Reusing work already done on a repeated prompt start instead of paying for it again.

Learn: The cost of serving AI ›
Carbon intensity

Grams of CO₂ released per kWh of electricity.

Learn: Energy and the environment ›
Causal mask

A mask that hides later tokens from each token.

Learn: Causal masking ›
Cell state

The memory belt that carries information along.

Learn: LSTM: long short-term memory ›
Centroid (centre)

The average position of the points in a cluster.

Learn: Clustering (unsupervised) ›
Chain of thought

Writing out intermediate reasoning steps before the final answer.

Learn: Chain of thought ›
Chain rule

The total slope of a chain of steps is the product of each step's slope.

Learn: The chain rule, intuitively ›
Character-level model

A model that reads and predicts one character at a time.

Learn: Train a tiny model from scratch ›
Chat model

A model tuned to follow instructions and hold a conversation.

Learn: Base models vs chat models ›
Chat template

The recipe that turns messages into one tagged stream.

Learn: System prompts and chat roles ›
Check script

A small program that tests your setup and reports PASS or FAIL.

Learn: Check your workbench end to end ›
Chunk

A piece of a document, stored and searched on its own.

Learn: Chunking documents ›also in 1 more lesson
Chunk size

How long each piece is, in words or tokens.

Learn: Chunking documents ›
Circuit

A small group of connected features that does one job.

Learn: Interpretability ›
Citation

A marker that points to the source passage behind a claim.

Learn: Grounding and citations ›
Classification

Predicting a category, like spam or not spam.

Learn: Labels and supervised learning ›
Classifier

A model that puts an input into one of a set of labels.

Learn: Teach a model to recognise your photos ›
Client

The AI app that asks for tools.

Learn: Model Context Protocol (MCP) ›
Closed model

A model kept by its maker and used only through a service.

Learn: Open-weight vs closed models ›
Cluster

A group of points that sit close together.

Learn: Clustering (unsupervised) ›also in 1 more lesson
CNN

A network built from stacked convolution layers, made for pictures.

Learn: Convolutional neural networks ›
Code (latent)

The small list of numbers the encoder produces.

Learn: Autoencoders ›
Cold start

Having no history yet for a new person or item.

Learn: Recommendation systems ›
Collaborative filtering

Recommending by using the choices of many people.

Learn: Recommendation systems ›
Collection

One named set of items inside the database, such as all your notes.

Learn: Set up a local vector database ›
Compaction

Shortening old text so more fits.

Learn: Context engineering ›
Complexity

How flexible a model is. Here, the degree of the curve.

Learn: Overfitting and underfitting ›
Compliance

Following the rules that apply to you.

Learn: AI regulation ›
Composition

Feeding the output of one function into the next.

Learn: The chain rule, intuitively ›
Compute

The amount of calculation needed, which drives time and cost.

Learn: Training vs inference ›also in 1 more lesson
Computer use

An agent operating apps through the screen, pointer and keyboard.

Learn: Computer-using agents ›
Concatenate

Glue several lists of numbers end to end.

Learn: Multi-head attention ›
Confidence score

How sure the detector is about a box and its label.

Learn: Object detection ›also in 1 more lesson
Config entry

The settings lines that tell an app how to start one MCP server.

Learn: Connect your MCP server to an AI app ›
Confusion matrix

A grid that counts how often each predicted answer met each real answer.

Learn: The confusion matrix ›
Constraint

A limit on the answer, such as length, tone or format.

Learn: Prompt basics ›
Content credential

Signed information attached to a file about where it came from and how it was edited.

Learn: Deepfakes, watermarks and provenance ›
Context

The words around a word.

Learn: Word vectors (word2vec) ›also in 5 more lessons
Context engineering

Building the model's input on purpose: facts, instructions and tools.

Learn: Context engineering ›
Context vector

The single fixed-size vector passed from encoder to decoder.

Learn: Sequence to sequence ›
Context window

How many tokens a model can hold in view at once.

Learn: Tokens across languages ›also in 3 more lessons
Contrastive learning

Pull true pairs together, push wrong pairs apart.

Learn: CLIP: matching images and text ›
Contribution

The small change one layer adds to the stream.

Learn: The residual stream ›
Convolution

Sliding a filter over a picture, multiplying and adding at each spot.

Learn: Convolution ›
Cooling overhead

Extra power spent keeping chips cool.

Learn: Energy and the environment ›
Copyright

The legal right of a creator to control copies of their work.

Learn: Copyright and training data ›
Cosine similarity

A score from −1 to 1 for how closely two vectors point the same way.

Learn: Cosine similarity ›also in 1 more lesson
Cost per million tokens

The usual way prices are quoted: a price for every million tokens processed.

Learn: The cost of serving AI ›
CPU

A processor with a few fast, flexible workers.

Learn: GPUs and the deep learning boom ›
Critique

A note saying how a draft breaks a principle.

Learn: Constitutional AI and AI feedback ›
Cross-attention

Attention where the decoder looks at the encoder’s output.

Learn: Encoders, decoders and both ›
CTC

The rule that squashes repeats and drops blanks, so the audio needs no hand-marked timings.

Learn: Speech recognition ›

D

Dartmouth workshop

The 1956 summer research project that named the field.

Learn: The birth of 'artificial intelligence' ›
Data leakage

When test information sneaks into training, making scores look better than reality.

Learn: Train, validation and test sets ›
Data poisoning

Slipping bad examples into training data to change what a model learns.

Learn: Data poisoning and model theft ›
Data vs command

Content to read, versus an instruction to obey.

Learn: Prompt injection ›
Dataset

All the examples together, as a table.

Learn: Data and features ›also in 2 more lessons
Decision map

The map shaded by what a new point would be called at each place.

Learn: k-nearest neighbours ›
Decision tree

A model made of yes/no questions that lead to an answer.

Learn: Decision trees ›
Decoder

The half that rebuilds the input from the code.

Learn: Autoencoders ›also in 3 more lessons
Decoder-only

A model built only from decoder blocks, like GPT-style models.

Learn: Encoders, decoders and both ›
Decoding

Turning the model's word odds into actual text, one word at a time.

Learn: Greedy decoding vs sampling ›
Deep learning

Training networks with many layers.

Learn: Recap: the AI timeline ›
Deep network

One with many layers.

Learn: Residual connections (ResNet) ›
Deep reinforcement learning

Learning from rewards, such as winning, using deep neural networks.

Learn: AlphaGo and deep reinforcement learning ›
Deepfake

Media made or altered by AI to pass as real.

Learn: Deepfakes, watermarks and provenance ›
Defence in depth

Using several different layers of protection.

Learn: Jailbreaks and misuse ›
Denoiser

The network that guesses a cleaner picture from a noisy one.

Learn: Diffusion models ›
Dense vector

A short vector where most or all entries are non-zero numbers.

Learn: One-hot vs dense vectors ›
Dependency

A task that must finish before another can start.

Learn: Planning and subagents ›also in 1 more lesson
Derivative

Another word for slope: the rate of change.

Learn: The chain rule, intuitively ›
Dictation

Speaking so that software types what you say.

Learn: Talk to AI: voice and dictation ›
Diff

A view of what changed: lines removed and lines added.

Learn: Coding agents ›
Differential privacy

Adding noise so the result barely changes if any one person is included or not.

Learn: Privacy-preserving AI ›
Diffusion

Starting from noise and cleaning it up in steps to form a picture.

Learn: Make images on your computer ›
Dimension

One feature or direction of measurement.

Learn: Dimensionality reduction ›also in 2 more lessons
Dimensionality reduction

Describing data with fewer features while keeping its main shape.

Learn: Dimensionality reduction ›
Disclosure

Telling others that AI helped.

Learn: Using AI responsibly ›
Discriminator

The network that scores examples as real or fake.

Learn: GANs: generator vs discriminator ›
Disk space

Storage for files, including downloaded models.

Learn: How the hands-on tutorials work ›
Distance

How far apart two points are. Close means similar.

Learn: k-nearest neighbours ›
Diverge

When steps keep growing and the loss explodes instead of falling.

Learn: Learning rate ›also in 1 more lesson
Docstring

The text in triple quotes under a function; for a tool, it tells the model what the tool is for.

Learn: Build a simple MCP server ›
Domain randomisation

Varying friction, lighting and other settings during practice so the skill becomes robust.

Learn: Robots that learn ›
Dot product

The sum of multiplying matching numbers. Divide it by both lengths and you get cosine.

Learn: Cosine similarity ›also in 1 more lesson
Downsampling

Making something smaller by keeping fewer numbers.

Learn: Pooling ›
Draft

A first version that you plan to improve.

Learn: Write and edit with AI ›
Dropout

Randomly switching off neurons during training.

Learn: Dropout ›
Dropout rate

The chance that each neuron is off on a given step, such as 0.3 for 30%.

Learn: Dropout ›
Duplicate

The same record appearing more than once.

Learn: Collecting and cleaning data ›
Duration

How long a sound lasts.

Learn: Text to speech ›

E

Edit pass

One round of changes with a single aim, such as shortening.

Learn: Write and edit with AI ›
Efficiency

Doing the same work with less energy.

Learn: Energy and the environment ›
ELIZA

A 1966 chatbot by Joseph Weizenbaum that used pattern matching.

Learn: ELIZA, the first chatbot ›
ELIZA effect

Seeing understanding in a program that only follows patterns.

Learn: ELIZA, the first chatbot ›
Embedding

A learned dense vector for a word or token.

Learn: One-hot vs dense vectors ›also in 3 more lessons
Embedding model

A small model whose only job is to turn text into embeddings.

Learn: Make embeddings on your computer ›
Emergent ability

A skill that seems absent in small models and present in larger ones.

Learn: Emergent abilities ›
Encoder

The half that squeezes the input into a code.

Learn: Autoencoders ›also in 3 more lessons
Encoder-decoder

An encoder reads the input, a decoder writes the output.

Learn: Encoders, decoders and both ›
End token

A special token the model writes when it thinks the answer is complete.

Learn: Stop sequences and max tokens ›
Ensemble

A group of models whose answers are combined.

Learn: Random forests and ensembles ›
Environment variable

A named value kept by your computer outside your files.

Learn: Use a hosted model API safely ›
Epoch

One full pass over all the training data.

Learn: Epochs, batches and steps ›
Epoch (round)

One full pass through all the training photos.

Learn: Teach a model to recognise your photos ›
Epsilon

The chance of trying a random move instead of the best known one.

Learn: Reinforcement learning ›
Equal opportunity

Qualified people in each group have a similar chance of being chosen.

Learn: Bias and fairness ›
Error

The gap between a prediction and the true answer for one example.

Learn: Loss: measuring mistakes ›
Error signal

The blame passed back to a neuron from the layer after it.

Learn: Backpropagation ›
Eval

A test you run to check a model for a purpose. Short for evaluation.

Learn: Benchmarks and evaluations ›also in 1 more lesson
Exact match

Counts an answer only if every part is right.

Learn: Emergent abilities ›
Example

One piece of data, often with the right answer attached.

Learn: Rules vs learning ›also in 3 more lessons
Exit code

A number a program returns: zero for success, non-zero for failure.

Learn: Check your workbench end to end ›
Expert

One of several parallel sub-networks inside a block.

Learn: Mixture of experts ›
Expert system

A program that answers questions in one field using rules written from a specialist's knowledge.

Learn: Expert systems ›
Exploding gradient

The signal grows huge and updates overshoot.

Learn: Vanishing and exploding gradients ›
Extended thinking

A mode where a model is allowed a longer scratchpad.

Learn: Thinking tokens and extended thinking ›

F

Fact-check

Confirming a claim with a trusted source.

Learn: Hallucination ›
Failing test

A check that does not pass yet. It is the agent’s clue.

Learn: Coding agents ›
Fair use

A US-style idea that allows some uses without permission; other countries use different rules.

Learn: Copyright and training data ›
Fake media

Pictures, audio or video made or changed to mislead.

Learn: Spot AI scams and fake media ›
False alarm

A harmless request that gets blocked by mistake.

Learn: Jailbreaks and misuse ›also in 1 more lesson
False negative (miss)

Predicted no but it was yes.

Learn: The confusion matrix ›
False positive (false alarm)

Predicted yes but it was no.

Learn: The confusion matrix ›
Feature

One measurable property of every example. A column in the table.

Learn: Data and features ›also in 4 more lessons
Feature map

The grid of answers the filter produced.

Learn: Convolution ›also in 1 more lesson
Federated learning

Training a shared model on many devices, sending only updates, not data.

Learn: Privacy-preserving AI ›
Feed-forward

The part of a layer that mixes each token's numbers on its own.

Learn: Parameters and model size ›
Feed-forward block

A small neural network applied to each token separately after attention.

Learn: The feed-forward block ›
Feedback

A comment on your own answer that shows what to fix.

Learn: Learn anything with an AI tutor ›
Feedback tuning

Further training using people's ratings of replies.

Learn: Base models vs chat models ›
Few-shot

A prompt with a handful of examples.

Learn: Zero-shot and few-shot prompting ›
Filter (kernel)

A small grid of weights that looks for one pattern.

Learn: Convolution ›
Filtering

Removing examples that fail checks.

Learn: Synthetic data ›
Fine-tuning

Training a pretrained model a little more on your own data.

Learn: Transfer learning ›also in 4 more lessons
Finish reason

Which stop sign ended the reply.

Learn: Stop sequences and max tokens ›
Flask

A small, popular Python library for writing web servers.

Learn: Give your RAG app a simple web page ›
Flicker

Unwanted jumping of colour, shape or position between frames.

Learn: Video generation ›
FLOP

One floating-point operation: a single sum or product of decimals.

Learn: Compute and FLOPs ›
FLOP/s

A speed: operations per second.

Learn: Compute and FLOPs ›
Focused training

Training mostly on data for one kind of job.

Learn: Small language models ›
Folding

The chain curling into its working 3D shape.

Learn: AI for science: protein folding ›
Forecast

A prediction about the future, often too hopeful in early AI.

Learn: The birth of 'artificial intelligence' ›also in 1 more lesson
Forecast band

The range where the future value will probably land.

Learn: Forecasting over time ›
Forget

Delete a saved note so it is no longer used.

Learn: Memory in assistants ›
Forget gate

Decides how much old memory stays.

Learn: LSTM: long short-term memory ›
Forgetting

Losing earlier general skills after too much narrow training.

Learn: Fine-tuning ›
Format

The length and layout of the answer, such as bullets or a table.

Learn: Prompts for everyday tasks ›
Formula

An instruction in a spreadsheet cell that works something out.

Learn: AI with documents and spreadsheets ›
Forward chaining

Firing rules from known facts and adding the results until nothing new appears.

Learn: Symbolic AI ›
Forward pass

Sending an input through all the layers to get an output.

Learn: The forward pass ›
Forward process

Adding noise step by step until only static remains.

Learn: Diffusion models ›
Frame

One picture of a video.

Learn: Video generation ›
Frame rate

How many frames are shown each second.

Learn: Video generation ›
Framing

How close or wide the view is.

Learn: Make images with AI ›
From scratch

Training with random starting numbers and no earlier knowledge.

Learn: Transfer learning ›
Frozen

Parameters that no longer change after training ends.

Learn: Training vs inference ›also in 1 more lesson
Frozen layers

Layers kept as they are while you train something else.

Learn: Transfer learning ›
Function

A rule that turns inputs into an output.

Learn: Why networks can learn almost anything ›
Function calling

The model writing a structured request to use a tool.

Learn: Tool use and function calling ›
Funding

Money from governments and companies that pays for research.

Learn: AI winters ›

G

Game tree

A diagram of every position reachable by legal moves.

Learn: Search and game playing ›
GAN

Generative adversarial network: two networks trained against each other.

Learn: GANs: generator vs discriminator ›
Generalisation

How well a model does on data it has never seen.

Learn: Overfitting and underfitting ›
Generate

Make something new, such as text or a picture.

Learn: What is AI? ›
Generator

The network that makes fake examples from random noise.

Learn: GANs: generator vs discriminator ›
Gini impurity

A score for how mixed a group is. Zero means all one kind.

Learn: Decision trees ›
Goodhart’s law

When a measure becomes a target, it stops being a good measure.

Learn: Alignment ›
GPT

Generative pretrained transformer: a decoder-only model trained to predict the next token.

Learn: GPT: generative pretraining ›
GPU

A chip with thousands of simple workers, built first for graphics.

Learn: GPUs and the deep learning boom ›also in 1 more lesson
GPU memory

The memory on a graphics card; models must fit in it to run fast.

Learn: Make images on your computer ›
Gradient

The slope of the loss: which way is uphill and how steep.

Learn: Gradient descent ›also in 2 more lessons
Gradient clipping

Capping the signal at a maximum size to stop an explosion.

Learn: Vanishing and exploding gradients ›
Grapheme to phoneme

Turning spelling into sounds.

Learn: Text to speech ›
Greedy

Always picking the top token. It is what a temperature near zero approaches.

Learn: Temperature ›
Greedy decoding

Always choosing the single likeliest next word.

Learn: Greedy decoding vs sampling ›
Greedy pick

Always choosing the token with the highest logit.

Learn: Logits ›
Grounding

Giving the model real text or data to base its answer on.

Learn: Hallucination ›also in 3 more lessons
Guardrail

A check placed around a model to keep it safe.

Learn: Jailbreaks and misuse ›also in 1 more lesson
Guidance scale

How hard the prompt pushes the result.

Learn: Latent diffusion and text-to-image ›

H

Hallucination

A fluent, confident reply that contains false or invented content.

Learn: Hallucination ›
Hand-off

Passing work from one agent to the next.

Learn: Multi-agent systems ›
Harness

The software around a model that runs it as an agent.

Learn: The agent harness ›
Head

The final small part that produces the answer for your task.

Learn: Transfer learning ›also in 1 more lesson
Held-out

Kept out of training so it is a fair test.

Learn: Benchmarks and evaluations ›
Hidden layer

A middle layer of neurons between the inputs and the output.

Learn: The XOR problem ›also in 2 more lessons
Hidden state

The memory an RNN carries from one step to the next.

Learn: Recurrent neural networks ›
Hierarchy

Simple things first, then combinations of them.

Learn: Convolutional neural networks ›
High risk

Uses that can seriously affect people’s lives or rights.

Learn: AI regulation ›
High stakes

A situation where a mistake could cause serious harm or loss.

Learn: When not to use AI ›
High-dimensional

Having many dimensions, too many to draw.

Learn: Vectors and dimensions ›
Horizon

How many steps ahead the agent imagines.

Learn: World models ›
Hosted model

A model you use through a company’s service over the internet.

Learn: The model landscape: open and closed ›
Human baseline

Careful human scores used to check the judge.

Learn: AI judging AI ›
Human in the loop

A person approves risky actions.

Learn: Prompt injection ›also in 1 more lesson
Human judgement

A person weighing the full situation and taking responsibility.

Learn: When not to use AI ›
Human oversight

A person who can check and overrule the system.

Learn: AI regulation ›
Hybrid search

Combining keyword search and vector search.

Learn: Hybrid search and reranking ›
Hype

Excitement and expectations that run ahead of what a technology can do.

Learn: AI winters ›
Hyperparameter

A setting chosen by people before training, like the learning rate.

Learn: Learning rate ›

I

Identity

Passing something along unchanged.

Learn: Residual connections (ResNet) ›
Image prompt

A written description an AI uses to make a picture.

Learn: Make images with AI ›
ImageNet

A huge library of labelled pictures used to test recognition.

Learn: ImageNet and AlexNet ›
Imbalanced data

Data where one class is far rarer than the other.

Learn: Accuracy, precision and recall ›
Imitation game

Turing's own name for the test.

Learn: The Turing test ›
Imitation learning

Learning a skill by copying demonstrations from people.

Learn: Robots that learn ›
In-context learning

Picking up a pattern from the prompt itself, without changing the weights.

Learn: Zero-shot and few-shot prompting ›
Index

A structure that narrows the search so not every vector is compared.

Learn: Vector databases ›
Indirect injection

The planted text comes from a document, page or tool result, not from the user.

Learn: Prompt injection ›
Inference

Using a trained model to answer a new input. Parameters stay fixed.

Learn: Training vs inference ›also in 3 more lessons
Inference engine

The part that checks which rules match and combines their answers.

Learn: Expert systems ›
Input

A number fed into the neuron.

Learn: A neuron: weights and bias ›
Input gate

Decides how much new information is written.

Learn: LSTM: long short-term memory ›
Input tokens

Everything sent to the model, including the system message and past turns.

Learn: Tokens, limits and cost ›
Instance segmentation

Separating each individual object of a type.

Learn: Image segmentation ›
Instruction

A request written in plain words.

Learn: Instruction tuning ›
Instruction tuning

Extra training so a model follows instructions, often using human feedback.

Learn: GPT: generative pretraining ›also in 2 more lessons
Intercept

The line's height when the input is zero.

Learn: Models and parameters ›
Interpretability

Research that explains what happens inside a model.

Learn: Interpretability ›
IoU

Shared area of two boxes divided by their combined area.

Learn: Object detection ›
Iterate

Change, check and change again.

Learn: Coding agents ›
Iteration

One step of the loop.

Learn: Gradient descent ›

J

Jailbreak

An attempt to make a model ignore its safety rules.

Learn: Jailbreaks and misuse ›
JSON

A plain-text format of labels and values, like {"age": 12}.

Learn: Structured output ›also in 4 more lessons
Judge

The person asking the questions and deciding.

Learn: The Turing test ›
Judge model

A model that scores or compares other answers.

Learn: AI judging AI ›also in 1 more lesson

K

k

How many nearest neighbours get a vote.

Learn: k-nearest neighbours ›also in 2 more lessons
k-means

A method that alternates between assigning points and moving centres.

Learn: Clustering (unsupervised) ›
Key

What a word advertises, to be matched against queries.

Learn: Queries, keys and values ›
Key and value

The parts of each token that attention looks up, from the attention lesson.

Learn: The KV cache ›
Keyword search

Search that looks for the words you typed.

Learn: Semantic search ›
Knowledge base

The collection of facts and rules a system uses.

Learn: Symbolic AI ›also in 2 more lessons
Knowledge cutoff

The date after which the model's training data has nothing.

Learn: Knowledge cutoff ›
Knowledge engineer

A person who turns a specialist's know-how into rules.

Learn: Expert systems ›
KV cache

Stored keys and values of earlier tokens, reused for every new token.

Learn: The KV cache ›
kWh

A kilowatt-hour: 1,000 watts used for one hour.

Learn: Energy and the environment ›

L

L2 (ridge)

A penalty on the squared size of the weights. It shrinks them smoothly.

Learn: Regularisation ›
Label

The right answer attached to an example.

Learn: Labels and supervised learning ›also in 1 more lesson
Latency

How long you wait, which grows with the length of the answer.

Learn: Tokens, limits and cost ›also in 2 more lessons
Latent

A small code that stands for a picture.

Learn: Latent diffusion and text-to-image ›
Layer

A row of neurons that work on the same inputs together.

Learn: Layers and the multi-layer network ›also in 4 more lessons
Layer norm

Normalises across the numbers of a single example. Used in Transformers.

Learn: Batch and layer normalisation ›
Lazy learner

A model that keeps the examples and does its work when asked.

Learn: k-nearest neighbours ›
Leaf

An end box of the tree that gives the answer.

Learn: Decision trees ›
Leak

Private text coming back out in a model’s answer.

Learn: Privacy and training data ›
Leakage

Test questions or answers appearing in training data. Also called contamination.

Learn: Benchmarks and evaluations ›
Learning rate

The number that scales each gradient step.

Learn: Learning rate ›also in 1 more lesson
Learning rule

The recipe for changing weights after a mistake.

Learn: The perceptron ›
Least privilege

Give the assistant only the access it needs.

Learn: Prompt injection ›
Least squares

Choosing the line with the smallest total of squared errors.

Learn: Linear regression ›
Length (magnitude)

How long the arrow is, from the origin to the tip.

Learn: Vectors and dimensions ›
Licence

The rules for how you may use a model.

Learn: Open-weight vs closed models ›also in 2 more lessons
Lighthill report

A 1973 UK review that criticised AI's progress and led to funding cuts.

Learn: AI winters ›
Likeness

How a real person looks. Do not misuse it.

Learn: Make images with AI ›
Linear growth

Work that grows in step with the length.

Learn: Beyond Transformers: state-space models ›
Linearly separable

Two groups that one straight line can split.

Learn: The XOR problem ›
LLM

Large language model: a big transformer trained to predict the next token.

Learn: What is a large language model? ›
LLM as a judge

Using a language model to mark another model’s answers.

Learn: Test your app with an AI judge ›
Local model

A model that runs on your own computer.

Learn: Run an open model on your computer ›
Local slope

How much one step's output changes when its own input changes.

Learn: The chain rule, intuitively ›
localhost

A name that means this same computer.

Learn: Call your local model from code ›
localhost / 127.0.0.1

Names for this same computer; the page is reachable only from here.

Learn: Give your RAG app a simple web page ›
Log loss

The loss for probabilities: small if confident and right, huge if confident and wrong.

Learn: Logistic regression ›
Log scale

A chart axis where each step is ten times bigger.

Learn: Scaling laws ›
Logic gate

A tiny rule on on/off values, like AND, OR or NOT.

Learn: The first artificial neuron ›
Logit

A raw score before softmax.

Learn: Softmax ›also in 2 more lessons
Long context

A very large input, such as many pages.

Learn: Long context and 'lost in the middle' ›
Long tail

The many unlikely words at the end of the list.

Learn: Top-k and top-p sampling ›
Look-ahead

Peeking at later tokens, which the causal mask forbids.

Learn: Causal masking ›
Loop

Text that keeps repeating the same few words.

Learn: Greedy decoding vs sampling ›also in 1 more lesson
Loop limit

The most rounds an agent may take before it is stopped.

Learn: The agent loop ›
LoRA

Low-rank adaptation: train two thin matrices instead of the full weight.

Learn: LoRA and efficient fine-tuning ›also in 1 more lesson
Loss

One number that measures how wrong the model's predictions are. Lower is better.

Learn: Loss: measuring mistakes ›also in 2 more lessons
Loss floor

The error that remains however big you scale. Language itself has surprise in it.

Learn: Scaling laws ›
Lost in the middle

Facts in the middle of a long input are used less reliably.

Learn: Long context and 'lost in the middle' ›
LSTM

Long short-term memory: a recurrent network with gates, from 1997.

Learn: LSTM: long short-term memory ›

M

Machine learning

Systems that improve at a task by learning from data.

Learn: Rules vs learning ›
Made-up detail

A fact, quote or source the chatbot invented but wrote as if true.

Learn: Check AI answers ›
Majority vote

Choose the answer that most paths agree on.

Learn: Self-consistency and voting ›
Margin

How much more the model favours the preferred answer than the reference did, compared with the rejected one.

Learn: Direct preference optimisation ›
Mask

The set of pixels that belong to one object.

Learn: Image segmentation ›also in 1 more lesson
Masked

Blocked from being used. Masked scores become exactly 0% after softmax.

Learn: Causal masking ›
Masked language model

A model trained to fill in hidden words in a sentence.

Learn: BERT: reading both ways ›
Masking

Replacing private details with a label like [EMAIL].

Learn: Privacy and training data ›
Matrix

A grid of numbers. A layer of sums is one matrix times another.

Learn: GPUs and the deep learning boom ›
Max pooling

Keeps the largest value in each block.

Learn: Pooling ›
Max tokens

The most tokens the model may write in one reply.

Learn: Stop sequences and max tokens ›also in 1 more lesson
MCP

Model Context Protocol, an open standard for connecting AI apps to tools and data.

Learn: Model Context Protocol (MCP) ›also in 1 more lesson
MCP client

The AI app side that starts a server, lists its tools and calls them for the model.

Learn: Connect your MCP server to an AI app ›
MCP server

A small program that offers tools to AI apps.

Learn: Build a simple MCP server ›
Mean squared error (MSE)

The average of the squared errors.

Learn: Loss: measuring mistakes ›
Media literacy

The habit of checking before believing or sharing.

Learn: Deepfakes, watermarks and provenance ›
Memorisation

When a model stores some text almost exactly instead of just the pattern.

Learn: Privacy and training data ›
Memory

Notes saved between chats and added to later prompts.

Learn: Memory in assistants ›
Memory (VRAM)

The fast storage next to the chip where the model's weights must fit.

Learn: GPUs, TPUs and accelerators ›
Memory cost

The space the cache takes. It grows with tokens and layers.

Learn: The KV cache ›
Merge rule

One learned step, such as “l + o → lo”. Rules are replayed in order on new text.

Learn: How tokenizers are built (BPE) ›
Metadata

Labels saved with each item, such as the file name and line number.

Learn: Set up a local vector database ›
Metric

The rule used to score results, such as exact match or partial credit.

Learn: Emergent abilities ›
Milestone

A step that changed what AI could do.

Learn: Recap: the AI timeline ›
Minimax

Pick your best move assuming the opponent picks their best reply.

Learn: Search and game playing ›
Minimum

The bottom of the loss curve, where the model is best.

Learn: Gradient descent ›
Missing value

An empty cell where a number should be.

Learn: Collecting and cleaning data ›
MLP

Multi-layer perceptron: neurons arranged in layers, each feeding the next.

Learn: Layers and the multi-layer network ›
Modality

One kind of input, such as sound.

Learn: Multimodal LLMs ›
Model

A formula that turns inputs into a prediction.

Learn: Models and parameters ›
Model card

The page that describes a model, its use and its limits.

Learn: The model landscape: open and closed ›
Model collapse

Quality slowly narrowing when models keep training on their own output.

Learn: Synthetic data ›
Model size

Bigger speech models are usually more accurate but slower and need more memory.

Learn: Turn speech into text on your computer ›
Model theft

Copying a model by training another on its answers to many queries.

Learn: Data poisoning and model theft ›
Momentum

Adds a running memory of past gradients so steps build speed.

Learn: Optimisers: SGD, momentum, Adam ›
Multi-agent system

Several agents with different roles working on one job.

Learn: Multi-agent systems ›
Multi-head attention

Several heads run in parallel and joined together.

Learn: Multi-head attention ›
Multi-layer network

A network with one or more hidden layers.

Learn: The XOR problem ›
Multi-step problem

A problem whose answer needs several operations in a row.

Learn: Chain of thought ›
Multilingual tokenizer

One trained on many languages so each gets fair-sized pieces.

Learn: Tokens across languages ›
Multimodal

Able to take more than one kind of input: text, pictures, sound.

Learn: Multimodal LLMs ›

N

Narrow AI

AI built for particular jobs.

Learn: AGI and the road ahead ›
Nearest neighbour

The stored item whose numbers are closest to yours.

Learn: Embeddings ›also in 1 more lesson
Nearest word

The word whose vector is closest to a point.

Learn: Word vectors (word2vec) ›
Needle in a haystack

A test that hides one fact in long text and asks the model to find it.

Learn: Long context and 'lost in the middle' ›
Neighbour

One of the stored examples nearest to the new point.

Learn: k-nearest neighbours ›
Neuron

One tiny detector that adds up inputs and fires if the total is high enough.

Learn: The feed-forward block ›
Neuron (artificial)

A unit that adds weighted inputs and decides whether to fire.

Learn: The first artificial neuron ›
Next-token prediction

Guessing which token comes next, given all the tokens so far.

Learn: Next-token prediction ›
Noise

Variation that has nothing to do with the answer, like a door colour.

Learn: Data and features ›also in 6 more lessons
Non-linear

Not a straight line: it can curve.

Learn: Activation functions ›
Non-maximum suppression

Keeping the best box and dropping overlapping copies.

Learn: Object detection ›
Normalisation

Rescaling a column into a fixed range, often 0 to 1 (min-max).

Learn: Normalisation and feature scaling ›also in 1 more lesson
Note

A short saved fact, such as a preference.

Learn: Memory in assistants ›

O

Objective

The thing training tries to improve, here, the loss.

Learn: Loss: measuring mistakes ›
Observation

The result of an action, added to the agent's context.

Learn: The agent loop ›
OCR

Text recognition that reads words from pictures; scanned PDFs need it first.

Learn: Turn PDFs into searchable chunks ›
Offline

Working without an internet connection.

Learn: On-device AI ›also in 1 more lesson
On-device

Running on your own phone or computer.

Learn: Small language models ›also in 1 more lesson
On-device AI

A model that runs on your own phone or laptop instead of a server.

Learn: On-device AI ›
One-hot vector

A vector with a single 1 and all other entries 0, one slot per word.

Learn: One-hot vs dense vectors ›
One-shot

A prompt with one example.

Learn: Zero-shot and few-shot prompting ›
Open-weight

A model whose weights you can download and run. The training data and code may still be private.

Learn: Open-weight vs closed models ›
Open-weight model

A model whose weights are published so you can download them, under a licence.

Learn: Run an open model on your computer ›also in 1 more lesson
Opt-out

A signal from an owner saying their work should not be used for training.

Learn: Copyright and training data ›
Optimiser

The rule that turns gradients into weight updates.

Learn: Optimisers: SGD, momentum, Adam ›
Option (flag)

An extra word like --version that changes what a command does.

Learn: Terminal basics for AI work ›
Optional field

A field that may be missing, written so the model can honestly say null.

Learn: Pull structured data out of documents ›
Orchestrator

The main agent that plans, delegates and joins the results.

Learn: Planning and subagents ›also in 1 more lesson
Out-of-vocabulary

A word not in the list as a whole. It gets split into smaller known pieces.

Learn: Vocabulary and token IDs ›
Outlier

A value far from the rest, often a typing slip.

Learn: Collecting and cleaning data ›also in 1 more lesson
Output

The squashed result, here between 0 and 1.

Learn: A neuron: weights and bias ›also in 1 more lesson
Output gate

Decides how much memory is read out.

Learn: LSTM: long short-term memory ›
Output tokens

The tokens the model writes back.

Learn: Tokens, limits and cost ›
Overfitting

The model memorises noise in the training data, so it does well on training data and badly on new data.

Learn: Overfitting and underfitting ›also in 4 more lessons
Overlap

Words repeated at the edges of neighbouring chunks.

Learn: Chunking documents ›also in 1 more lesson
Overshoot

Stepping past the bottom to the other side.

Learn: Learning rate ›

P

Package

A ready-made library you install, such as requests.

Learn: Set up Python and a virtual environment ›
Padding

Extra zeros around the border so the output keeps the same size.

Learn: Convolution ›
Parallel

Many things worked on at the same time.

Learn: GPUs and the deep learning boom ›also in 2 more lessons
Parameter

An adjustable number inside the model, such as the slope. Also called a weight.

Learn: Models and parameters ›also in 2 more lessons
Parameters

The learned numbers inside a model. More of them means a bigger model.

Learn: GPT: generative pretraining ›also in 2 more lessons
Partial credit

Gives points for each part that is right, so progress shows up gradually.

Learn: Emergent abilities ›
Passage

A short piece of a document, small enough to fit in a prompt.

Learn: Retrieval-augmented generation (RAG) ›
Patch

A fix made after a weak spot is found.

Learn: Red teaming and safety testing ›also in 2 more lessons
Pattern

A repeated kind of link, such as noun to adjective.

Learn: Multi-head attention ›
Pattern matching

Checking whether text fits a shape, like I am …

Learn: ELIZA, the first chatbot ›
PCA

Principal component analysis: finds the directions along which the data spreads most.

Learn: Dimensionality reduction ›
PEFT

Parameter-efficient fine-tuning: changing only a few parameters.

Learn: LoRA and efficient fine-tuning ›
Penalty

The extra charge added to the error. A stronger penalty means a simpler model.

Learn: Regularisation ›
Perceptron

A single artificial neuron that learns weights from examples.

Learn: The perceptron ›
Permission

What an agent is allowed to do.

Learn: Permissions, sandboxes and guardrails ›
Permission prompt

The harness asking you before a risky action.

Learn: The agent harness ›
Perplexity

A score of how surprised a model is by text. Lower means better prediction.

Learn: Perplexity ›
Personal data

Facts that identify a real person, such as a name with an address.

Learn: Keep your data safe with chatbots ›also in 1 more lesson
Personalisation

Answers shaped by what the assistant knows about you.

Learn: Memory in assistants ›
Phishing

A message that tricks you into giving up details or money.

Learn: Spot AI scams and fake media ›
Phoneme

One basic sound of speech, such as the s in see.

Learn: Speech recognition ›also in 1 more lesson
pip

Python’s tool for installing packages.

Learn: Set up Python and a virtual environment ›
Pixel

One dot of a picture, stored as a number.

Learn: Convolution ›also in 1 more lesson
Plan

A list of smaller tasks that together reach a goal.

Learn: Planning and subagents ›
Planning

Using a model to compare imagined futures before acting.

Learn: World models ›
Plausible

Sounds likely or reasonable, whether or not it is true.

Learn: Hallucination ›
Policy

The agent's habit: which action it prefers in each state. Shown here as arrows.

Learn: Reinforcement learning ›also in 3 more lessons
Pooling

Shrinking a map by summarising each block with one number.

Learn: Pooling ›
Port

A numbered door on your computer that a program listens on.

Learn: Give your RAG app a simple web page ›
Position

Where in the input a fact sits.

Learn: Long context and 'lost in the middle' ›also in 1 more lesson
Position embedding

A learned position code instead of a fixed formula.

Learn: Position: how models know word order ›
Positional encoding

Extra numbers added to each token that say where it sits in the text.

Learn: Position: how models know word order ›
Power law

A curve where each tenfold increase changes the result by a steady fraction.

Learn: Scaling laws ›
Pre-activation

The weighted sum plus bias, before the activation function.

Learn: The forward pass ›
Precision

Of the items flagged yes, the share that really were yes.

Learn: Accuracy, precision and recall ›also in 1 more lesson
Predict

Make a best guess about what comes next.

Learn: What is AI? ›
Predicted rating

The system's guess of how much you would like something.

Learn: Recommendation systems ›
Preference

A human choice that one answer is better than another.

Learn: RLHF: learning from human feedback ›
Preference pair

Two answers to one question, with one marked better.

Learn: Direct preference optimisation ›
Prefill

The model reading your whole prompt before it starts writing.

Learn: Time to first token and tokens per second ›
Prefix

The start of a prompt, from its first token up to a point.

Learn: Prompt caching ›
Pretrained model

A model already trained on a large task.

Learn: Transfer learning ›
Pretraining

Learning from a huge amount of general text before any specific task.

Learn: GPT: generative pretraining ›also in 2 more lessons
Principle

A short written rule the answers should follow.

Learn: Constitutional AI and AI feedback ›
Privacy–accuracy trade-off

More privacy costs some accuracy, and the reverse.

Learn: Privacy-preserving AI ›
Probability

How likely something is, from 0 (never) to 1 (certain).

Learn: Logistic regression ›also in 2 more lessons
Probe

A test prompt written to check one kind of risk.

Learn: Red teaming and safety testing ›
Projection

A flat shadow of higher-dimensional data.

Learn: Dimensionality reduction ›
Prompt

The text you give a model to tell it what to do.

Learn: Prompt basics ›also in 3 more lessons
Prompt caching

Saving the model's work on an unchanged start of a prompt so later requests can skip it.

Learn: Prompt caching ›
Prompt injection

Text that tries to give a model new instructions it should not follow.

Learn: Prompt injection ›also in 1 more lesson
Prosody

The rhythm, stress and tune of speech.

Learn: Text to speech ›
Protein

A chain of amino acids that does a job in a living thing.

Learn: AI for science: protein folding ›
Protocol

An agreed set of rules for how two programs talk.

Learn: Model Context Protocol (MCP) ›
Provenance

The record of where something came from.

Learn: Deepfakes, watermarks and provenance ›also in 1 more lesson
Proxy

A stand-in measure that usually tracks what we really want.

Learn: Reward models ›
Pruning

Skipping branches that cannot change the decision.

Learn: Search and game playing ›
Public domain

Work whose copyright has ended, free to use.

Learn: Copyright and training data ›
Pull

Download a model to your computer.

Learn: Run an open model on your computer ›
Python

A programming language used by most AI tools.

Learn: Set up Python and a virtual environment ›

Q

Q-value

The learned score of taking one action in one state.

Learn: Reinforcement learning ›
Quadratic growth

Work that grows with the square of the length.

Learn: Beyond Transformers: state-space models ›
Qualified person

Someone trained and accountable for advice in their field.

Learn: When not to use AI ›
Quantisation

Storing numbers with fewer bits by rounding them to a coarser grid.

Learn: Quantisation ›also in 2 more lessons
Query

What a word is looking for.

Learn: Queries, keys and values ›

R

R²

How much of the variation in the answers the line explains: 0 is none, 1 is all.

Learn: Linear regression ›
RAG

Retrieval-augmented generation: fetch passages first, then write an answer from them.

Learn: Retrieval-augmented generation (RAG) ›also in 1 more lesson
RAM

The computer’s working memory. A model must fit in it to run.

Learn: How the hands-on tutorials work ›
Random forest

An ensemble of many decision trees, each trained on a shuffled copy of the data.

Learn: Random forests and ensembles ›
Rank

The thin side shared by A and B. Higher rank means more trainable numbers.

Learn: LoRA and efficient fine-tuning ›
Rate limit

A cap on how many requests a user can make in a given time.

Learn: Data poisoning and model theft ›
Re-index

Rebuild the database after changing how the chunks are made.

Learn: Test and improve your RAG app ›
Re-planning

Imagining again after each real step, using what was actually seen.

Learn: World models ›
ReAct

Reason plus act: an agent pattern that alternates thoughts and actions.

Learn: Reason + act (ReAct) ›
Reasoning model

A model trained to produce such working on its own.

Learn: Chain of thought ›also in 1 more lesson
Reasoning path

One full line of working that ends in an answer.

Learn: Self-consistency and voting ›
Recall

Of the items that really were yes, the share that were found.

Learn: Accuracy, precision and recall ›also in 1 more lesson
Recognise

Tell what something is, such as a face, a word or a sound.

Learn: What is AI? ›
Reconstruction

The rebuilt output, compared with the original.

Learn: Autoencoders ›
Recurrent

Using the same step again and again in a loop.

Learn: Recurrent neural networks ›
Red team

People who try to make a system fail, so it can be improved.

Learn: Red teaming and safety testing ›
Red teaming

Testing a system on purpose to find weaknesses.

Learn: Jailbreaks and misuse ›
Reference model

A frozen copy of the model from before training.

Learn: Direct preference optimisation ›
Reflection

Swapping I and you so your words can be echoed back.

Learn: ELIZA, the first chatbot ›
Regression

Predicting a number, such as a price or a temperature.

Learn: Linear regression ›also in 2 more lessons
Regularisation

Adding a cost for complexity so the model prefers simpler answers.

Learn: Regularisation ›also in 1 more lesson
Reliable source

An official page, original document or expert that is accountable for its facts.

Learn: Check AI answers ›
ReLU

Zero for negatives, unchanged for positives.

Learn: Activation functions ›also in 2 more lessons
Renormalise

Scale the surviving odds so they add up to 100% again.

Learn: Top-k and top-p sampling ›
Request / response

The message you send and the one that comes back.

Learn: Call your local model from code ›
Required field

A field that must be present.

Learn: Structured output ›
Reranking

Re-scoring a short list of results with a slower, more careful method.

Learn: Hybrid search and reranking ›
Residual

The gap between a dot and the line.

Learn: Linear regression ›also in 1 more lesson
Residual (add back)

Adding a block's result onto its input instead of replacing it.

Learn: The feed-forward block ›
Residual connection

The shortcut that keeps the old vector and adds the layer’s result to it. Also called a skip connection.

Learn: The residual stream ›
Residual stream

The running vector of a token that every layer reads from and adds to.

Learn: The residual stream ›
ResNet

A family of networks built from residual blocks, introduced in 2015.

Learn: Residual connections (ResNet) ›
Retrieval

Fetching the most relevant documents for a query.

Learn: Semantic search ›also in 3 more lessons
Retrieval test

Checking whether search returns the chunk that holds the answer.

Learn: Test and improve your RAG app ›
Reverse process

Starting from static and removing noise step by step.

Learn: Diffusion models ›
Revision

The rewritten answer after the critique.

Learn: Constitutional AI and AI feedback ›
Revoke

Cancel a key so it stops working.

Learn: Use a hosted model API safely ›
Reward

A number the world gives back after an action: good is positive, bad is negative.

Learn: Reinforcement learning ›also in 1 more lesson
Reward hacking

Raising the score without improving what the score was meant to measure.

Learn: Reward models ›also in 2 more lessons
Reward model

A model that scores answers the way people would.

Learn: RLHF: learning from human feedback ›also in 1 more lesson
Risk-based

Rules that get stricter as the possible harm grows.

Learn: AI regulation ›
RLHF

Reinforcement learning from human feedback.

Learn: RLHF: learning from human feedback ›
Role

Who a message is from: system, user or assistant.

Learn: System prompts and chat roles ›also in 1 more lesson
Round

One step in which every busy worker finishes a cell.

Learn: GPUs and the deep learning boom ›
Route

An address on the server, such as / or /ask, that runs some code.

Learn: Give your RAG app a simple web page ›
Router

A small layer that scores the experts for each token.

Learn: Mixture of experts ›
Routing

Sending each request to the model best suited to it, such as a smaller one for easy tasks.

Learn: The cost of serving AI ›
Rubric

A written list of what a good answer must have.

Learn: AI judging AI ›also in 1 more lesson
Rule

An instruction a person wrote, such as if X then Y.

Learn: Rules vs learning ›also in 1 more lesson
Runner

A program that loads a model file and lets you chat with it.

Learn: Run an open model on your computer ›

S

Safety check

A score for rules broken along the way, even when the task finished.

Learn: Evaluating agents ›
Safety testing

Checking that a model behaves well in risky situations.

Learn: Red teaming and safety testing ›
Sample rows

A few rows you check by hand to test a formula.

Learn: AI with documents and spreadsheets ›
Sampling

Picking the next token at random, following the odds.

Learn: Temperature ›also in 3 more lessons
Sandbox

A fenced, throw-away space where mistakes cannot reach the real thing.

Learn: Permissions, sandboxes and guardrails ›
Saturation

When scores are so high that the test cannot separate models.

Learn: Benchmarks and evaluations ›
Scale

The factor that maps whole-number codes back to real weight values.

Learn: Quantisation ›
Scaling law

A formula that predicts loss from model size, data and compute.

Learn: Scaling laws ›
Scaling laws

Findings that results improve smoothly as model size, data and compute grow.

Learn: GPT: generative pretraining ›
Schema

The agreed shape: which fields, which types, which are required.

Learn: Structured output ›also in 1 more lesson
Score fusion

Blending several scores into one ranking.

Learn: Hybrid search and reranking ›
Scratchpad

Space where the model writes working that it can read back.

Learn: Chain of thought ›also in 1 more lesson
Screenshot

A picture of the screen the agent reads each turn.

Learn: Computer-using agents ›
Script

The writing system, such as Latin, Devanagari or Tamil.

Learn: Tokens across languages ›
Scripted bot

A program that answers from fixed patterns or canned lines.

Learn: The Turing test ›
Search

Exploring possible moves to find a good one.

Learn: Search and game playing ›also in 1 more lesson
Seasonality

A pattern that repeats on a fixed cycle.

Learn: Forecasting over time ›
Seed

The number that decides the starting noise. Same seed and prompt, same picture.

Learn: Make images on your computer ›
Segment

A short stretch of speech with its own start time, end time and text.

Learn: Turn speech into text on your computer ›
Selection rate

The share of a group that gets the positive outcome.

Learn: Bias and fairness ›
Self-attention

Each token in a sentence looks at the other tokens in the same sentence.

Learn: Self-attention ›
Self-hosting

Running a model on hardware you control.

Learn: Open-weight vs closed models ›
Self-play

Learning by playing against a copy of yourself.

Learn: AlphaGo and deep reinforcement learning ›
Self-supervised

Learning where the data supplies its own answers, like the hidden next token.

Learn: Pretraining ›
Semantic search

Search that compares meaning using embeddings instead of exact words.

Learn: Semantic search ›
Semantic segmentation

Labelling pixels by type of thing.

Learn: Image segmentation ›
Sensitive data

Very private details, such as health, money and ID numbers.

Learn: Keep your data safe with chatbots ›
Sequence

Things in an order, like the words of a sentence.

Learn: Recurrent neural networks ›also in 2 more lessons
Server

A tool or data source that offers its abilities through the protocol.

Learn: Model Context Protocol (MCP) ›
Serving

Running a trained model to answer real requests.

Learn: The cost of serving AI ›
Settings file

A file holding choices like model name and temperature.

Learn: Save prompts and settings as files ›
SGD

Stochastic gradient descent: step against the gradient of a batch.

Learn: Optimisers: SGD, momentum, Adam ›
Shared space

One space where pictures and text can be compared directly.

Learn: CLIP: matching images and text ›
Shift tolerance

A pattern still counts when it moves a little.

Learn: Pooling ›
Shortlist

The top results kept for a closer look.

Learn: Hybrid search and reranking ›
Shot

One example in the prompt.

Learn: Zero-shot and few-shot prompting ›
Sigmoid

An S-shaped function that squeezes any number into the range 0 to 1.

Learn: Logistic regression ›also in 1 more lesson
Sim-to-real gap

The drop in performance when a skill learned in simulation meets the real world.

Learn: Robots that learn ›
Similarity

A score for how alike two people's ratings are.

Learn: Recommendation systems ›
Similarity score

A number that says how close two embeddings are; higher means closer in meaning.

Learn: Make embeddings on your computer ›
Simulate

Copy how something works closely enough to behave like it.

Learn: The birth of 'artificial intelligence' ›
Simulation

A computer model of the robot and its surroundings used for safe, fast practice.

Learn: Robots that learn ›
Sinusoidal

Made of sine and cosine waves, as in the original Transformer.

Learn: Position: how models know word order ›
Skew

Data that over-represents some kinds of example.

Learn: Collecting and cleaning data ›
Skip connection

A wire that jumps over one or more layers.

Learn: Residual connections (ResNet) ›
Slice (frame)

A short piece of sound, around 20 to 30 milliseconds.

Learn: Speech recognition ›
Slipped through

A wrong answer that every active check passed.

Learn: Verifiers and checking ›
Slope

How steeply the line rises for each step in the input.

Learn: Models and parameters ›
Small language model

A language model with relatively few parameters, built to run cheaply.

Learn: Small language models ›
Small model

A model with fewer parameters, so it needs less memory and runs faster.

Learn: On-device AI ›
Soft labels

The teacher’s full set of probabilities, not just its top choice.

Learn: Distillation ›
Softmax

A function that turns scores into probabilities that add up to 1.

Learn: Softmax ›also in 1 more lesson
Sound-alike words

Words that sound the same but mean different things, like two and to.

Learn: Talk to AI: voice and dictation ›
Spacetime patch

A small box covering a little area over a few frames: one video token.

Learn: Video generation ›
Sparse

Mostly zeros, like a one-hot vector.

Learn: One-hot vs dense vectors ›
Special token

A reserved marker, such as a role tag, that ordinary typing cannot make.

Learn: System prompts and chat roles ›
Specification gaming

Hitting the letter of a goal while missing its point.

Learn: Alignment ›
Spectrogram

A picture of a sound: time across, frequency up, brightness for loudness.

Learn: Speech recognition ›
Split

One question that cuts a group of examples into two.

Learn: Decision trees ›
Splitter

The code that cuts a document into chunks.

Learn: Chunking documents ›
Stakes

How much harm a mistake could cause.

Learn: Using AI responsibly ›
Stale

Out of date because the world moved on after the cutoff.

Learn: Knowledge cutoff ›
Stale information

Out-of-date text that can mislead the model.

Learn: Context engineering ›
Stand-in

A made-up name or number used in place of a real one.

Learn: Keep your data safe with chatbots ›
Standardisation

Shifting a column to average 0 and spread 1 (z-score).

Learn: Normalisation and feature scaling ›
State

The fixed-size summary a model carries from word to word.

Learn: Beyond Transformers: state-space models ›
State-space model

A sequence model that updates a running state for each token.

Learn: Beyond Transformers: state-space models ›
Step (iteration)

One update of the weights.

Learn: Epochs, batches and steps ›
Step by step

A common prompt phrase that asks for chain of thought.

Learn: Chain of thought ›
Step limit

A hard cap on how many times the loop may run.

Learn: Build a tiny agent with tools ›
Step size

How far to move each time. Also called the learning rate.

Learn: Gradient descent ›
Steps

How many actions it took.

Learn: Evaluating agents ›also in 1 more lesson
Stop sequence

Text that halts generation as soon as it appears.

Learn: Stop sequences and max tokens ›
Streaming

Showing the reply as it is written instead of all at the end.

Learn: Time to first token and tokens per second ›
Structure prediction

Guessing the 3D shape from the sequence alone.

Learn: AI for science: protein folding ›
Structured output

A reply in a fixed, machine-readable format.

Learn: Structured output ›also in 1 more lesson
Student

The smaller model trained to imitate the teacher.

Learn: Distillation ›
Style

The look of the image, such as watercolour or pencil sketch.

Learn: Make images with AI ›
Subagent

A helper agent started for one job with its own context.

Learn: Planning and subagents ›
Subtitle file (.srt)

A text file of numbered, timed lines that video players can show.

Learn: Turn speech into text on your computer ›
Subword

A piece of a word, between a single letter and a whole word.

Learn: How tokenizers are built (BPE) ›
Summary

A shorter version that keeps the main points.

Learn: AI with documents and spreadsheets ›
Supervised learning

Learning from examples that come with labels.

Learn: Labels and supervised learning ›
Surprise

How unlikely the model thought the word that actually came.

Learn: Perplexity ›
Symbol

A written label for an idea, such as bird or mammal.

Learn: Symbolic AI ›
Synthetic data

Data made by a model or program rather than collected.

Learn: Synthetic data ›
System prompt

The first message that sets rules and style for the whole chat.

Learn: System prompts and chat roles ›also in 1 more lesson

T

t-SNE

Another reduction method, used mostly to draw maps that keep neighbours together.

Learn: Dimensionality reduction ›
Tanh

An S-shaped squash into −1 to 1.

Learn: Activation functions ›
Task

The action you want done, said clearly.

Learn: Prompt basics ›
Task success

Whether the agent reached the goal.

Learn: Evaluating agents ›
Teacher

The large model whose behaviour is copied.

Learn: Distillation ›
Teacher and student

The model that writes examples, and the one that learns from them.

Learn: Synthetic data ›
Temperature

A dial that divides scores first: low makes softmax sharper, high makes it flatter.

Learn: Softmax ›also in 4 more lessons
Temporal consistency

Things staying the same from frame to frame.

Learn: Video generation ›
Terminal

A text window where you type commands.

Learn: How the hands-on tutorials work ›also in 1 more lesson
Test set

Examples kept sealed until the end, for one honest score.

Learn: Train, validation and test sets ›also in 3 more lessons
Test suite

A set of small checks that say whether the code works.

Learn: Coding agents ›
Test time

The moment the model answers, as opposed to training time.

Learn: Test-time compute ›
Text extraction

Pulling the written text out of a PDF file.

Learn: Turn PDFs into searchable chunks ›
Text-to-image

Making a picture from a written prompt.

Learn: Latent diffusion and text-to-image ›
Text-to-image model

A model that makes a picture from a written description.

Learn: Make images on your computer ›
Thinking budget

How much working the model may write before answering.

Learn: Test-time compute ›
Thinking tokens

Working-out the model writes before its final answer.

Learn: Thinking tokens and extended thinking ›
Thought

A written reason for the next step.

Learn: Reason + act (ReAct) ›
Threshold

The total the neuron must reach before it fires.

Learn: The first artificial neuron ›also in 2 more lessons
Throughput

How much work a system finishes per second across all users.

Learn: Latency vs throughput ›
Tie

Two answers with the same count, so more samples are needed.

Learn: Self-consistency and voting ›
Time series

Measurements taken in time order.

Learn: Forecasting over time ›
Time step

One turn of the loop: one word read.

Learn: Recurrent neural networks ›
Time to first token

How long until the first piece of the reply appears.

Learn: Time to first token and tokens per second ›
Token

A chunk of text the model treats as one unit.

Learn: Tokens ›also in 3 more lessons
Token budget

How many tokens you can afford to put in context.

Learn: Context engineering ›
Token count

How many tokens the dataset has. It sets how much the model sees.

Learn: Pretraining ›
Token ID

The number that stands for a token inside the model.

Learn: Tokens ›also in 1 more lesson
Token limit

Another name for the size of the window.

Learn: The context window ›
Tokenizer

The tool that splits text into tokens and back.

Learn: Tokens ›
Tokens per second

How many tokens the model writes each second once it starts.

Learn: Time to first token and tokens per second ›
Tokens per word

A rough measure of how finely a tokenizer cuts a language.

Learn: Tokens across languages ›
Tone

How a message feels: warm, formal, firm or casual.

Learn: Write and edit with AI ›
Tool

A function the app can run for the model, such as a calculator or search.

Learn: Tool use and function calling ›also in 2 more lessons
Tool call

A structured request from the model that the harness carries out.

Learn: The agent harness ›
Tool result

What the tool returned, which goes back into the prompt.

Learn: Tool use and function calling ›
Top-5 error

How often the right label is missing from the five best guesses.

Learn: ImageNet and AlexNet ›
Top-k

Keep only the k likeliest words, cut the rest.

Learn: Top-k and top-p sampling ›also in 3 more lessons
Top-p

Keep the smallest group of top words whose odds add up to p. Also called nucleus sampling.

Learn: Top-k and top-p sampling ›
Trace

The full record of thoughts, actions and observations.

Learn: Reason + act (ReAct) ›also in 1 more lesson
Trade-off

Lowering one usually raises the other, so you look for the balance.

Learn: Bias and variance ›also in 2 more lessons
Training

Showing a system many examples so it adjusts itself.

Learn: Rules vs learning ›also in 2 more lessons
Training data

The labelled examples used to teach the model.

Learn: Labels and supervised learning ›also in 2 more lessons
Training deck

The set of examples used to tune the model.

Learn: Instruction tuning ›
Training set

Examples the model learns from.

Learn: Train, validation and test sets ›
Transcript

The written text of what was said.

Learn: Talk to AI: voice and dictation ›
Transcription

Turning speech in a recording into written text.

Learn: Turn speech into text on your computer ›
Transfer learning

Reusing a pretrained model and training only a small part for your task.

Learn: Teach a model to recognise your photos ›
Transformer

A neural network design built from stacked attention and feed-forward blocks.

Learn: The Transformer at a glance ›also in 1 more lesson
Transparency duty

A rule to tell people when AI is involved.

Learn: AI regulation ›
Trap test

A question the facts cannot answer, to catch made-up replies.

Learn: Test your app with an AI judge ›
Trend

The slow rise or fall over time.

Learn: Forecasting over time ›
Troubleshooting

Finding which part broke by testing them one at a time.

Learn: Check your workbench end to end ›
True negative (correct rejection)

Predicted no and it really was no.

Learn: The confusion matrix ›
True positive (hit)

Predicted yes and it really was yes.

Learn: The confusion matrix ›
Truncate

Cut off the oldest tokens so the rest fit.

Learn: The context window ›
Tuning

A shorter stage that teaches the model to follow instructions and be helpful.

Learn: What is a large language model? ›
Turing test

A game where a judge tries to tell a machine from a person by text conversation.

Learn: The Turing test ›
Tutor prompt

A request that asks the chatbot to teach rather than to answer.

Learn: Learn anything with an AI tutor ›

U

Underfitting

The model is too simple to capture the pattern, so it does badly on training and test data.

Learn: Overfitting and underfitting ›
Unit vector

A vector scaled to length 1. Only its direction remains.

Learn: Cosine similarity ›
Universal approximation

The result that a network with enough neurons can approximate any smooth function as closely as you like.

Learn: Why networks can learn almost anything ›
Unseen (validation) photos

Photos held back from training, used to test honestly.

Learn: Teach a model to recognise your photos ›
Unsupervised learning

Learning from data with no labels, by finding structure.

Learn: Clustering (unsupervised) ›
Unsupported claim

A statement no source backs up.

Learn: Grounding and citations ›
Update

Moving each weight a small step against its gradient.

Learn: Backpropagation ›
Upsert

Add an item, or replace it if the same id already exists.

Learn: Set up a local vector database ›
Utilisation

The share of a chip's peak speed that is really used.

Learn: Compute and FLOPs ›

V

Validate

Check a reply against the schema.

Learn: Structured output ›
Validation

Checking that data matches the expected shape and types.

Learn: Pull structured data out of documents ›
Validation set

Examples used to compare models and settings.

Learn: Train, validation and test sets ›
Value

How good the AI thinks a position is for it.

Learn: AlphaGo and deep reinforcement learning ›also in 1 more lesson
Vanishing gradient

The signal shrinks to almost zero, so early layers stop learning.

Learn: Vanishing and exploding gradients ›
Vanishing memory

Old information fading as steps pile up.

Learn: Recurrent neural networks ›
Variance

How much a model's answers change when it is trained on a different sample of data.

Learn: Bias and variance ›
Vector

Another name for a list of numbers. An embedding is a vector.

Learn: Embeddings ›also in 1 more lesson
Vector database

A store built to keep embeddings and find the nearest ones fast.

Learn: Vector databases ›also in 1 more lesson
Vector store

A database that keeps embeddings and finds the nearest ones quickly.

Learn: Semantic search ›
Verbosity bias

Favouring longer answers just because they are longer.

Learn: AI judging AI ›
Verifiable reward

A reward from an exact check, such as a right answer or passing tests.

Learn: Reinforcement learning from verifiable rewards ›
Verifier

A separate check that decides whether an answer is acceptable.

Learn: Verifiers and checking ›also in 1 more lesson
Verify

Check a claim against a reliable source.

Learn: Check AI answers ›also in 2 more lessons
Virtual environment

A private folder of packages for one project.

Learn: Set up Python and a virtual environment ›also in 1 more lesson
ViT

Short for vision transformer.

Learn: Vision transformers ›
Vocabulary

The fixed, numbered list of every token a model knows.

Learn: Tokens ›also in 3 more lessons
Vocabulary size

How many entries the list has, often tens of thousands or more.

Learn: Vocabulary and token IDs ›
Vocoder

The part that makes the actual waveform.

Learn: Text to speech ›
Voice cloning

Software that copies how a person sounds from a short recording.

Learn: Spot AI scams and fake media ›
Vote

Counting each model's answer; the most common wins.

Learn: Random forests and ensembles ›
VRAM

The memory on a graphics card.

Learn: Choose a model that fits your computer ›

W

Watermark

A pattern hidden in a picture, sound or text that a scanner can find.

Learn: Deepfakes, watermarks and provenance ›
Web server

A program that answers requests from a browser.

Learn: Give your RAG app a simple web page ›
Weight

How strongly one input pushes the neuron. Negative weights hold it back.

Learn: The first artificial neuron ›also in 4 more lessons
Weight decay

Another name for L2 in neural-network training.

Learn: Regularisation ›
Weighted mix

An average where some items count more than others.

Learn: Attention is born ›
Weighted sum

Every input times its weight, added together.

Learn: A neuron: weights and bias ›
Width

How many numbers describe each token inside the model.

Learn: Parameters and model size ›
Word vector

A list of numbers that stands for a word's meaning.

Learn: Word vectors (word2vec) ›
word2vec

A 2013 method that learns a vector for each word from its neighbours.

Learn: Word vectors (word2vec) ›
Working directory

The folder the terminal is currently in.

Learn: Terminal basics for AI work ›
World model

A learned model that predicts what will happen next if you take an action.

Learn: World models ›

X

XOR

Exclusive or: true when exactly one of two inputs is true.

Learn: The XOR problem ›

Z

Zero-shot

A prompt with the task and no examples.

Learn: Zero-shot and few-shot prompting ›also in 1 more lesson
YOUR BROWSER, YOUR CHOICE