Chapter 5 · Turning language into numbers
Vectors and dimensions
A vector is a list of numbers; each number is one dimension, and models use hundreds or thousands.
Chapter 5 · Turning language into numbers
A vector is a list of numbers; each number is one dimension, and models use hundreds or thousands.
2 numbers: an arrow on a flat page.
Two numbers place an arrow on a page. Three place it in space. Models use hundreds or thousands, which we cannot draw, but the rules stay the same.
[0.6, 0.5] is a vector with two numbers. You can read it as an arrow: go 0.6 across and 0.5 up. Vectors are how models hold anything they work with: a word, a picture, a sentence.
Choose how many dimensions, then slide each number. With 2 or 3 dimensions you see an arrow. With more, you can only see the numbers as bars.
Real embeddings have hundreds or thousands of dimensions, not 2, 3 or 6. This tool shrinks the idea so you can see it.
How many dimensions does the vector [0.2, −0.5, 0.9] have?
Why do models use hundreds of dimensions for a word?
Can we draw a 768-dimensional arrow?