Chapter 4 · The deep learning era (2006–2016)2013
Word vectors (word2vec)
word2vec learned word meanings as vectors, where king − man + woman ≈ queen.
Chapter 4 · The deep learning era (2006–2016)2013
word2vec learned word meanings as vectors, where king − man + woman ≈ queen.
Words start scattered
On the map, the step from man to woman is an arrow. The same arrow, placed at king, ends at queen. Relationships became directions you can add and subtract.
From the embeddings lesson: each word gets a vector, a list of numbers, and similar words sit close together. Here we use just two numbers, so you can see the map.
Pick A, B and C. The map shows A − B + C as a pink star, and the nearest word is the answer. Try the examples, then try something odd like king − Paris + India.
king − man + woman ≈ queen
Next nearest: aunt, Delhi. Distance to the answer: 0.07 (smaller is a better match).
A teaching version: fourteen words placed by hand on a flat map. Cyan is the arrow from B to A, pink dashed is that arrow applied to C. Real word2vec vectors are learned, have hundreds of numbers, and analogies only work roughly.
How did word2vec learn its vectors?
In king − man + woman, what does “− man + woman” do?
Why does the answer exclude the words you typed?