Chapter 19 · Hands-on: build real projects
Set up a local vector database
A vector database stores embeddings with their text and finds the closest ones to a question, all on your computer.
Chapter 19 · Hands-on: build real projects
A vector database stores embeddings with their text and finds the closest ones to a question, all on your computer.
[0.12, −0.40, …]closest[0.31, 0.22, …]closest[−0.50, 0.08, …]closest[0.02, 0.61, …]closestYour notes arrive…
A vector database is like a filing cabinet that files each note under its numbers. Ask a question, turn it into numbers too, and the cabinet opens the nearest drawers.
Last time, in Make embeddings on your computer, you turned sentences into numbers and compared them in a script. That works for six sentences, but a real project has thousands, and you do not want to recompute them every time. A vector database stores the text, its numbers and a label for each, and finds the closest ones fast. It stays in a folder on your computer.
We use the same pdf-chat folder and virtual environment as before. One popular option is Chroma; other local databases work in a similar way.
Open a terminal, move into pdf-chat and activate the virtual environment (see Set up Python and a virtual environment if you forgot how). Then install Chroma, which is called chromadb on pip.
cd pdf-chat
.venv\Scripts\Activate.ps1
pip install chromadb Check: The line starts with (.venv) and the install ends without a red error.
Create a plain text file called notes.txt in pdf-chat, with one note per line. Use the example below or, better, write notes about something you know: a hobby, your class, a recipe. Aim for at least five lines so searching feels real.
The library closes at 8 pm on weekdays.
Books can be borrowed for two weeks.
The canteen serves lunch from 12 to 2.
Wi-Fi passwords are changed every month.
Exam forms must be submitted before the 10th.Check: Each note sits on its own line, and the file is saved as plain text (not .docx).
This script reads each line, makes its embedding, and saves the text, the numbers and a label (the file name and line number) in a local folder called db. We ask for cosine distance, which compares meaning. upsert means “add, or replace if the id already exists”, so you can run it again safely. Save it as add_notes.py, or use the builder below to make one with your own names.
import chromadb
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("all-MiniLM-L6-v2")
# A folder called "db" is created next to this script and holds everything
client = chromadb.PersistentClient(path="db")
collection = client.get_or_create_collection(
name="notes", metadata={"hnsw:space": "cosine"}
)
with open("notes.txt", encoding="utf-8") as f:
lines = [line.strip() for line in f if line.strip()]
collection.upsert(
ids=[f"notes-{n}" for n in range(1, len(lines) + 1)],
documents=lines,
embeddings=model.encode(lines, normalize_embeddings=True).tolist(),
metadatas=[{"source": "notes.txt", "line": n} for n in range(1, len(lines) + 1)],
)
print("Stored", collection.count(), "notes") Check: Nothing to see yet; the file is saved next to notes.txt.
Run the script. A folder named db appears in pdf-chat; that is your database. It is ordinary files, so you can copy or delete it like any folder.
python add_notes.py Check: The last line says Stored 5 notes (or the number of lines in your file). Running it again prints the same number, not double.
Now the payoff. This script turns your question into numbers, asks the database for the two closest notes (n_results=2) and prints each with its score and label. It only reads the database; nothing is stored again. Save as search_db.py. In the terminal, put the question after the file name.
import sys
import chromadb
from sentence_transformers import SentenceTransformer
question = " ".join(sys.argv[1:]) or "How long can I keep a book?"
model = SentenceTransformer("all-MiniLM-L6-v2")
collection = chromadb.PersistentClient(path="db").get_collection("notes")
result = collection.query(
query_embeddings=[model.encode(question, normalize_embeddings=True).tolist()],
n_results=2,
)
for text, meta, dist in zip(result["documents"][0], result["metadatas"][0], result["distances"][0]):
print(f"{1 - dist:.2f} {text} ({meta['source']}, line {meta['line']})")python search_db.py "When do I have to hand in the exam form?" Check: The exam form note comes first with a clearly higher score than the second line, and the label shows notes.txt, line 5. Scores are similarity: higher means closer.
Close the terminal, open it again, activate the environment and run another search. The database is still there because it was saved in the db folder. Try a question with none of the note’s words, such as “Is the building open in the evening?”. Then add a new line to notes.txt, run add_notes.py again and search for it.
python search_db.py "Is the building open in the evening?"Check: The library closing time is near the top, even though the question shares no keywords.
Choose your collection name, database folder and notes file, then copy the script. This writes Python for you; it does not run a model.
import chromadb
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("all-MiniLM-L6-v2")
client = chromadb.PersistentClient(path="db")
collection = client.get_or_create_collection(name="notes", metadata={"hnsw:space": "cosine"})
with open("notes.txt", encoding="utf-8") as f:
lines = [line.strip() for line in f if line.strip()]
collection.upsert(
ids=[f"notes-{n}" for n in range(1, len(lines) + 1)],
documents=lines,
embeddings=model.encode(lines, normalize_embeddings=True).tolist(),
metadatas=[{"source": "notes.txt", "line": n} for n in range(1, len(lines) + 1)],
)
print("Stored", collection.count(), "notes")Use the same collection name and folder in your search script. Nothing you type here leaves this page.
What do you see?
Why store embeddings in a database instead of recomputing them each time?
What happens if you store with one embedding model and search with another?
Where is your Chroma data kept in this tutorial?