Chapter 20 · Hands-on: more projects
Give your RAG app a simple web page
A small Python web library can put a chat box in front of your RAG app, running on your own computer.
Chapter 20 · Hands-on: more projects
A small Python web library can put a chat box in front of your RAG app, running on your own computer.
You type a question in the page…
A tiny web server on your own computer. Your browser shows a question box; the server reuses the search, prompt and model code from the PDF chat tutorial and sends back the answer with its sources.
This builds on Chat with your PDFs: a simple RAG app. You need that project working: the folder with ask.py, chunks.json, the db folder, the virtual environment, and your local model running. If you have not done it yet, do that tutorial first.
A web server is a program that answers requests from a browser. Flask is one popular small Python library for writing one. Your page will run on your own computer; nothing is put on the internet. Windows, macOS and Linux desktops all work.
Go into the pdf-chat folder from the earlier tutorial, activate its virtual environment and install Flask.
cd pdf-chat
.venv\Scripts\Activate.ps1
pip install flask Check: The prompt starts with (.venv) and the install ends without red errors. Running python ask.py "test" still works.
Your ask.py has three functions: retrieve (find chunks), build_prompt (put question and chunks together) and ask_model (call the local model). Its last lines run only when you start it directly (if __name__ == "__main__"), so another file can import the functions without running the command-line part. The web server will do exactly that.
Check: You can find the three function names in ask.py.
Create webapp.py next to ask.py. It has two routes (addresses the server answers): / sends the page, and /ask takes a question, runs your three functions and returns the answer and sources as JSON. The page’s small script sends the question and shows the reply.
from flask import Flask, jsonify, request
from ask import ask_model, build_prompt, retrieve
app = Flask(__name__)
PAGE = """<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>Chat with my PDFs</title>
<style>
body { font-family: system-ui, sans-serif; max-width: 42rem; margin: 2rem auto; padding: 0 1rem; line-height: 1.5; }
form { display: flex; gap: 0.5rem; }
input { flex: 1; padding: 0.6rem; font-size: 1rem; }
button { padding: 0.6rem 1rem; font-size: 1rem; }
#answer { white-space: pre-wrap; margin-top: 1.5rem; }
.source { background: #f4f0fb; border-radius: 8px; padding: 0.6rem; margin-top: 0.6rem; font-size: 0.9rem; }
.error { color: #b3261e; }
</style>
</head>
<body>
<h1>Chat with my PDFs</h1>
<form id="form">
<input id="question" placeholder="Ask about your documents" autocomplete="off" required>
<button id="go">Ask</button>
</form>
<div id="answer" aria-live="polite"></div>
<div id="sources"></div>
<script>
const form = document.getElementById("form");
const box = document.getElementById("question");
const go = document.getElementById("go");
const answer = document.getElementById("answer");
const sources = document.getElementById("sources");
form.addEventListener("submit", async (event) => {
event.preventDefault();
go.disabled = true;
answer.className = "";
answer.textContent = "Thinking...";
sources.replaceChildren();
try {
const reply = await fetch("/ask", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ question: box.value }),
});
const data = await reply.json();
if (!reply.ok) throw new Error(data.error || "Something went wrong.");
answer.textContent = data.answer;
for (const s of data.sources) {
const div = document.createElement("div");
div.className = "source";
div.textContent = "[" + s.n + "] " + s.source + ", page " + s.page + ": " + s.text;
sources.appendChild(div);
}
} catch (problem) {
answer.className = "error";
answer.textContent = problem.message;
} finally {
go.disabled = false;
}
});
</script>
</body>
</html>
"""
@app.get("/")
def home():
return PAGE
@app.post("/ask")
def ask():
data = request.get_json(silent=True) or {}
question = str(data.get("question", "")).strip()
if not question:
return jsonify(error="Type a question first."), 400
try:
hits = retrieve(question)
answer = ask_model(build_prompt(question, hits))
except Exception as problem:
return jsonify(error=f"Could not get an answer: {problem}"), 500
sources = [
{"n": n, "source": meta["source"], "page": meta["page"], "text": text}
for n, (text, meta) in enumerate(hits, start=1)
]
return jsonify(answer=answer, sources=sources)
if __name__ == "__main__":
# 127.0.0.1 means "only this computer can open the page"
app.run(host="127.0.0.1", port=8000) Check: The file is saved as webapp.py in the same folder as ask.py.
Run the file. The first start loads the embedding model, as before, so give it a moment. The program keeps running: that is the server. Leave this terminal open.
python webapp.py Check: You see a message that it is running on http://127.0.0.1:8000.
In your browser go to http://127.0.0.1:8000. Type a question your PDFs can answer and press Ask. Small local models can take a while on a CPU, and the button stays disabled until the answer arrives.
Check: You see an answer, and under it boxes such as [1] handbook.pdf, page 1: ….
Real apps need to cope with problems. Try an empty question (the page asks you to type one). Then stop your model runner and ask again: you should see a readable error in red instead of a blank page. Start the runner again afterwards.
Check: Both cases show a clear message and the page keeps working.
Click the terminal and press Ctrl+C to stop the server. Closing the browser tab does not stop it. Later you can improve the page: add a history of questions, a button to clear it or a drop-down to choose which PDF to search.
Check: The terminal prompt returns and the page no longer loads.
Choose who may open the page and a port number to see the exact last line of webapp.py. This only writes text for you.
Who should be able to open the page?
app.run(host="127.0.0.1", port=8000)Then open http://127.0.0.1:8000 in your browser.
Safe default. Only programs on this computer can open the page. Nobody else on your Wi-Fi or network can.
What do you see?
Why can webapp.py import functions from ask.py without running a question?
What does host 127.0.0.1 mean?
Why does the page use textContent instead of putting the answer in as HTML?