Chapter 18 · Hands-on: set up your AI workbench
Call your local model from code
Local runners offer a small web address on your own computer, so your scripts can send prompts and read replies.
Chapter 18 · Hands-on: set up your AI workbench
Local runners offer a small web address on your own computer, so your scripts can send prompts and read replies.
Your script writes a request…
The word localhost means “this same computer”. The runner you installed listens there, so any program on your computer can send it a message, the same way a web page asks a website.
You need the runner from the run-a-model lesson with at least one model downloaded, and the virtual environment with requests from the Python lesson. Open a terminal in my-ai-project and activate the environment.
Open http://localhost:11434 in your web browser. Ollama answers with a short message that it is running. If the page does not load, start the Ollama app (or ollama serve on Linux).
Check: the browser shows a plain message saying the runner is running.
In your terminal, go to your project folder and switch the box on. Skip this if the prompt already starts with (.venv).
cd my-ai-project
.venv\Scripts\Activate.ps1Check: the prompt starts with (.venv).
Create a file called ask.py inside my-ai-project with a plain-text editor (Notepad, or a code editor such as VS Code). Paste this in and save. Change the model name to the one ollama list shows.
The script posts a chat message as JSON, waits for the whole reply ("stream": False), and prints the text.
import requests
URL = "http://localhost:11434/api/chat"
MODEL = "llama3.2:1b" # use the exact name from `ollama list`
reply = requests.post(
URL,
json={
"model": MODEL,
"messages": [{"role": "user", "content": "Say hello in five words."}],
"stream": False,
},
timeout=120,
)
reply.raise_for_status()
print(reply.json()["message"]["content"])Check: the file is saved as ask.py, not ask.py.txt.
The first run is slower because the model loads into memory. Be patient for up to a minute.
python ask.pyCheck: a short greeting from the model prints in your terminal.
Edit the text after "content":, save, and run again. Add a first message with "role": "system" to set how the model should behave, as in the system prompt lesson. Try the request builder below to see the exact shape.
Check: the reply changes when your question changes.
Change the fields and watch the JSON the script sends. This shows the shape only. Nothing is sent from this page.
{
"model": "llama3.2:1b",
"messages": [
{
"role": "user",
"content": "What is a token?"
}
],
"options": {
"temperature": 0.2
},
"stream": false
}The reply is JSON too, and the answer is at message → content.
What do you see?
What does localhost mean?
Where is the model's answer in the reply?
Your script says connection refused. What is the likely cause?