How to Install Ollama on Windows and Run AI Offline

Watch the video

Install a local AI model on Windows

The goal of this tutorial is to install Ollama on Windows, download a small language model, talk to it locally, and then prove that the model can answer with the Wi-Fi switched off.

I use Claude and ChatGPT every day, but this was my first time installing a local model. I approached it from the beginning and wrote down what actually happened, including where the model gave me a confusing answer.

What you need

Ollama’s Windows documentation lists these system requirements:

Windows 10 22H2 or newer, Home or Pro
NVIDIA driver 551.61 or newer if you have an NVIDIA card
At least 4 GB of disk space for Ollama, plus space for each model you download
No administrator rights for the normal Windows installer

The qwen3.5:2b model used here is another 2.7 GB, so allow about 7 GB total for this walkthrough.

I tested this on a Lenovo 20YQ009QUS running Windows 11 Pro, with 16 GB of RAM, an Intel i5-11500H, and an NVIDIA T1200 with 3,935 MB of VRAM. I already had my browser and editor open, so this was an ordinary work-laptop test rather than a clean benchmark.

Step 1: Check your NVIDIA card and driver

If your computer has an NVIDIA card, open PowerShell and run:

nvidia-smi

This shows the installed driver and the available GPU memory. The Windows requirement is NVIDIA driver 551.61 or newer.

If you do not have an NVIDIA card, read Ollama’s current Windows and hardware-support documentation for the acceleration options that apply to your computer.

Step 2: Download and install Ollama

Go to the Ollama download page, choose Windows, and download OllamaSetup.exe. Run the installer and select Install.

The normal installer does not require administrator rights. By default, it installs Ollama in your user account and makes the ollama command available in PowerShell, Command Prompt, and other terminal applications. Ollama then runs in the background.

If you had Windows Terminal open before the installation, close every Windows Terminal window and open it again. Opening only a new tab may leave you with the old environment, which means PowerShell may not find the new ollama command yet.

Step 3: Confirm the installation

Open a fresh PowerShell window and run:

ollama --version

If PowerShell prints the installed Ollama version, the command is available and you can continue.

Step 4: Download and run the model

Run:

ollama run qwen3.5:2b

This one command downloads the model if you do not already have it and then opens an interactive chat.

The Ollama library lists qwen3.5:2b as a 2.27 billion parameter Q8 model with a 2.7 GB download. It accepts text and images and uses the Apache 2.0 license. The model page lists a maximum context window of 256K tokens, but Ollama currently defaults to 4K on systems with less than 24 GiB of VRAM. A larger context uses more memory.

Watch the prompt while the model starts:

PS C:\Users\your-name>

means you are in PowerShell. After the model loads, the prompt changes to:

>>>

That means you are talking to the model, not PowerShell. PowerShell commands will not run at the >>> prompt. If you type an ollama command there, the model may try to answer it as a question instead of running the command.

Step 5: Ask the model a question

For my first test, I asked:

Define a built-in function in Python

The model gave me an answer, but it initially mixed together built-in functions and user-defined functions. It then offered both interpretations. That was a useful reminder that a local model can produce a confident answer without telling you it is wrong or confused. Read the result and fact-check anything that matters.

The smaller model was useful for a quick question, but it was not a replacement for the larger cloud models I use for difficult reasoning and large codebases.

Step 6: Leave the model chat

At the >>> prompt, run:

/bye

You should return to the PowerShell prompt. You can also use /? inside the model chat to see the available slash commands in your installed version.

Step 7: Prove the model runs offline

Turn off Wi-Fi. I used the Wi-Fi toggle instead of Airplane mode because Airplane mode would also disconnect my Bluetooth headset.

If you want to check the connection from PowerShell, run:

Test-NetConnection ollama.com

This is a basic reachability check for one host, not proof that every possible network path is offline. In this walkthrough, switching off Wi-Fi is the step that disconnects the computer before the local-model test.

Then start the same model again:

ollama run qwen3.5:2b

The model was already stored on my laptop, so it loaded without downloading again. I asked:

Explain why Python is indented by spaces

It answered while the computer had no internet connection. Ollama’s documentation also states that when you run a model locally, Ollama does not see your prompts or responses. Windows and macOS versions of the Ollama application can still download software updates, so this test is specifically about the local model and its prompts.

Step 8: See what is installed and running

Back at the PowerShell prompt, list the downloaded models:

ollama ls

To see which models are currently loaded in memory, run:

ollama ps

Run ollama ps shortly after a response if you want to inspect the loaded model. Ollama keeps a model in memory for five minutes by default and then unloads it.

The PROCESSOR column shows where the model is loaded. 100% GPU means it is entirely in GPU memory. A split such as 47%/53% CPU/GPU means Ollama placed part of it in system memory and part in GPU memory.

On my installed version, running ollama with no arguments opened an interactive menu. The arrow keys moved through it, Enter launched the selected item, and Escape closed it.

Where Ollama stores its files on Windows

Open the main Ollama user folder with:

explorer "$env:USERPROFILE\.ollama"

The default Windows locations are:

Models and configuration: %USERPROFILE%\.ollama
Program files: %LOCALAPPDATA%\Programs\Ollama
Logs and downloaded updates: %LOCALAPPDATA%\Ollama

On my installation, the downloaded model data appeared under .ollama\models\blobs in files whose names started with sha256-. The .ollama directory can also contain an id_ed25519 private key, so do not share that file.

Remove the model or uninstall Ollama

To unload the model, delete its downloaded files, and confirm that it is gone, run these commands from PowerShell:

ollama stop qwen3.5:2b
ollama rm qwen3.5:2b
ollama ls

ollama rm deletes the local model, so you will need to download it again before using it later. If the model has already unloaded from memory, ollama stop may report that it is not running.

To uninstall Ollama itself, quit the background application, open Windows Settings, go to Apps and then Installed apps, find Ollama, and choose Uninstall.

If you also want to remove any remaining models, configuration, keys, logs, and downloaded updates, first check these paths carefully. The following commands permanently delete the two user-data directories:

Remove-Item -Recurse -Force "$env:USERPROFILE\.ollama"
Remove-Item -Recurse -Force "$env:LOCALAPPDATA\Ollama"

What I found

On this laptop, the 2B model was useful for quick questions and explanations, but its first answer also showed why I would not trust it without checking its work. It ran without an internet connection, required no account or API key for the local workflow, and did not create an API bill.

I am not replacing my cloud tools with it. I am adding a smaller, local option beside them.