How to Install Ollama: The Complete 2026 Guide

Ollama is the fastest path from zero to running a large language model on your own hardware. No API keys, no data leaving your machine, no per-token billing. Since its launch in 2023, Ollama has become the de-facto standard for local LLM inference — it handles model downloading, quantization selection, GPU acceleration, and exposes an OpenAI-compatible HTTP API, all with a single binary. This guide covers installation on every platform, model selection for your hardware, and the first things to do after install.

What Ollama Actually Does

Ollama is a runtime that manages open-weight language models on your local machine. It downloads models from the Ollama registry (which mirrors Hugging Face), selects the best quantization for your available VRAM, loads the model into GPU memory, and serves it via a REST API on port 11434. The API is intentionally compatible with OpenAI's format — meaning any tool that supports a custom base URL (Continue.dev, Open WebUI, AnythingLLM, LangChain, the OpenAI Python SDK) works with Ollama without modification. Your prompts never leave your machine.

Hardware Requirements

Ollama runs on any modern Mac, Linux machine, or Windows PC. GPU acceleration is optional but strongly recommended — without it, inference is 5–20x slower. The practical minimum for a useful experience is 8 GB of system RAM (for a 7B model on CPU) or 8 GB of VRAM (for a 7B model on GPU). The formula for estimating whether a model fits in your VRAM is: Model Parameters (billions) × (bits per weight / 8) = GB required. A 7B model at 4-bit quantization needs approximately 4.4 GB of VRAM. A 32B model at 4-bit needs approximately 20 GB.

Install on macOS

Download the Ollama macOS app from ollama.com. It installs as a menu bar application and automatically detects your Apple Silicon or NVIDIA GPU. After installation, open Terminal and run your first model:

# Pull and run Llama 3.2 3B (fast, fits in 4 GB VRAM)
ollama run llama3.2

# For Apple Silicon with 16+ GB RAM — Qwen3.5 9B is excellent
ollama run qwen3.5:9b

# For 32+ GB RAM — run a 32B model at full quality
ollama run qwen3.5:27b

# List all downloaded models
ollama list

Install on Linux

On Linux, Ollama installs as a systemd service via a one-line installer. It automatically detects NVIDIA GPUs (via CUDA) and AMD GPUs (via ROCm). Intel Arc GPUs are supported in preview.

# Install Ollama (installs to /usr/local/bin, creates systemd service)
curl -fsSL https://ollama.com/install.sh | sh

# Verify the service is running
systemctl status ollama

# Pull and run a model
ollama run qwen3.5:9b

# Check GPU is being used (should show GPU layers loaded)
ollama run qwen3.5:9b --verbose

Install on Windows

Download the Ollama Windows installer from ollama.com. It requires Windows 10 or 11 and installs as a background service. NVIDIA GPU acceleration works via CUDA; AMD GPU support on Windows is in preview. After installation, open PowerShell or Command Prompt:

# In PowerShell or Command Prompt
ollama run llama3.2

# Check available models
ollama list

# Check Ollama version
ollama --version

Choosing the Right Model

The right model depends on your hardware and use case. As of 2026, Qwen3.5 models from Alibaba are the best-in-class for most tasks at every size. For coding, DeepSeek V3 is the strongest option. For reasoning tasks, DeepSeek-R1 (distilled versions) punches above its weight. Llama 4 Scout (17B active parameters via MoE) is Meta's latest and runs well on 24 GB VRAM.

Model Selection by Hardware

Use this table as a starting point. All models listed are available via ollama pull:

The Ollama API

Once Ollama is running, it exposes a REST API at http://localhost:11434. The two most useful endpoints are /api/generate (single-turn completion) and /api/chat (multi-turn conversation). Both support streaming responses. The API is intentionally compatible with OpenAI's format at /v1/chat/completions, so you can use the OpenAI Python SDK directly:

from openai import OpenAI

# Point the OpenAI SDK at your local Ollama instance
client = OpenAI(
    base_url="http://localhost:11434/v1",
    api_key="ollama",  # Required but ignored by Ollama
)

response = client.chat.completions.create(
    model="qwen3.5:9b",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Explain quantization in one paragraph."}
    ]
)
print(response.choices[0].message.content)

What to Do Next

Once Ollama is running, the most common next step is adding a web interface. Open WebUI is the best option — it gives you a ChatGPT-like interface that connects to your local Ollama instance, supports multi-model switching, document upload (RAG), and user management. Install it with Docker: docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway ghcr.io/open-webui/open-webui:main. For running Ollama on a cloud GPU with more VRAM than your local machine, RunPod Secure Cloud is the recommended option — an RTX 3090 pod costs approximately $0.44/hour and gives you 24 GB of dedicated VRAM.