Ollama: How to Run AI Models Locally on Your Computer

👁 117 reads

Ollama is a free program that runs language models on your computer: you install it from ollama.com, type ollama run gemma4, and after downloading the model you can chat with it offline, without an account and without sending anything outside. It works on Windows 10 and 11, macOS 14 and up, and Linux. For practical work, you need a graphics card with 8 GB of memory or a Mac with an M processor. Below: installation, commands, memory requirements, which model to choose, and what changed in September 2026.

Ollama at a Glance

This answers the question everyone asks after thinking about pasting a company document into a chat: can you have this on your own machine? You can, with one caveat I’ll return to below: a model on a regular computer is smaller than the one behind a paid chat service.

How to Install Ollama

1. Go to ollama.com and click Download. On Windows, the installer doesn’t require administrator privileges and installs in the user directory. The program itself takes about 4 GB, models are downloaded separately.
2. On Mac, drag the application to the Applications folder. On first start, Ollama will check if the ollama command is available in the terminal and ask permission to add it.
3. At first launch, choose to work locally. Login is only needed for cloud models.
4. Download your first model: in the application window select it from the list or type in the terminal ollama run gemma4:12b.

Ollama runs in the background and provides a local endpoint at http://localhost:11434, compatible with the OpenAI standard. That’s why many ready-made tools work with it after changing just the address, without rewriting code.

Commands That Are Enough to Start

You can work in the chat window application without the terminal, but the terminal is faster. For the first week, a few commands are enough.

Command	What it does
ollama run model-name	downloads model on first run and immediately opens a conversation
ollama pull model-name	download only, without running, useful before traveling without internet
ollama list	shows what you have on disk and how much it weighs
ollama ps	shows which model is currently in memory and whether it’s running on the card or processor
ollama stop model-name	frees card memory without closing the program
ollama rm model-name	deletes model when you run out of space
ollama launch	configures and launches external tools, e.g. Claude Code, Codex or OpenCode, on a model from Ollama

How Much Memory You Really Need

Rule of thumb: a model in its default, compressed version takes roughly half of its billions of parameters, expressed in gigabytes, plus overhead for conversation context. Example from Ollama’s library: Gemma 4 12B weighs 7.6 GB.

Model size	Space required	What you can realistically do
3-4 billion	about 2-3 GB	summaries, simple rewrites, experimentation and learning
7-8 billion	about 4-5 GB	first level where work starts to make sense
12-14 billion	about 8-9 GB	longer texts, simple programming tasks, good quality
20-30 billion	12 to 20 GB	card with 16-24 GB memory or Mac with large shared memory

Most important in this table: graphics card memory matters, not processor speed. If the model doesn’t fit in card memory, the processor takes over part of the work and responses slow from seconds to minutes. That’s why when choosing a model I first look at whether it will fit, and only then at rankings. Detailed tables for specific models and cards are in the article how much memory you need to run an AI model locally.

Hardware Requirements According to Documentation

• Windows: 10 version 22H2 or newer, Home or Pro. NVIDIA cards with driver 551.61 or newer, AMD Radeon cards through ROCm v7 or Vulkan (for some Radeon RX 6000 cards Vulkan is the recommended path).
• macOS: Sonoma (14) or newer. On Macs with M processor both card and processor compute, on older Intel ones only the processor.
• NVIDIA generally: cards with compute capability 5.0 and newer. The oldest ones (5.0-6.2) need driver 570+.

Which Model to Choose

Open models differ in quality much more than commercial chats. Some write well, some produce grammatically correct sentences but sound like translations. On September 26, 2026, we tested seven free models on an RTX 4090 card with the same test as cloud models: 13 writing and editing tasks plus a quiz, evaluated blindly by judges from three companies. We ran the test in llama.cpp, but you can download the same models in Ollama. We tested in Polish, a heavily inflected language, making this a useful proxy for non-English writing quality.

For comparison, Claude Opus 5.5 from the cloud got 94.2 points in the same round, and GPT-6 Luna 86.4. Full results are in the article test of 7 local models in Polish. In Ollama you run Gemma with the command ollama run gemma4:12b, Qwen with ollama run qwen3.8, and you’ll find Bielik in the library under the name SpeakLeash/bielik-11b-v3.0-instruct. Step-by-step instructions for Bielik are in the article how to run Bielik locally.

One trap from this test: models with thinking mode (Gemma 4, Qwen) with default settings can consume ten times more time and tokens, and in extreme cases not return an answer at all. For regular writing, turn off thinking. From version 0.34.3, Ollama shows in /api/show what thinking settings each model has and which are default. Why a local model is sometimes slower and weaker than the model card promises is described in the article why local LLM works worse than it should.

My order when choosing: I take the smallest model that gives a sensible result on my task, not the largest that fits. Smaller responds faster, and when working with text the number of attempts matters more than the quality of a single response. I check two things that open models don’t like: proper name declension and maintaining the certainty level of statements. Rankings calculated in English are a weak guide here. Among open models, Chinese solutions are also strong, described in the guide to DeepSeek, and I collect differences between commercial versions in the AI model ranking.

When This Makes Sense

• Documents you can’t send out. Contracts, personal data, confidential materials. This is the main and strongest reason.
• Working without internet or in a place with cut-off access.
• Mass tasks. Processing tens of thousands of texts, where billing for usage at a provider would become expensive.
• Learning and experiments. You check how different models behave without paying for each attempt.
• Automation. A local endpoint compatible with OpenAI lets you connect the model to your own scripts and tools like n8n, which can also be run locally.

What’s New in Ollama in September 2026

• 0.34.0 (September 5): models from Ollama can be used in the ChatGPT desktop application; configuration is launched from the Ollama application on macOS.
• 0.34.2 (September 15): at first launch, the wizard asks whether to log in or work locally.
• 0.34.3 (September 19): information about thinking settings of each model in API, fixes for macOS application and downloading models from Hugging Face.
• 0.34.4 (September 23): current stable version.
• In testing: 0.40 moves supported models on Macs with M processor to the MLX engine, and 0.35 adds so-called decision models that return a choice with probabilities instead of text, e.g. for classifying requests.

The company is also developing a paid cloud: the same open models (DeepSeek, Kimi, GLM, Qwen and others) running on Ollama’s servers in the US, Europe and Singapore. The free plan has a starting credit pool, Pro costs $20 per month with $60 in credits, Team $500. According to Ollama, queries from the cloud are not recorded or used for training, but this is no longer local work: data leaves your computer. Models run locally remain without limit and without fees. I wrote about the company’s financing in the article about Ollama raising $65 million.

Three Things That Break Most Often

Model responds very slowly. Almost always means it doesn’t fit in card memory and is computing on the processor. ollama ps will show the split between card and processor. The solution is a smaller model or more compressed version, not a faster computer.

Ran out of disk space. Models don’t delete themselves, and it’s easy to download a dozen when testing. ollama list and ollama rm solve this in a minute.

Responses cut off mid-way. This is usually a response length limit or overflowed conversation context, not a crash. With long documents, it’s better to divide the material than to count on the model maintaining the whole.

What Ollama Won’t Do

It won’t replace a commercial chat in tasks requiring the highest quality. In our test, the best local model was over 20 points behind Claude Opus 5.5 and this shows in longer reasoning. You’re not buying quality here, but privacy, no limits and no bill.

It also won’t give access to the latest information. The model’s knowledge ends at its training date, unless you connect web search, which Ollama offers as a separate function for logged-in users.

And something easy to forget: local doesn’t automatically mean secure. Data doesn’t go outside, but it sits on disk, in conversation history, just like any other file.

Ollama or LM Studio

Both run the same models, so the choice doesn’t close any path; they differ in how you work. Ollama is a local server with command line and simple chat window: best when the model is to be part of a script, automation or workflow in n8n, and when you want the same environment on multiple machines. LM Studio is a full-featured program with model search, setting sliders and conversation history: better when you want to compare models and parameters without the terminal. Many people keep both: LM Studio for browsing and testing models, Ollama for running what they’ve already chosen. If you want to first try an OpenAI model on your own hardware, start with the guide GPT-OSS-20B locally.

See also: What is LLM and what is a token - two concepts without which it’s hard to choose a model.

Models on your own computer: Ollama from command line, LM Studio in a window, Whisper for recordings and Bielik for Polish-language tasks.

Frequently Asked Questions

Is Ollama free?

Yes. The program and running models on your own computer are free and without limit. Only optional models in Ollama’s cloud are paid, starting with the Pro plan at $20 per month.

How to install Ollama on Windows?

Download the installer from ollama.com and run it; it doesn’t require administrator privileges. You need Windows 10 22H2 or newer. After installation, type in the terminal ollama run gemma4:12b, and the program will download the model and open a conversation. You can also use the application window without the terminal.

How much RAM does Ollama need?

Graphics card memory is what mainly counts. A 7-8 billion parameter model takes about 4-5 GB, 12-14 billion about 8-9 GB, and 20-30 billion from 12 to 20 GB. If the model doesn’t fit in card memory, the processor takes over part of the work and responses slow down significantly.

Which Ollama model for non-English writing?

In our test from September 26, 2026, Gemma 4 12B (72.3 points, 8 GB card memory) and Qwen 3.8 27B (72.2 points, 17 GB) performed best. Bielik 11B has the cleanest output and best knowledge about Poland in Polish, but follows instructions less well.

Does Ollama require an account?

No. At first launch you can choose to work locally. An account is only needed for cloud models and web search.

Does Ollama work without internet?

Yes, after downloading the model. Internet is only needed to install the program and download models, so before traveling use ollama pull.

Does Ollama send my data?

Local models run entirely on your computer. Data goes outside only when you consciously use a cloud model (has “cloud” in the name) or web search.

See also: Local AI: text models, images and video on your own computer · test of 7 local models in Polish

Kinetyra