Ollama: Your Own AI on a Computer Without the Cloud – Installation, Models, Polish Bielik, and What Hardware to Choose (2026)

ChatGPT, Claude, and Gemini operate in the cloud: what you type goes to the manufacturer's servers. There is a second way: download an AI model to your own computer and talk to it without the internet, without an account, without a subscription, and without sending anything to anyone. The simplest program for this is called Ollama. This guide explains in simple terms what it is, what it can do and cannot do, how it differs from paid assistants, how to install it on Mac, Windows, and Linux, what hardware is needed, and why a Mac with an Apple Silicon processor is particularly good for this. It also includes the Polish model Bielik, which performs better in Polish than most open models.

Last verification: September 6, 2026. Informational material. Software versions, model names and sizes, and hardware parameters change every few months; numbers come from Ollama, Apple, NVIDIA, and AMD documentation as of the verification date. Before purchasing hardware for AI, check the current data from the manufacturer.

In Brief

What is Ollama and a “local model,” in simple terms

An AI assistant consists of two things: a model, which is a huge file of “knowledge” created from training on texts, and a program that runs this file and allows you to talk to it. In ChatGPT, both things are on OpenAI's servers. However, some companies publicly share their models as downloadable files; these are called open models or models with open weights. Ollama is a program that downloads such files and runs them on your computer, then gives you a chat window similar to ChatGPT.

Models are sized by the number of parameters: 1B, 4B, 8B, 27B, 70B (B for billion). The more parameters, the smarter the model, but the more memory it requires and the slower it responds. A 4B model fits on a regular laptop and responds quickly; a 70B model requires a computer costing several thousand dollars. In comparison, cloud models, like those in ChatGPT or Claude, are significantly larger and run on thousands of graphics cards; therefore, a local model will never be as “smart,” but it does not need to be for many everyday tasks.

Cloud or own computer: a fair comparison

When local makes sense: when working with documents you do not want to send to anyone (contracts, client data, family, medical, accounting matters); when you want an assistant without a subscription and limits; when the internet can be unreliable; when you want to learn how it works. When it does not make sense: when you need the highest quality for difficult tasks, current information from the web, or images and video. Many people have both: the cloud for difficult tasks, Ollama for confidential ones.

Hardware: one rule that explains everything

For the model to respond quickly, its entire file must reside in fast memory accessible to the computing unit (graphics card). When writing each subsequent word, the computer scans the entire model from start to finish. This leads to two conclusions:

1. The amount of memory determines how large a model you can run. Ollama's rule: 7B needs 8 GB, 13B 16 GB, 33B 32 GB, and 70B 64 GB (Ollama documentation). Models in Ollama are by default “compressed” (quantization), which means they take about half a byte per parameter: 4B is about 3 GB, 8B about 5 GB, 27B about 17 GB (sizes from Gemma 3 page, Qwen3 page). Add a few gigabytes for conversation and system to the file size.
2. Memory bandwidth determines how fast the model writes. Since each word requires reading the entire model, the number of words per second is approximately equal to memory bandwidth divided by model size. Memory with 460 GB/s and a 7 GB model is at most about 60 words per second; regular PC laptop memory, a few dozen GB/s, writes the same model a few words per second. This is an approximation, but it explains well why some computers “fly” while others “grind.”

PC with a graphics card: fast, but little memory

A regular PC has two types of memory: RAM for the CPU (large, slow) and VRAM on the graphics card (small, very fast). The model runs well only if it fits in VRAM. The strongest consumer NVIDIA card, GeForce RTX 5090, has 32 GB (NVIDIA); RTX 5080 has 16 GB with a bandwidth of 960 GB/s (NVIDIA). This means: a card costing several thousand dollars is very fast, but fits a model of up to about 30B; 70B models require two cards or partial use of free RAM, which drastically slows down. Ollama supports NVIDIA cards (from the generation with compute capability 5.0, driver 550 or newer), selected AMD cards, and Intel chips via Vulkan; without a card, it runs slowly on the CPU (about hardware support).

Mac with Apple Silicon: unified memory, or why it works so well

In Apple M processors (M1, M2, M3, M4, M5), the CPU and GPU share one unified, fast memory; Apple calls this unified memory. The effect for AI is simple: almost all of the computer's memory can be used for the model, and this memory is fast. The MacBook Pro with the M5 Max chip has up to 128 GB of memory with a bandwidth of 460 or 614 GB/s, M5 Pro 307 GB/s, basic M5 153 GB/s (Apple data). The Mac Studio with M5 Ultra has 96, 256, or 512 GB at 1.2 TB/s (Apple data); the previous generation with M3 Ultra offered up to 512 GB at 819 GB/s (Apple data). Ollama uses the Apple GPU via Metal (documentation).

Compare: a laptop costing about three thousand dollars with 64 GB of unified memory will run a 70B model, which no single consumer card can fit on a PC. A PC card will be faster with small models, but a Mac will run larger ones. Therefore, among those working with local models, the Mac Studio with large memory has become the standard. Note two things: a Mac with 8 GB of memory will only run the smallest models, and a Mac cannot be upgraded after purchase; you choose the memory once.

The third way: PC with unified memory

AMD produces Ryzen AI Max+ 395 processors with up to 128 GB of unified memory, of which up to 96 GB can be used for the GPU (AMD data). This is PC's answer to Apple's idea; computers with this chip are cheaper than a Mac with similar memory, but memory bandwidth is lower, so large models write slower. Ollama supports these chips as AMD cards (documentation).

Recommendations: what to buy or what you have

Model sizes: Gemma 3 4B 3.3 GB, 12B 8.1 GB, 27B 17 GB (Ollama); Qwen3 8B 5.2 GB, 14B 9.3 GB, 30B 19 GB, 235B 142 GB (Ollama); Llama 3.3 70B 43 GB (Ollama); gpt-oss 20B about 14 GB, “for computers with 16 GB memory”, 120B about 65 GB (Ollama); Bielik 11B from 6.7 GB (Q4) to 22 GB (full precision) (Ollama). If you are just buying a computer and thinking about AI: more memory is more important than a faster processor.

Installation Step by Step

Mac

Requirements: macOS Sonoma (14) or newer; Apple M processors with GPU support, Intel only on the processor (documentation).

1. Download

   Go to ollama.com/download/mac and download the file. Open it and drag the Ollama icon to the Applications folder.

2. Run

   Open Ollama from Applications. On the first launch, it will ask for permission to add a command to the terminal; agree, it will be useful later.

3. Choose a model

   In the chat window, there is a list of models to choose from; the first use downloads the model (a few GB, it takes a moment). To start, choose Gemma 3 4B or Qwen3 8B if you have 16 GB of memory.

4. Write your first question

   The window works like ChatGPT. You can drag PDF and text files into the window; images are understood by models marked as supporting images (about the application).

Windows

1. Download and install

   ollama.com/download/windows, run the installer, click next (documentation).

2. Graphics card

   If you have an NVIDIA card, keep the drivers up to date; Ollama will detect it automatically. Without a card, it runs on the CPU, slower.

3. Choose a model and write

   Same as on Mac: chat window, list of models, dragging files.

Linux

One command in the terminal, from Ollama documentation; for NVIDIA cards, CUDA drivers are needed, for AMD ROCm version 7:

First Commands in the Terminal

The application with the chat window is sufficient for most people. The terminal provides more control and is necessary on Linux. On Mac, use the “Terminal” program, on Windows “PowerShell”; paste the command and press Enter. Commands come from Ollama documentation.

After “ollama run,” a prompt appears, and you write as in chat. The first run downloads the model; subsequent starts take seconds. The model is released from memory after five minutes of inactivity and loads again on the next question (documentation).

Which Model to Start With

Sources of sizes: model pages in the Ollama library linked above and in the sources table. Names with a colon (gemma3:12b) are specific variants; without a colon, Ollama takes the default. The Q4 variant is a model compressed four times: smaller and faster, slightly less accurate; the Bielik page warns that compressed models “exhibit reduced response quality and possible hallucinations.” For everyday tasks, Q4 is sufficient; for precision work, take Q6 or Q8 if you have the memory.

Prompts to Try Locally

The local model does not have the internet, so ask about things it has “inside” or in the attached file. Four elements of a good prompt (role, context, task, format) from the ChatGPT guide work the same way here, and even more importantly, because a smaller model guesses what you mean worse.

The response to prompt 4 says a lot about the model: a good one will write that it does not know the date or the news. A weak one will make something up. Making things up is the same flaw as in the cloud, only more frequent with smaller models; you check important facts.

Settings Worth Knowing

Context length – how much text the model sees at once. By default, Ollama sets 4096 tokens, which is a few pages; with long documents, this needs to be increased in the application settings or the OLLAMA_CONTEXT_LENGTH variable, at the cost of higher memory usage (documentation). If the model “does not see” the end of the attached file, this is the reason.

Where are the models – on Mac in the folder ~/.ollama/models, on Windows in C:\Users\name\.ollama\models; this can be changed with the OLLAMA_MODELS variable, e.g., to an external drive (documentation). Models take from a few to several dozen GB each; periodically delete unused ones.

Access from other devices at home – by default, Ollama listens only on its own computer. Setting OLLAMA_HOST to 0.0.0.0:11434 makes it available on the home network, e.g., to a laptop or phone via the chat application (documentation). Do not expose it to the internet; there is no password.

Models in Ollama's cloud – Ollama also offers running large models on its servers: a free plan with credits, Pro for $20, and Max for $100 per month, with a declaration of no logging, no training, and zero data storage (Ollama Cloud page). This is convenient, but it is no longer “locally”; if you want to ensure that nothing leaves, set OLLAMA_NO_CLOUD=1 (documentation).

Other Programs for Local Models

• Open WebUI – a chat interface in the browser that connects to Ollama; looks like ChatGPT, has users, history, document handling; good when Ollama is on one computer and the whole family uses it (project).
• LM Studio – an alternative to Ollama with an extended window, model selection from a catalog, and hardware settings; for those who prefer clicking over typing commands (site).
• llama.cpp – the engine on which Ollama and many other programs are based; for advanced users (project).

Security and Privacy

Common Mistakes of Beginners

• Too large a model for too small a computer. A model that does not fit in memory runs dozens of times slower or not at all. Start with 4B, then increase.
• Expecting knowledge of today’s events. The local model does not have the internet. For news, use Perplexity.
• Too short context with a long file. Increase the context length in settings.
• Disk full of models. Each “ollama run new” adds more gigabytes; delete unused ones.
• Mac with 8 GB “for AI”. It will only run the smallest models. When purchasing for AI, choose 32 GB and more.
• Prompt like for ChatGPT. A smaller model needs more context and clearer formatting; the smaller the model, the better the prompt must be.
• Opening a port to the internet “to have access from work”. Instead, use a VPN to home.

Frequently Asked Questions

Is Ollama really free?

Yes, the program and open models are free. Only the optional Ollama cloud is paid. The cost is hardware and electricity.

Does it work without the internet?

Yes, after downloading the model. The internet is only needed to download the program and models.

Do local models speak Polish?

Larger models (Gemma 3 12B and 27B, Qwen3 from 8B, gpt-oss, Llama 3.3) do, although with worse Polish than the cloud. Bielik is designed for Polish and writes in Polish best at a similar size.

I have a PC laptop with 16 GB RAM and no graphics card. Is it worth it?

For trying, yes: 4B to 8B models will run on the CPU, slowly, but they will run. For everyday work, a graphics card or a Mac with unified memory is needed.

Mac or PC for AI at home?

If you want to run large models (30B and more) on one quiet computer, a Mac with 64 GB or more. If mainly small and medium models and you care about speed and gaming, a PC with an NVIDIA card 16–32 GB. If budget is important and you want a lot of memory, a computer with AMD Ryzen AI Max+.

Can I use Ollama with ChatGPT or Claude?

These are separate programs. You can have both: the cloud for difficult tasks and current events, Ollama for confidential documents. Developers connect Ollama with coding tools, including Claude Code, but that is an advanced topic.

Is Bielik the same as ChatGPT in Polish?

No. It is a smaller, open model trained with Polish in mind that runs on your device. It knows Polish very well but has less general knowledge than the largest cloud models and does not have internet access.

Fact-Check Summary

Sources