---
title: Ollama
url: 'https://www.niklas-stephan.de/en/wiki/free-open-source-software/ollama'
markdown: 'https://www.niklas-stephan.de/en/wiki/free-open-source-software/ollama.md'
lang: en
date: '2026-03-19'
description: 'Ollama Language models Since my server has no GPU, I am limited in the choice of language models. Name Type Context File size Performance Quality Published Publisher deepseek-r1:8b-llama-distill-q4_K_M ? ? ? ? Deepseek (MIT) deepseek-coder-v2:16b-lite-base-q2_K ? ? ? ⭐⭐ ⭐ ? Deepseek (…'
---

#  Ollama 

## Language models

Since my server has no GPU, I am limited in the choice of language models.

| Name | Type | Context | File size | Performance | Quality | Published | Publisher |
|---|---|---|---|---|---|---|---|
| deepseek-r1:8b-llama-distill-q4\_K\_M | ? | ? | ? |  |  | ? | Deepseek (MIT) |
| deepseek-coder-v2:16b-lite-base-q2\_K | ? | ? | ? | ⭐⭐ | ⭐ | ? | Deepseek (MIT) |
| deepseek-r1:8b-llama-distill-q4\_K\_M |  |  |  | ⭐ (6.2 Tokens/s) | ⭐⭐ |  | Deepseek (MIT) |
| gemma3:1b |  |  |  | ⭐⭐⭐⭐ (30.7 Tokens/s) | ⭐⭐ |  |  |
| qwen3.5:9b | Text, Image | 256K | 6.6GB | - | - | 2026-03 | Alibaba |
| qwen3.5:4b | Text, Image | 256K | 3.4GB | - | - | 2026-03 | Alibaba |
| qwen3.5:2b | Text, Image | 256K | 2.7GB | ⭐⭐ (9.6 Tokens/s) | ⭐⭐⭐ | 2026-03 | Alibaba |
| qwen2.5-coder:3b | Text | 32k | 2.7GB | ⭐⭐⭐⭐ (38.2 Tokens/s) | ⭐⭐ | 2025-07 | Alibaba |

## Installation and configuration

The guys from the "Open WebUI" project made it extremely easy to get your chatbot running. Basically you just create a new docker-compose.yml file like the one in the example below and start the thing as usual by the command "docker compose up -d". That's it, no joke!

```yaml
services:
  chat:
    container_name: chat
    image: ghcr.io/open-webui/open-webui:ollama
    volumes:
      - ./ollama:/root/.ollama
      - ./open-webui:/app/backend/data
    restart: unless-stopped
    #ports:
    #  - 8080:8080
    networks:
      caddy:
networks:
  caddy:
    external: true
```

As you can see in my example file, I customized the network configuration and also configured my reverse proxy caddy to point access to [chat.handtrixxx.com](http://chat.handtrixxx.com) at my new container. As the following screenshot shows, you can click on "Sign up" to create a new user account for yourself as administrator.

![](https://www.niklas-stephan.de/user/pages/wiki/free-open-source-software/ollama/02-grafik-1.png)

Now, after you logged in, there are just two steps more to do to start your AI chats. At first you should go to the admin panel and then to "Admin settings" to disable registration for other users, so that other users cannot just create an account on your instance. Then in the settings, at the models tab, you have to download one or more language models. There are plenty to choose from. An overview is available at <https://ollama.com/library>. You are done – and as you can see it does not have to take more than 5 minutes if you are a bit experienced with docker and setting up tools in general.

## Costs

Since everything I introduced and described is based on open-source software, there are no costs or licensing fees at all. Great, isn't it? But saying it is completely free is also not completely true, since you have to cover the charges for the server if you do not "have" one anyway 🙂.

## Performance

As mentioned before, a dedicated graphics card would speed up the response times of the chatbot tremendously. By running it only on CPU, like I did in my example, every generation of a response took all the CPU power I have (and I have a lot) for some seconds. So the whole thing feels a bit like the early versions of ChatGPT. That's no drama, but definitely noticeable.

## Conclusion

As a conclusion I let the OpenChat language model answer my prompt itself:

![](https://www.niklas-stephan.de/user/pages/wiki/free-open-source-software/ollama/03-grafik-2.png)

---

## Navigation

- Parent: [Free Open-Source Software](https://www.niklas-stephan.de/en/wiki/free-open-source-software.md)
- Previous: [Matomo](https://www.niklas-stephan.de/en/wiki/free-open-source-software/matomo.md)
- Next: [Open Cloud](https://www.niklas-stephan.de/en/wiki/free-open-source-software/open-cloud.md)
