How to Install Ollama on Your Synology NAS

How to Install Ollama on Your Synology NAS

Ollama is an open-source project that serves as a powerful and user-friendly platform for running LLMs on your local machine. Get up and running with large language models. In this step by step guide I will show you how to install Ollama on your Synology NAS using Docker & Portainer.

💡Note: This guide works perfectly with the latest version of Ollama v0.3.12

  • STEP 1

Please Support My work by Making a Donation.

  • STEP 2

Install Portainer using my step by step guide. If you already have Portainer installed on your Synology NAS, skip this STEP. Attention: Make sure you have installed the latest Portainer version.

  • STEP 3

Make sure you have a synology.me Wildcard Certificate. Follow my guide to get a Wildcard Certificate. If you already have a synology.me Wildcard certificate, skip this STEP.

  • STEP 4

Go to Control Panel / Login Portal / Advanced Tab / click Reverse Proxy. Follow the instructions in the image below.

Ollama Synology NAS Set up 1

  • STEP 5

Now click the “Create” button. Follow the instructions in the image below.

Ollama Synology NAS Set up 2

  • STEP 6

After you click the Create button, the window below will open. Follow the instructions in the image below.

On the General area, set the Reverse Proxy Name description: type in Ollama. After that, add the following instructions:

Source:
Protocol: HTTPS
Hostname: ollama.yourname.synology.me
Port: 443

Check Enable HSTS

Destination:
Protocol: HTTP
Hostname: localhost
Port: 8271

Ollama Synology NAS Set up 3

  • STEP 7

On the Reverse Proxy Rules click the Custom Header tab. Click Create and then, from the drop-down menu, click WebSocket. After you click on WebSocket, two Header Names and two Values will be automatically added. Click Save. Follow the instructions in the image below.

Synology Proxy WebSocket

  • STEP 8

Go to Control Panel / Network / Connectivity tab/ Check Enable HTTP/2 then click Apply. Follow the instructions in the image below.

Ollama Synology NAS Set up 4

  • STEP 9

Go to Control Panel / Security / Advanced tab/ Check Enable HTTP Compression then click Apply. Follow the instructions in the image below.

Ollama Synology NAS Set up 5

  • STEP 10

Go to File Station and open the docker folder. Inside the docker folder, create one new folder and name it ollama. Follow the instructions in the image below.
Note: Be careful to enter only lowercase, not uppercase letters.

Ollama Synology NAS Set up 6

  • STEP 11

Now create three new folders inside the ollama folder that you have previously created at STEP 10 and name them data, entrypoint, webui. Follow the instructions in the image below.
Note: Be careful to enter only lowercase, not uppercase letters.

Ollama Synology NAS Set up 7

  • STEP 12

Download (click on the blue link below) and upload the entrypoint.sh file below in the entrypoint folder that you have previously created at STEP 11. Follow the instructions in the image below. 🔒Note: Support my work to unlock the password. You can use this password to download any file on mariushosting forever!

👉🏻entrypoint.sh

Ollama Synology NAS Set up 8

  • STEP 13

Log into Portainer using your username and password. On the left sidebar in Portainer, click on Stacks then + Add stack. Follow the instructions in the image below.

1 Synology Portainer Add Stack

  • STEP 14

In the Name field type in ollama. Follow the instructions in the image below.

Note: Copy Paste the code below in the Portainer Stacks Web editor.

services:
  webui:
    container_name: OLLAMA-WEBUI
    image: ghcr.io/open-webui/open-webui:0.3
    volumes:
      - /volume1/docker/ollama/webui:/app/backend/data:rw
    environment:
      OLLAMA_BASE_URL: http://ollama:11434
    healthcheck:
      test: timeout 10s bash -c ':> /dev/tcp/127.0.0.1/8080' || exit 1
      interval: 10s
      timeout: 5s
      retries: 3
      start_period: 90s
    ports:
      - 8271:8080
    restart: on-failure
    depends_on:
      ollama:
        condition: service_healthy

  ollama:
    container_name: OLLAMA
    image: ollama/ollama:latest #For a NAS with an AMD CPU, use the following image ollama/ollama:rocm instead of ollama/ollama:latest
    entrypoint: ["/usr/bin/bash", "/entrypoint.sh"]
    volumes:
      - /volume1/docker/ollama/data:/root/.ollama:rw
      - /volume1/docker/ollama/entrypoint/entrypoint.sh:/entrypoint.sh
    environment:
      MODELS: llama3.2 #Check all the models at the following link https://ollama.com/library - You can separate models by commas like llama3.2,gemma2,mistral
      OLLAMA_INSTALL_MODELS: llama3.2 #Check all the models at the following link https://ollama.com/library - You can separate models by commas like llama3.2,gemma2,mistral
      OLLAMA_HOSTNAME: ollama.yourname.synology.me
    ports:
      - 11434:11434
    healthcheck:
      test: ["CMD", "ollama", "--version"]
      interval: 10s
      timeout: 5s
      retries: 3
      start_period: 30s
    restart: on-failure:5

Note: Before you paste the code above in the Web editor area below, change the value for OLLAMA_HOSTNAME and type in your own your own synology.me DDNS without https:// at the beginning that you have previously created at STEP 6.

Ollama Synology NAS Set up 9

  • STEP 15

Scroll down on the page until you see a button named Deploy the stack. Click on it. Follow the instructions in the image below. The installation process can take up to a few minutes. It will depend on your Internet speed connection.

Ollama Synology NAS Set up 10

⌛Now just wait because the Ollama image is about 8GB.

  • STEP 16

If everything goes right, you will see the following message at the top right of your screen: “Success Stack successfully deployed“.

Ollama Synology NAS Set up 11

  • STEP 17

Go back to STEP 1 or you will deal with karma 🙂

  • STEP 18

Please wait approximately 5-10 minutes for the installation to be completed or you will get a blank page if you try to connect too soon. Now open your browser and type in your HTTPS/SSL certificate like this https://ollama.yourname.synology.me that you have previously created at STEP 6. In my case it’s https://ollama.mariushosting.synology.me If everything goes right, you will see the Ollama Login page. Click Sign up. Follow the instructions in the image below.

Ollama Synology NAS Set up 12

  • STEP 19

Type in your own details (Name, Email Address, Password), then click Create Account. Follow the instructions in the image below.

Ollama Synology NAS Set up 13

  • STEP 20

At the top of the page, select your AI model. Click on the little down arrow. Select your AI model. Follow the instructions in the image below.

Ollama Synology NAS Set up 14

  • STEP 21

In the input box, type in your question, then click enter by clicking the little up arrow icon. Follow the instructions in the image below.

Ollama Synology NAS Set up 15

  • STEP 22

Your answer at a glance! At the top right of the page, click on the user icon then Settings. Follow the instructions in the image below.

Ollama Synology NAS Set up 16

  • STEP 23

Switch your theme to Dark! On the left sidebar, click Admin Settings. Follow the instructions in the image below.

Ollama Synology NAS Set up 17

  • STEP 24

On the Manage Ollama Models, click on the little update icon to update your current AI Models. Follow the instructions in the image below.

Ollama Synology NAS Set up 18

  • STEP 25

You can also download new AI models. In the field, type in the AI model name that you want to download, then click the little download icon to download it. Follow the instructions in the image below.

Ollama Synology NAS Set up 19

  • STEP 26

Your new AI model will be automatically downloaded. Follow the instructions in the image below.

Ollama Synology NAS Set up 20

  • STEP 27

If you have an OpenAI account, you can also type in your API Key in the apposite area, then Save it. Follow my guide from STEP 3 to STEP 5 to discover how to get your OpenAI API Key.

Ollama Synology NAS Set up 21

Enjoy Ollama!

🆘TROUBLESHOOTING

If you encounter issues by using this container, make sure to check out the Common Docker issues article.

Note: Can I run Docker on my Synology NAS? See the supported models.
Note: How to Back Up Docker Containers on your Synology NAS.
Note: Find out how to update the Ollama container with the latest image.
Note: How to Free Disk Space on Your NAS if You Run Docker.
Note: How to Schedule Start & Stop For Docker Containers.
Note: How to Activate Email Notifications.
Note: How to Add Access Control Profile on Your NAS.
Note: How to Change Docker Containers Restart Policy.
Note: How to Use Docker Containers With VPN.
Note: Convert Docker Run Into Docker Compose.
Note: How to Clean Docker.
Note: How to Clean Docker Automatically.
Note: Best Practices When Using Docker and DDNS.
Note: Some Docker Containers Need WebSocket.
Note: Find out the Best NAS Models For Docker.
Note: Activate Gmail SMTP For Docker Containers.

This post was updated on Monday / September 30th, 2024 at 8:36 AM