Ollama is an open-source project that serves as a powerful and user-friendly platform for running LLMs on your local machine. Get up and running with large language models. In this step by step guide I will show you how to install Ollama on your Synology NAS using Docker & Portainer.
STEP 1
Please Support My work by Making a Donation.
STEP 2
Install Portainer using my step by step guide. If you already have Portainer installed on your Synology NAS, skip this STEP. Attention: Make sure you have installed the latest Portainer version.
STEP 3
Make sure you have a synology.me Wildcard Certificate. Follow my guide to get a Wildcard Certificate. If you already have a synology.me Wildcard certificate, skip this STEP.
STEP 4
Go to Control Panel / Login Portal / Advanced Tab / click Reverse Proxy. Follow the instructions in the image below.
STEP 5
Now click the “Create” button. Follow the instructions in the image below.
STEP 6
After you click the Create button, the window below will open. Follow the instructions in the image below.
On the General area, set the Reverse Proxy Name description: type in Ollama. After that, add the following instructions:
Source:
Protocol: HTTPS
Hostname: ollama.yourname.synology.me
Port: 443
Check Enable HSTS
Destination:
Protocol: HTTP
Hostname: localhost
Port: 8271
STEP 7
On the Reverse Proxy Rules click the Custom Header tab. Click Create and then, from the drop-down menu, click WebSocket. After you click on WebSocket, two Header Names and two Values will be automatically added. Click Save. Follow the instructions in the image below.
STEP 8
Go to Control Panel / Network / Connectivity tab/ Check Enable HTTP/2 then click Apply. Follow the instructions in the image below.
STEP 9
Go to Control Panel / Security / Advanced tab/ Check Enable HTTP Compression then click Apply. Follow the instructions in the image below.
STEP 10
Go to File Station and open the docker folder. Inside the docker folder, create one new folder and name it ollama. Follow the instructions in the image below.
Note: Be careful to enter only lowercase, not uppercase letters.
STEP 11
Now create three new folders inside the ollama folder that you have previously created at STEP 10 and name them data, entrypoint, webui. Follow the instructions in the image below.
Note: Be careful to enter only lowercase, not uppercase letters.
STEP 12
Download (click on the blue link below) and upload the entrypoint.sh file below in the entrypoint folder that you have previously created at STEP 11. Follow the instructions in the image below. 🔒Note: Support my work to unlock the password. You can use this password to download any file on mariushosting forever!
STEP 13
Log into Portainer using your username and password. On the left sidebar in Portainer, click on Stacks then + Add stack. Follow the instructions in the image below.
STEP 14
In the Name field type in ollama. Follow the instructions in the image below.
services:
webui:
container_name: OLLAMA-WEBUI
image: ghcr.io/open-webui/open-webui:0.3
volumes:
- /volume1/docker/ollama/webui:/app/backend/data:rw
environment:
OLLAMA_BASE_URL: http://ollama:11434
healthcheck:
test: timeout 10s bash -c ':> /dev/tcp/127.0.0.1/8080' || exit 1
interval: 10s
timeout: 5s
retries: 3
start_period: 90s
ports:
- 8271:8080
restart: on-failure
depends_on:
ollama:
condition: service_healthy
ollama:
container_name: OLLAMA
image: ollama/ollama:latest #For a NAS with an AMD CPU, use the following image ollama/ollama:rocm instead of ollama/ollama:latest
entrypoint: ["/usr/bin/bash", "/entrypoint.sh"]
volumes:
- /volume1/docker/ollama/data:/root/.ollama:rw
- /volume1/docker/ollama/entrypoint/entrypoint.sh:/entrypoint.sh
environment:
MODELS: llama3.2 #Check all the models at the following link https://ollama.com/library - You can separate models by commas like llama3.2,gemma2,mistral
OLLAMA_INSTALL_MODELS: llama3.2 #Check all the models at the following link https://ollama.com/library - You can separate models by commas like llama3.2,gemma2,mistral
OLLAMA_HOSTNAME: ollama.yourname.synology.me
ports:
- 11434:11434
healthcheck:
test: ["CMD", "ollama", "--version"]
interval: 10s
timeout: 5s
retries: 3
start_period: 30s
restart: on-failure:5
Note: Before you paste the code above in the Web editor area below, change the value for OLLAMA_HOSTNAME and type in your own your own synology.me DDNS without https:// at the beginning that you have previously created at STEP 6.
STEP 15
Scroll down on the page until you see a button named Deploy the stack. Click on it. Follow the instructions in the image below. The installation process can take up to a few minutes. It will depend on your Internet speed connection.
STEP 16
If everything goes right, you will see the following message at the top right of your screen: “Success Stack successfully deployed“.
STEP 17
Go back to STEP 1 or you will deal with karma 🙂
STEP 18
Please wait approximately 5-10 minutes for the installation to be completed or you will get a blank page if you try to connect too soon. Now open your browser and type in your HTTPS/SSL certificate like this https://ollama.yourname.synology.me that you have previously created at STEP 6. In my case it’s https://ollama.mariushosting.synology.me If everything goes right, you will see the Ollama Login page. Click Sign up. Follow the instructions in the image below.
STEP 19
Type in your own details (Name, Email Address, Password), then click Create Account. Follow the instructions in the image below.
STEP 20
At the top of the page, select your AI model. Click on the little down arrow. Select your AI model. Follow the instructions in the image below.
STEP 21
In the input box, type in your question, then click enter by clicking the little up arrow icon. Follow the instructions in the image below.
STEP 22
Your answer at a glance! At the top right of the page, click on the user icon then Settings. Follow the instructions in the image below.
STEP 23
Switch your theme to Dark! On the left sidebar, click Admin Settings. Follow the instructions in the image below.
STEP 24
On the Manage Ollama Models, click on the little update icon to update your current AI Models. Follow the instructions in the image below.
STEP 25
You can also download new AI models. In the field, type in the AI model name that you want to download, then click the little download icon to download it. Follow the instructions in the image below.
STEP 26
Your new AI model will be automatically downloaded. Follow the instructions in the image below.
STEP 27
If you have an OpenAI account, you can also type in your API Key in the apposite area, then Save it. Follow my guide from STEP 3 to STEP 5 to discover how to get your OpenAI API Key.
Enjoy Ollama!
If you encounter issues by using this container, make sure to check out the Common Docker issues article.
Note: Can I run Docker on my Synology NAS? See the supported models.
Note: How to Back Up Docker Containers on your Synology NAS.
Note: Find out how to update the Ollama container with the latest image.
Note: How to Free Disk Space on Your NAS if You Run Docker.
Note: How to Schedule Start & Stop For Docker Containers.
Note: How to Activate Email Notifications.
Note: How to Add Access Control Profile on Your NAS.
Note: How to Change Docker Containers Restart Policy.
Note: How to Use Docker Containers With VPN.
Note: Convert Docker Run Into Docker Compose.
Note: How to Clean Docker.
Note: How to Clean Docker Automatically.
Note: Best Practices When Using Docker and DDNS.
Note: Some Docker Containers Need WebSocket.
Note: Find out the Best NAS Models For Docker.
Note: Activate Gmail SMTP For Docker Containers.
This post was updated on Monday / September 30th, 2024 at 8:36 AM