How to Install LlamaGPT on Your Synology NAS

How to Install LlamaGPT on Your Synology NAS

LlamaGPT is a self-hosted, offline, ChatGPT-like chatbot, powered by Llama 2, similar to Serge. 100% private, with no data leaving your device. To make LlamaGPT work on your Synology NAS you will need a minimum of 8GB of RAM installed. In this step by step guide I will show you how to install LlamaGPT on your Synology NAS using Docker & Portainer.

  • STEP 1

Please Support My work by Making a Donation.

  • STEP 2

Install Portainer using my step by step guide. If you already have Portainer installed on your Synology NAS, skip this STEP. Attention: Make sure you have installed the latest Portainer version.

  • STEP 3

Log into Portainer using your username and password. On the left sidebar in Portainer, click on Stacks then + Add stack. Follow the instructions in the image below.

1 Synology Portainer Add Stack

  • STEP 4

In the Name field type in llamagpt. Follow the instructions in the image below.

Note: Copy Paste the code below in the Portainer Stacks Web editor.

version: "3.9"
    container_name: LlamaGPT-api
    hostname: llamagpt-api
    mem_limit: 8g
    cpu_shares: 768
      - no-new-privileges:true
      MODEL: /models/llama-2-7b-chat.bin
      USE_MLOCK: 1
      - IPC_LOCK
    restart: on-failure:5

    container_name: LlamaGPT
    hostname: llamagpt
    mem_limit: 1g
    cpu_shares: 768
      - no-new-privileges:true
      test: wget --no-verbose --tries=1 --spider http://localhost:3000
      - 3136:3000
     - 'OPENAI_API_HOST=http://llamagpt-api:8000'
     - 'DEFAULT_MODEL=/models/llama-2-7b-chat.bin'
     - 'WAIT_HOSTS=llamagpt-api:8000'
     - 'WAIT_TIMEOUT=600'
    restart: on-failure:5

LlamaGPT Synology NAS Set up 1

  • STEP 5

Scroll down on the page until you see a button named Deploy the stack. Click on it. Follow the instructions in the image below. The installation process can take up to a few minutes. It will depend on your Internet speed connection.

LlamaGPT Synology NAS Set up 2

⌛Now just wait because the LlamaGPT image is about 5.5GB.

  • STEP 6

If everything goes right, you will see the following message at the top right of your screen: “Success Stack successfully deployed“.

LlamaGPT Synology NAS Set up 3

  • STEP 7

Go back to STEP 1 or you will deal with karma 🙂.

  • STEP 8

The installation process can take up to a few seconds/minutes. It will depend on your Internet speed connection. Please wait approximately 3 minutes or you will get a blank error page if you try to connect too soon. Now open your browser and type in http://Synology-ip-address:3136 Select your favorite answer model: Precise, Neutral or Creative. Type in your question in the input area. Click the enter icon to send the message to the AI. Follow the instructions in the image below.

LlamaGPT Synology NAS Set up 4

  • STEP 9

Your AI answer at a glance!

LlamaGPT Synology NAS Set up 5

Enjoy LlamaGPT!

Note: If you want to run the LlamaGPT container over HTTPS, check How to Run Docker Containers Over HTTPS. In order to make LlamaGPT work via HTTPS, it’s mandatory to activate WebSocket. ⚠️Warning: I do not recommend running LlamaGPT via Reverse Proxy. This product should be used only on localhost. Why don’t I recommend running it via Reverse Proxy? Because if someone were to know your HTTPS address, then they can use your LlamaGPT AI to ask questions and overload your NAS RAM and CPU.


If you encounter issues by using this container, make sure to check out the Common Docker issues article.

Note: Can I run Docker on my Synology NAS? See the supported models.
Note: How to Back Up Docker Containers on your Synology NAS.
Note: Find out how to update the LlamaGPT container with the latest image.
Note: How to Free Disk Space on Your NAS if You Run Docker.
Note: How to Schedule Start & Stop For Docker Containers.
Note: How to Activate Email Notifications.
Note: How to Add Access Control Profile on Your NAS.
Note: How to Change Docker Containers Restart Policy.
Note: How to Use Docker Containers With VPN.
Note: Convert Docker Run Into Docker Compose.
Note: How to Clean Docker.
Note: How to Clean Docker Automatically.
Note: Best Practices When Using Docker and DDNS.
Note: Some Docker Containers Need WebSocket.
Note: Find out the Best NAS Models For Docker.
Note: Activate Gmail SMTP For Docker Containers.

This post was updated on Saturday / February 24th, 2024 at 5:26 AM