How to Self-Host Ollama and Open WebUI on Cloud Linux VPS (Private AI Assistant)

The Case for Self-Hosted and Private Artificial Intelligence

While commercial AI platforms like OpenAI ChatGPT and Anthropic Claude provide capable intelligence, sending proprietary enterprise data, confidential source code, customer records, and internal emails to external third-party cloud servers poses severe data privacy and compliance risks. Furthermore, monthly subscription fees and token-based API billing escalate rapidly as usage expands across development teams.

Ollama is the leading open-source framework for running state-of-the-art Large Language Models (LLMs)—including Meta Llama 3, Mistral, DeepSeek Coder, and Google Gemma—locally on CPU and GPU infrastructure. Paired with Open WebUI (a feature-rich ChatGPT-style web interface with RAG document uploads, web browsing, and multi-model arena chats), you can deploy a 100% private, self-hosted AI assistant on your cloud VPS with zero token fees.

In this technical tutorial, we will configure Ollama on Ubuntu 24.04/22.04 LTS, download optimized quantized models, deploy Open WebUI via Docker Compose, and configure an Nginx reverse proxy with SSL encryption.

Step 1: Installing Ollama on Ubuntu Linux VPS

Install Ollama using the official automated installation binary:

# Download and install Ollama core engine
curl -fsSL https://ollama.com/install.sh | sh

# Verify Ollama service is active
sudo systemctl status ollama --no-pager

Step 2: Configuring Ollama for Internal Network Listening

By default, Ollama only listens on 127.0.0.1:11434. To allow Docker containers (like Open WebUI) to communicate with Ollama, configure systemd environment overrides in /etc/systemd/system/ollama.service.d/override.conf:

# Create systemd override directory
sudo mkdir -p /etc/systemd/system/ollama.service.d

# Add host binding configuration
sudo tee /etc/systemd/system/ollama.service.d/override.conf << 'EOF'
[Service]
Environment="OLLAMA_HOST=0.0.0.0:11434"
Environment="OLLAMA_ORIGINS=*"
Environment="OLLAMA_NUM_PARALLEL=4"
EOF

# Reload systemd and restart Ollama
sudo systemctl daemon-reload
sudo systemctl restart ollama

Step 3: Downloading Optimized Open-Source LLMs

Pull high-performance 4-bit quantized (Q4_K_M) models tailored for your server’s available RAM:

# For 4GB to 8GB RAM VPS: Llama 3 8B or Mistral 7B
ollama pull llama3:8b
ollama pull mistral:7b

# For Code Autocompletion & Programming: DeepSeek Coder 6.7B
ollama pull deepseek-coder:6.7b

# Test model execution inside terminal
ollama run llama3:8b "Explain how Nginx reverse proxy works in 3 sentences."

Step 4: Deploying Open WebUI with Docker Compose

Create a dedicated directory /var/www/ai-webui and create docker-compose.yml:

services:
  open-webui:
    image: ghcr.io/open-webui/open-webui:main
    container_name: open-webui
    restart: always
    ports:
      - "127.0.0.1:8080:8080"
    extra_hosts:
      - "host.docker.internal:host-gateway"
    environment:
      - OLLAMA_BASE_URL=http://host.docker.internal:11434
      - WEBUI_SECRET_KEY=SuperSecureRandomAiKey2026!
      - ENABLE_RAG_WEB_SEARCH=true
      - RAG_WEB_SEARCH_ENGINE=duckduckgo
    volumes:
      - open_webui_data:/app/backend/data
    deploy:
      resources:
        limits:
          memory: 1024M

volumes:
  open_webui_data:

Launch the container fleet:

docker compose up -d
docker compose ps

Step 5: Nginx Reverse Proxy with Streaming WebSockets & SSL

Open WebUI utilizes server-sent events (SSE) and WebSockets for real-time token streaming. Create /etc/nginx/sites-available/ai.example.com:

server {
    listen 80;
    server_name ai.example.com;

    client_max_body_size 100M;

    location / {
        proxy_pass http://127.0.0.1:8080;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_buffering off;
        proxy_read_timeout 600s;
    }
}

Enable the site and issue an SSL certificate:

sudo ln -s /etc/nginx/sites-available/ai.example.com /etc/nginx/sites-enabled/
sudo nginx -t && sudo systemctl reload nginx
sudo certbot --nginx -d ai.example.com

Popular Open-Source LLMs for Cloud VPS Infrastructure

Integrating Ollama APIs with Python, LangChain, and n8n

Ollama provides a native OpenAI-compatible REST API endpoint, allowing you to drop local models directly into existing AI applications, LangChain pipelines, or n8n workflow automations by changing the base URL to http://127.0.0.1:11434/v1:

# Python Integration with OpenAI SDK & Local Ollama
from openai import OpenAI

client = OpenAI(
    base_url="http://127.0.0.1:11434/v1",
    api_key="ollama" # Required by SDK but unused locally
)

response = client.chat.completions.create(
    model="llama3:8b",
    messages=[
        {"role": "system", "content": "You are an expert DevOps engineer."},
        {"role": "user", "content": "How do I optimize Nginx gzip compression?"}
    ]
)

print(response.choices[0].message.content)

Fine-Tuning System Prompts with Custom Modelfiles

Create specialized custom AI agents tailored for your business using Ollama Modelfiles:

# Create custom Modelfile (SysAdminAssistant.Modelfile)
FROM llama3:8b
PARAMETER temperature 0.2
PARAMETER top_p 0.9
SYSTEM """You are a senior Linux system administrator. Provide concise, secure, copy-pasteable terminal commands for Ubuntu 24.04 LTS servers."""

Build and run the custom model: ollama create sysadmin-bot -f SysAdminAssistant.Modelfile.

Deploying Retrieval-Augmented Generation (RAG) with Open WebUI

Open WebUI includes an integrated document parser and vector database. You can upload internal PDF training manuals, company policy documents, or proprietary codebases directly into the web chat interface. When you ask questions with the #doc hashtag, Open WebUI performs local chunking, calculates vector embeddings using local embedding models (such as nomic-embed-text), and injects the most relevant context into the LLM prompt without sending data to external cloud APIs.

# Pull high-speed local embedding model
ollama pull nomic-embed-text

# Verify local embedding inference
curl http://localhost:11434/api/embeddings -d '{
  "model": "nomic-embed-text",
  "prompt": "Cloud hosting performance benchmarks"
}'

Securing Public Ollama Portals against Unauthorized Scanners

Ensure that port 11434 is never exposed publicly to the internet. Keep Ollama bound to 127.0.0.1 or isolated within the internal Docker bridge network (172.17.0.0/16), routing all client connections through Nginx with SSL and strong user password authentication.