{"id":1875,"date":"2026-09-05T09:36:02","date_gmt":"2026-09-05T04:06:02","guid":{"rendered":"https:\/\/cpanelfree.com\/blog\/how-to-self-host-ollama-private-llm-cloud-vps\/"},"modified":"2026-09-05T12:58:44","modified_gmt":"2026-09-05T07:28:44","slug":"how-to-self-host-ollama-private-llm-cloud-vps","status":"publish","type":"post","link":"https:\/\/cpanelfree.com\/blog\/how-to-self-host-ollama-private-llm-cloud-vps\/","title":{"rendered":"How to Self-Host Ollama and Open WebUI on Cloud Linux VPS (Private AI Assistant)"},"content":{"rendered":"

The Case for Self-Hosted and Private Artificial Intelligence<\/h2>\n

While commercial AI platforms like OpenAI ChatGPT and Anthropic Claude provide capable intelligence, sending proprietary enterprise data, confidential source code, customer records, and internal emails to external third-party cloud servers poses severe data privacy and compliance risks. Furthermore, monthly subscription fees and token-based API billing escalate rapidly as usage expands across development teams.<\/p>\n

Ollama<\/strong> is the leading open-source framework for running state-of-the-art Large Language Models (LLMs)\u2014including Meta Llama 3, Mistral, DeepSeek Coder, and Google Gemma\u2014locally on CPU and GPU infrastructure. Paired with Open WebUI<\/strong> (a feature-rich ChatGPT-style web interface with RAG document uploads, web browsing, and multi-model arena chats), you can deploy a 100% private, self-hosted AI assistant on your cloud VPS with zero token fees.<\/p>\n

In this technical tutorial, we will configure Ollama on Ubuntu 24.04\/22.04 LTS, download optimized quantized models, deploy Open WebUI via Docker Compose, and configure an Nginx reverse proxy with SSL encryption.<\/p>\n

Step 1: Installing Ollama on Ubuntu Linux VPS<\/h2>\n

Install Ollama using the official automated installation binary:<\/p>\n

# Download and install Ollama core engine\ncurl -fsSL https:\/\/ollama.com\/install.sh | sh\n\n# Verify Ollama service is active\nsudo systemctl status ollama --no-pager<\/code><\/pre>\n
Step 2: Configuring Ollama for Internal Network Listening<\/h2>\n
By default, Ollama only listens on 127.0.0.1:11434<\/code>. To allow Docker containers (like Open WebUI) to communicate with Ollama, configure systemd environment overrides in \/etc\/systemd\/system\/ollama.service.d\/override.conf<\/code>:<\/p>\n
# Create systemd override directory\nsudo mkdir -p \/etc\/systemd\/system\/ollama.service.d\n\n# Add host binding configuration\nsudo tee \/etc\/systemd\/system\/ollama.service.d\/override.conf << 'EOF'\n[Service]\nEnvironment=\"OLLAMA_HOST=0.0.0.0:11434\"\nEnvironment=\"OLLAMA_ORIGINS=*\"\nEnvironment=\"OLLAMA_NUM_PARALLEL=4\"\nEOF\n\n# Reload systemd and restart Ollama\nsudo systemctl daemon-reload\nsudo systemctl restart ollama<\/code><\/pre>\n
Step 3: Downloading Optimized Open-Source LLMs<\/h2>\n
Pull high-performance 4-bit quantized (Q4_K_M) models tailored for your server’s available RAM:<\/p>\n
# For 4GB to 8GB RAM VPS: Llama 3 8B or Mistral 7B\nollama pull llama3:8b\nollama pull mistral:7b\n\n# For Code Autocompletion & Programming: DeepSeek Coder 6.7B\nollama pull deepseek-coder:6.7b\n\n# Test model execution inside terminal\nollama run llama3:8b \"Explain how Nginx reverse proxy works in 3 sentences.\"<\/code><\/pre>\n
Step 4: Deploying Open WebUI with Docker Compose<\/h2>\n
Create a dedicated directory \/var\/www\/ai-webui<\/code> and create docker-compose.yml<\/code>:<\/p>\n
services:\n  open-webui:\n    image: ghcr.io\/open-webui\/open-webui:main\n    container_name: open-webui\n    restart: always\n    ports:\n      - \"127.0.0.1:8080:8080\"\n    extra_hosts:\n      - \"host.docker.internal:host-gateway\"\n    environment:\n      - OLLAMA_BASE_URL=http:\/\/host.docker.internal:11434\n      - WEBUI_SECRET_KEY=SuperSecureRandomAiKey2026!\n      - ENABLE_RAG_WEB_SEARCH=true\n      - RAG_WEB_SEARCH_ENGINE=duckduckgo\n    volumes:\n      - open_webui_data:\/app\/backend\/data\n    deploy:\n      resources:\n        limits:\n          memory: 1024M\n\nvolumes:\n  open_webui_data:<\/code><\/pre>\n
Launch the container fleet:<\/p>\n
docker compose up -d\ndocker compose ps<\/code><\/pre>\n
Step 5: Nginx Reverse Proxy with Streaming WebSockets & SSL<\/h2>\n
Open WebUI utilizes server-sent events (SSE) and WebSockets for real-time token streaming. Create \/etc\/nginx\/sites-available\/ai.example.com<\/code>:<\/p>\n
server {\n    listen 80;\n    server_name ai.example.com;\n\n    client_max_body_size 100M;\n\n    location \/ {\n        proxy_pass http:\/\/127.0.0.1:8080;\n        proxy_http_version 1.1;\n        proxy_set_header Upgrade $http_upgrade;\n        proxy_set_header Connection \"upgrade\";\n        proxy_set_header Host $host;\n        proxy_set_header X-Real-IP $remote_addr;\n        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;\n        proxy_set_header X-Forwarded-Proto $scheme;\n        proxy_buffering off;\n        proxy_read_timeout 600s;\n    }\n}<\/code><\/pre>\n
Enable the site and issue an SSL certificate:<\/p>\n
sudo ln -s \/etc\/nginx\/sites-available\/ai.example.com \/etc\/nginx\/sites-enabled\/\nsudo nginx -t && sudo systemctl reload nginx\nsudo certbot --nginx -d ai.example.com<\/code><\/pre>\n
Popular Open-Source LLMs for Cloud VPS Infrastructure<\/h2>\n\n\n\n\n\n\n\n
Model Name<\/th>\nParameters<\/th>\nRAM Requirement<\/th>\nIdeal Primary Use Case<\/th>\n<\/tr>\n<\/thead>\n
Meta Llama 3<\/strong><\/td>\n8 Billion<\/td>\n~4.8 GB RAM<\/td>\nGeneral assistant, reasoning, content writing<\/td>\n<\/tr>\n
Mistral Instruct<\/strong><\/td>\n7 Billion<\/td>\n~4.2 GB RAM<\/td>\nHigh-speed conversational dialog, summarization<\/td>\n<\/tr>\n
DeepSeek Coder<\/strong><\/td>\n6.7 Billion<\/td>\n~3.9 GB RAM<\/td>\nFull-stack programming, regex, debugging<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n
Integrating Ollama APIs with Python, LangChain, and n8n<\/h2>\n
Ollama provides a native OpenAI-compatible REST API endpoint, allowing you to drop local models directly into existing AI applications, LangChain pipelines, or n8n workflow automations by changing the base URL to http:\/\/127.0.0.1:11434\/v1<\/code>:<\/p>\n
# Python Integration with OpenAI SDK & Local Ollama\nfrom openai import OpenAI\n\nclient = OpenAI(\n    base_url=\"http:\/\/127.0.0.1:11434\/v1\",\n    api_key=\"ollama\" # Required by SDK but unused locally\n)\n\nresponse = client.chat.completions.create(\n    model=\"llama3:8b\",\n    messages=[\n        {\"role\": \"system\", \"content\": \"You are an expert DevOps engineer.\"},\n        {\"role\": \"user\", \"content\": \"How do I optimize Nginx gzip compression?\"}\n    ]\n)\n\nprint(response.choices[0].message.content)<\/code><\/pre>\n
Fine-Tuning System Prompts with Custom Modelfiles<\/h2>\n
Create specialized custom AI agents tailored for your business using Ollama Modelfiles:<\/p>\n
# Create custom Modelfile (SysAdminAssistant.Modelfile)\nFROM llama3:8b\nPARAMETER temperature 0.2\nPARAMETER top_p 0.9\nSYSTEM \"\"\"You are a senior Linux system administrator. Provide concise, secure, copy-pasteable terminal commands for Ubuntu 24.04 LTS servers.\"\"\"<\/code><\/pre>\n
Build and run the custom model: ollama create sysadmin-bot -f SysAdminAssistant.Modelfile<\/code>.<\/p>\n
Deploying Retrieval-Augmented Generation (RAG) with Open WebUI<\/h2>\n
Open WebUI includes an integrated document parser and vector database. You can upload internal PDF training manuals, company policy documents, or proprietary codebases directly into the web chat interface. When you ask questions with the #doc<\/code> hashtag, Open WebUI performs local chunking, calculates vector embeddings using local embedding models (such as nomic-embed-text<\/code>), and injects the most relevant context into the LLM prompt without sending data to external cloud APIs.<\/p>\n
# Pull high-speed local embedding model\nollama pull nomic-embed-text\n\n# Verify local embedding inference\ncurl http:\/\/localhost:11434\/api\/embeddings -d '{\n  \"model\": \"nomic-embed-text\",\n  \"prompt\": \"Cloud hosting performance benchmarks\"\n}'<\/code><\/pre>\n
Securing Public Ollama Portals against Unauthorized Scanners<\/h2>\n
Ensure that port 11434<\/code> is never exposed publicly to the internet. Keep Ollama bound to 127.0.0.1<\/code> or isolated within the internal Docker bridge network (172.17.0.0\/16<\/code>), routing all client connections through Nginx with SSL and strong user password authentication.<\/p>\n
\n
Recommended Related Technical Guides<\/h3>\n