riyaz-companion

llm stack — Ollama + Open WebUI + Music MCP

Local LLM stack: Ollama with gemma4:31b on AMD RX 9070 XT (ROCm), Open WebUI frontend, SearXNG web search, and a music personality MCP server.

Services

Models (in Open WebUI)

Both models are vanilla — no system prompt set on the model itself. Personality and tools come entirely from the music-mcp server.

music-mcp — MCP + OpenAPI personality server

Container built from music-mcp-Dockerfile + music-mcp-server.py. Exposes:

Personality injection (automatic)

OWUI fetches /api/config on startup. If features.system == true, it fetches /system and injects the personality as a system prompt on every chat — without it being set on the model. Swap personality = swap MCP server.

Raga knowledge base

166 Wikipedia articles on Hindustani ragas, thaats, talas, gharanas, and music theory. Mounted from /path/to/home/raga_kb into both open-webui and music-mcp containers. BM25 search, top-3 results.

GPU / KV cache

• KV cache: q8_0 (8-bit, requires flash_attn=auto — enforced by llama.cpp)
• GPU: ROCm, RX 9070 XT, 16GB VRAM — 43–48/61 layers on GPU (vision projector OOMs → CPU)
• Embedding (BGE-M3 in OWUI): DEVICE_TYPE=cpu — runs on CPU, no GPU competition
• Power: ~90–200W during inference depending on load

Post-deploy OWUI configuration (required after fresh install)

After the containers start, two config rows must be set in OWUI’s SQLite DB. These are not in the compose — they are runtime state in /path/to/home/llm/open-webui/webui.db.

sudo docker exec open-webui python3 << 'PYEOF'
import sqlite3, json, time

db = sqlite3.connect("/app/backend/data/webui.db")
now = int(time.time())

# 1. Register music-mcp as OpenAPI tool server (exposes search + raga tools)
tool_server = [{
    "url": "http://music-mcp:8765",
    "type": "openapi",
    "key": "",
    "auth_type": "none",
    "spec_type": "url",
    "path": "openapi.json",
    "info": {
        "id": "music-knowledge",
        "name": "Music Knowledge",
        "description": "Hindustani classical music knowledge base"
    },
    "config": {
        "enable": True,
        "name": "Music Knowledge",
        "description": "Hindustani classical music knowledge base"
    }
}]
db.execute("UPDATE config SET value=?, updated_at=? WHERE key='tool_server.connections'",
           (json.dumps(tool_server), now))

# 2. Register music-mcp as terminal server (OWUI auto-fetches /system for personality)
terminal_server = [{
    "id": "music-mcp",
    "url": "http://music-mcp:8765",
    "key": "",
    "auth_type": "none",
    "config": {
        "enable": True,
        "name": "Music Personality",
        "description": "Hindustani classical music riyaz companion"
    }
}]
db.execute("UPDATE config SET value=?, updated_at=? WHERE key='terminal_server.connections'",
           (json.dumps(terminal_server), now))

db.commit()
db.close()
print("Done — restart open-webui to apply")
PYEOF

sudo docker restart open-webui

Rebuilding music-mcp image

The music-mcp:latest image is local (not in a registry). After a fresh OS install:

sudo docker build -t music-mcp:latest /path/to/home/llm/music-mcp/

The Dockerfile and server code are in this repo (music-mcp-Dockerfile, music-mcp-server.py). Copy them to /path/to/home/llm/music-mcp/ before building.