AI Chatbot v2

v1 was a private ChatGPT-style surface: Open WebUI + Azure OpenAI. v2 is what actually runs on Rome today — a full operator stack where the model can shell into the server, call MCP tools, follow $plan/$ask/$agent pipes, and write structured notes/SOPs with dedicated skills.

This write-up mirrors the live deployment under /srv/docker/openwebui, /srv/agent-setup, LiteLLM, and mcpo — not a aspirational diagram.

Open WebUI

Container ghcr.io/open-webui/open-webui:main on 127.0.0.1:3020, published at https://ai.amora.dev. Postgres+pgvector for chat/RAG, Redis for websockets/sessions. Keycloak OIDC only (password login disabled). Subagents enabled for background tool work.

LiteLLM

Models are not hard-wired into Open WebUI. LiteLLM sits on the litellm_litellm Docker network (:4000) as the OpenAI-compatible gateway — Azure/Foundry and other providers stay swappable without rebuilding the chat UI.

Identity

OAuth against auth.amora.dev (Keycloak). Session cookies marked secure; signup closed; auto-redirect to IdP. Office/SOP editing shards reuse the same identity story via oauth2-proxy on office.amora.dev.

Open Terminal is the sandboxed shell path for git, npm, tests, and host ops when MCP filesystem is not enough. On Rome it is wired alongside Open WebUI so the agent can run real commands against volume-mounted project trees (notably /srv) instead of hallucinating shell output.

The agent-setup package documents the split clearly: MCP handles structured file/reasoning tools; Open Terminal handles git and unconstrained shell. Deployment notes even call out skipping a broken git MCP server in favor of Open Terminal for version control.

Pipes tell the model explicitly: use Open Terminal for ls/tree/shell, MCP filesystem for read/write/search, sequential-thinking for hard reasoning chains.

Host systemd unit mcpo.service exposes MCP servers as OpenAPI on 127.0.0.1:8100. Open WebUI reaches them via host.docker.internal:8100. Live servers: filesystem (/srv + /home/toni) and sequential-thinking.

That bridge is what lets chat sessions edit real project files under /srv without giving the model raw Docker socket access.

Custom pipes under /srv/agent-setup/pipes implement Cursor-like modes: $plan (architect, no code), $ask (researcher), $agent (implementer with safety gates).

Every mode runs a session-start protocol: resolve project root, ensure git, read AGENTS.md/CLAUDE.md, detect languages, load /srv/.ai-baselines/{stack}.md, refresh .ai-context.md. Agent mode branches to agent/{timestamp} and gates destructive ops behind explicit confirmation.

Beyond coding, the same Open WebUI hosts an SOP engine (/srv/docker/openwebui/sop-engine) registered as tools + skills. Skills are YAML playbooks the model follows — not one-off prompts.

sop-ingestion

Upload DOCX/XLSX/PDF → reader tools extract structure → sop_sop_writer saves a versioned SOP under sop-db (title, purpose, steps, metadata). That is the “write notes from a document” path.

recording-to-sop

Screen recordings go through Azure STT + keyword hits → drafted steps → sop_sop_writer. Training videos become written procedures without manual transcription.

sop-export + office

Export builders render DOCX/XLSX; file-server + OnlyOffice edit shards (edit1/edit2) let humans finish the note in the browser, sharded by Keycloak group.

Tool registry categories (sop, share, excel, media, conversion, visuals, preview) keep triggers aligned so the model picks the right writer/reader instead of improvising file formats.

• Chat UI, model gateway, tools, and identity are four different failure domains — compose them, do not fuse them.

• Open Terminal + MCP beats stuffing every capability into one plugin; shell and structured tools want different safety rails.

• Skills (YAML workflows) turn “write me notes” into a repeatable pipeline with artifacts on disk, not a one-shot chat blob.

• Newer portfolio/app versions should supersede older cards by default so the public site matches production.

Daily driver at ai.amora.dev: SSO chat, LiteLLM-routed models, host file edits via MCP, real shell via Open Terminal, and SOP/note generation that lands in sop-db with OnlyOffice editing when humans need to polish.