Route all chat through a local LiteLLM gateway, drop command prefixes

- New litellm service (pinned v1.98.0 — litellm 1.82.7/1.82.8 on PyPI were
  compromised with credential-stealing malware in March 2026; internal-only,
  no Traefik route, no reason to expose an LLM gateway with a master key
  publicly).
- Replaces !claude/!ai command prefixes with automatic routing: every plain
  message in the control room goes through a classifier (router.js) that
  decides chat vs code_task. Chat replies use OpenRouter's own auto-router
  (openrouter/auto) via LiteLLM; code_task requests go through the existing
  runChatTask() flow (Claude Code CLI, unchanged, still using the
  subscription token directly).
- Investigated routing Claude itself through LiteLLM via OAuth token
  forwarding (general_settings.forward_client_headers_to_llm_api) so the
  Pro/Max subscription could be one of the auto-routable options. Confirmed
  non-functional: Anthropic returns a generic rate_limit_error for any
  direct API call using this token type outside the real Claude Code CLI,
  reproduced with plain curl straight to api.anthropic.com. Not included.
- MATRIX_BOT_USER_ID now required and set explicitly (self-message filtering
  can no longer rely on a command-prefix mismatch once there isn't one).
This commit is contained in:
2026-08-23 15:32:55 +00:00
parent 4fc4433833
commit f93bfb25a2
7 changed files with 180 additions and 96 deletions
+34 -2
View File
@@ -36,8 +36,35 @@ services:
- "traefik.http.routers.matrix.tls.certresolver=letsencrypt"
- "traefik.http.services.matrix.loadbalancer.server.port=8008"
litellm:
# Pinned deliberately, not :latest or :main-latest — litellm==1.82.7/1.82.8 on PyPI
# were compromised with credential-stealing malware in March 2026 (fixed within the
# hour, but a floating tag could still land on a bad release in the future). v1.98.0
# verified clean as of this writing.
image: ghcr.io/berriai/litellm:v1.98.0
container_name: litellm
restart: unless-stopped
environment:
OPENROUTER_API_KEY: ${OPENROUTER_API_KEY}
LITELLM_MASTER_KEY: ${LITELLM_MASTER_KEY}
volumes:
# Absolute host path, NOT a repo-relative one — Portainer's git-stack deploy clones
# into its own directory (/data/compose/N/) whose checkout doesn't reliably persist
# for the container's runtime (see the act_runner config comment below for the same
# failure mode). An absolute path on the actual host filesystem always resolves the
# same way regardless of which tool ran `docker compose up`. Keep this local clone
# (/home/william/gitops-automation) pulled to latest when the config changes.
- /home/william/gitops-automation/litellm-config.yaml:/app/config.yaml:ro
command: ["--config", "/app/config.yaml", "--port", "4000"]
networks:
- web
# Internal only — no Traefik labels. No reason to expose an LLM gateway holding a
# master key and OAuth-forwarding config to the public internet.
claude-agent:
image: ${GITEA_REGISTRY_IMAGE}
depends_on:
- litellm
container_name: claude-agent
restart: unless-stopped
# Explicit vars, not env_file: .env — Portainer's git-based stack deploy clones the
@@ -53,8 +80,13 @@ services:
MATRIX_HOMESERVER_URL: ${MATRIX_HOMESERVER_URL}
MATRIX_BOT_TOKEN: ${MATRIX_BOT_TOKEN}
MATRIX_CONTROL_ROOM_ID: ${MATRIX_CONTROL_ROOM_ID}
OPENROUTER_API_KEY: ${OPENROUTER_API_KEY}
OPENROUTER_DEFAULT_MODEL: ${OPENROUTER_DEFAULT_MODEL:-openai/gpt-4o-mini}
MATRIX_BOT_USER_ID: ${MATRIX_BOT_USER_ID}
KNOWN_REPOS: ${KNOWN_REPOS}
# All model calls now go through the local litellm service, not OpenRouter directly —
# one gateway for OpenRouter's models (incl. its auto-router) and, for the
# claude-subscription route, Anthropic itself via the forwarded OAuth token above.
LITELLM_BASE_URL: http://litellm:4000
LITELLM_MASTER_KEY: ${LITELLM_MASTER_KEY}
volumes:
- agent_workspace:/workspace
networks: