Route all chat through a local LiteLLM gateway, drop command prefixes
- New litellm service (pinned v1.98.0 — litellm 1.82.7/1.82.8 on PyPI were compromised with credential-stealing malware in March 2026; internal-only, no Traefik route, no reason to expose an LLM gateway with a master key publicly). - Replaces !claude/!ai command prefixes with automatic routing: every plain message in the control room goes through a classifier (router.js) that decides chat vs code_task. Chat replies use OpenRouter's own auto-router (openrouter/auto) via LiteLLM; code_task requests go through the existing runChatTask() flow (Claude Code CLI, unchanged, still using the subscription token directly). - Investigated routing Claude itself through LiteLLM via OAuth token forwarding (general_settings.forward_client_headers_to_llm_api) so the Pro/Max subscription could be one of the auto-routable options. Confirmed non-functional: Anthropic returns a generic rate_limit_error for any direct API call using this token type outside the real Claude Code CLI, reproduced with plain curl straight to api.anthropic.com. Not included. - MATRIX_BOT_USER_ID now required and set explicitly (self-message filtering can no longer rely on a command-prefix mismatch once there isn't one).
This commit is contained in:
+34
-2
@@ -36,8 +36,35 @@ services:
|
||||
- "traefik.http.routers.matrix.tls.certresolver=letsencrypt"
|
||||
- "traefik.http.services.matrix.loadbalancer.server.port=8008"
|
||||
|
||||
litellm:
|
||||
# Pinned deliberately, not :latest or :main-latest — litellm==1.82.7/1.82.8 on PyPI
|
||||
# were compromised with credential-stealing malware in March 2026 (fixed within the
|
||||
# hour, but a floating tag could still land on a bad release in the future). v1.98.0
|
||||
# verified clean as of this writing.
|
||||
image: ghcr.io/berriai/litellm:v1.98.0
|
||||
container_name: litellm
|
||||
restart: unless-stopped
|
||||
environment:
|
||||
OPENROUTER_API_KEY: ${OPENROUTER_API_KEY}
|
||||
LITELLM_MASTER_KEY: ${LITELLM_MASTER_KEY}
|
||||
volumes:
|
||||
# Absolute host path, NOT a repo-relative one — Portainer's git-stack deploy clones
|
||||
# into its own directory (/data/compose/N/) whose checkout doesn't reliably persist
|
||||
# for the container's runtime (see the act_runner config comment below for the same
|
||||
# failure mode). An absolute path on the actual host filesystem always resolves the
|
||||
# same way regardless of which tool ran `docker compose up`. Keep this local clone
|
||||
# (/home/william/gitops-automation) pulled to latest when the config changes.
|
||||
- /home/william/gitops-automation/litellm-config.yaml:/app/config.yaml:ro
|
||||
command: ["--config", "/app/config.yaml", "--port", "4000"]
|
||||
networks:
|
||||
- web
|
||||
# Internal only — no Traefik labels. No reason to expose an LLM gateway holding a
|
||||
# master key and OAuth-forwarding config to the public internet.
|
||||
|
||||
claude-agent:
|
||||
image: ${GITEA_REGISTRY_IMAGE}
|
||||
depends_on:
|
||||
- litellm
|
||||
container_name: claude-agent
|
||||
restart: unless-stopped
|
||||
# Explicit vars, not env_file: .env — Portainer's git-based stack deploy clones the
|
||||
@@ -53,8 +80,13 @@ services:
|
||||
MATRIX_HOMESERVER_URL: ${MATRIX_HOMESERVER_URL}
|
||||
MATRIX_BOT_TOKEN: ${MATRIX_BOT_TOKEN}
|
||||
MATRIX_CONTROL_ROOM_ID: ${MATRIX_CONTROL_ROOM_ID}
|
||||
OPENROUTER_API_KEY: ${OPENROUTER_API_KEY}
|
||||
OPENROUTER_DEFAULT_MODEL: ${OPENROUTER_DEFAULT_MODEL:-openai/gpt-4o-mini}
|
||||
MATRIX_BOT_USER_ID: ${MATRIX_BOT_USER_ID}
|
||||
KNOWN_REPOS: ${KNOWN_REPOS}
|
||||
# All model calls now go through the local litellm service, not OpenRouter directly —
|
||||
# one gateway for OpenRouter's models (incl. its auto-router) and, for the
|
||||
# claude-subscription route, Anthropic itself via the forwarded OAuth token above.
|
||||
LITELLM_BASE_URL: http://litellm:4000
|
||||
LITELLM_MASTER_KEY: ${LITELLM_MASTER_KEY}
|
||||
volumes:
|
||||
- agent_workspace:/workspace
|
||||
networks:
|
||||
|
||||
Reference in New Issue
Block a user