Route all chat through a local LiteLLM gateway, drop command prefixes

- New litellm service (pinned v1.98.0 — litellm 1.82.7/1.82.8 on PyPI were
  compromised with credential-stealing malware in March 2026; internal-only,
  no Traefik route, no reason to expose an LLM gateway with a master key
  publicly).
- Replaces !claude/!ai command prefixes with automatic routing: every plain
  message in the control room goes through a classifier (router.js) that
  decides chat vs code_task. Chat replies use OpenRouter's own auto-router
  (openrouter/auto) via LiteLLM; code_task requests go through the existing
  runChatTask() flow (Claude Code CLI, unchanged, still using the
  subscription token directly).
- Investigated routing Claude itself through LiteLLM via OAuth token
  forwarding (general_settings.forward_client_headers_to_llm_api) so the
  Pro/Max subscription could be one of the auto-routable options. Confirmed
  non-functional: Anthropic returns a generic rate_limit_error for any
  direct API call using this token type outside the real Claude Code CLI,
  reproduced with plain curl straight to api.anthropic.com. Not included.
- MATRIX_BOT_USER_ID now required and set explicitly (self-message filtering
  can no longer rely on a command-prefix mismatch once there isn't one).
This commit is contained in:
2026-08-23 15:32:55 +00:00
parent 4fc4433833
commit f93bfb25a2
7 changed files with 180 additions and 96 deletions
+45
View File
@@ -0,0 +1,45 @@
import { chatCompletion } from "./litellm.js";
const ROUTER_MODEL = "router-classifier";
const CHAT_MODEL = "auto";
function systemPrompt(knownRepos) {
return [
"You are a routing classifier for a chat bot. Given a user message, decide whether it is:",
'- "chat": a question, discussion, or anything that just needs a text reply.',
'- "code_task": a request to change a specific code repository (add/edit/fix something)',
" where the repository is clearly one of the known repositories below.",
"",
`Known repositories: ${knownRepos.join(", ") || "(none configured)"}`,
"",
"Reply with ONLY a JSON object, nothing else:",
'{"type":"chat"}',
'or',
'{"type":"code_task","repo":"owner/repo","instruction":"clear imperative instruction"}',
"",
"If it sounds like a code change but you can't confidently match it to one of the known",
'repositories, reply {"type":"chat"} instead of guessing.',
].join("\n");
}
function parseDecision(raw) {
try {
const cleaned = raw.trim().replace(/^```(?:json)?\n?/, "").replace(/```$/, "");
const parsed = JSON.parse(cleaned);
if (parsed.type === "code_task" && parsed.repo && parsed.instruction) {
return parsed;
}
} catch {
// fall through to chat — an unparseable classification is not a reason to edit a repo
}
return { type: "chat" };
}
export async function routeMessage(text, knownRepos) {
const raw = await chatCompletion(ROUTER_MODEL, `${systemPrompt(knownRepos)}\n\nMessage: ${text}`);
return parseDecision(raw);
}
export async function chatReply(text) {
return chatCompletion(CHAT_MODEL, text);
}