Route all chat through a local LiteLLM gateway, drop command prefixes
- New litellm service (pinned v1.98.0 — litellm 1.82.7/1.82.8 on PyPI were compromised with credential-stealing malware in March 2026; internal-only, no Traefik route, no reason to expose an LLM gateway with a master key publicly). - Replaces !claude/!ai command prefixes with automatic routing: every plain message in the control room goes through a classifier (router.js) that decides chat vs code_task. Chat replies use OpenRouter's own auto-router (openrouter/auto) via LiteLLM; code_task requests go through the existing runChatTask() flow (Claude Code CLI, unchanged, still using the subscription token directly). - Investigated routing Claude itself through LiteLLM via OAuth token forwarding (general_settings.forward_client_headers_to_llm_api) so the Pro/Max subscription could be one of the auto-routable options. Confirmed non-functional: Anthropic returns a generic rate_limit_error for any direct API call using this token type outside the real Claude Code CLI, reproduced with plain curl straight to api.anthropic.com. Not included. - MATRIX_BOT_USER_ID now required and set explicitly (self-message filtering can no longer rely on a command-prefix mismatch once there isn't one).
This commit is contained in:
@@ -0,0 +1,45 @@
|
||||
import { chatCompletion } from "./litellm.js";
|
||||
|
||||
const ROUTER_MODEL = "router-classifier";
|
||||
const CHAT_MODEL = "auto";
|
||||
|
||||
function systemPrompt(knownRepos) {
|
||||
return [
|
||||
"You are a routing classifier for a chat bot. Given a user message, decide whether it is:",
|
||||
'- "chat": a question, discussion, or anything that just needs a text reply.',
|
||||
'- "code_task": a request to change a specific code repository (add/edit/fix something)',
|
||||
" where the repository is clearly one of the known repositories below.",
|
||||
"",
|
||||
`Known repositories: ${knownRepos.join(", ") || "(none configured)"}`,
|
||||
"",
|
||||
"Reply with ONLY a JSON object, nothing else:",
|
||||
'{"type":"chat"}',
|
||||
'or',
|
||||
'{"type":"code_task","repo":"owner/repo","instruction":"clear imperative instruction"}',
|
||||
"",
|
||||
"If it sounds like a code change but you can't confidently match it to one of the known",
|
||||
'repositories, reply {"type":"chat"} instead of guessing.',
|
||||
].join("\n");
|
||||
}
|
||||
|
||||
function parseDecision(raw) {
|
||||
try {
|
||||
const cleaned = raw.trim().replace(/^```(?:json)?\n?/, "").replace(/```$/, "");
|
||||
const parsed = JSON.parse(cleaned);
|
||||
if (parsed.type === "code_task" && parsed.repo && parsed.instruction) {
|
||||
return parsed;
|
||||
}
|
||||
} catch {
|
||||
// fall through to chat — an unparseable classification is not a reason to edit a repo
|
||||
}
|
||||
return { type: "chat" };
|
||||
}
|
||||
|
||||
export async function routeMessage(text, knownRepos) {
|
||||
const raw = await chatCompletion(ROUTER_MODEL, `${systemPrompt(knownRepos)}\n\nMessage: ${text}`);
|
||||
return parseDecision(raw);
|
||||
}
|
||||
|
||||
export async function chatReply(text) {
|
||||
return chatCompletion(CHAT_MODEL, text);
|
||||
}
|
||||
Reference in New Issue
Block a user