5. Circuit breaker + OpenRouter fallback #6

New Issue

doru · 2026-06-15T02:48:58+03:00

doru commented

2026-06-15 02:48:58 +03:00

Parent

#1 Epic: Model Switching via Sidecar

What to build

When the Sidecar is unreachable or llama-server fails to recover, the Router uses a circuit breaker pattern and falls back to OpenRouter (DeepSeek V4 Flash). Removes the old x-intelligence-level: High header routing — OpenRouter is now purely a fallback, not a direct routing rule.

Router — circuit breaker tracks Sidecar failures in memory. On Sidecar failure, attempts to recover llama-server via Sidecar up to 3 times. After 3 failed attempts, circuit opens → routes to OpenRouter. Circuit resets on successful Sidecar interaction.
Router — removes x-intelligence-level: High header logic entirely. Adds OpenRouter as fallback backend with OPENROUTER_API_KEY from .env.

Acceptance criteria

Circuit breaker tracks Sidecar failures (in-memory counter)
Up to 3 auto-recovery attempts via Sidecar before opening circuit
Open circuit → routes request to OpenRouter (DeepSeek V4 Flash)
Successful Sidecar interaction resets circuit breaker
Old x-intelligence-level: High header routing removed
OPENROUTER_API_KEY read from environment
Tests: Sidecar down triggers circuit breaker, fallback to OpenRouter, circuit reset on recovery

Blocked by

#4 (Automatic model detection and switch)

User stories covered

7

## Parent - #1 Epic: Model Switching via Sidecar ## What to build When the Sidecar is unreachable or llama-server fails to recover, the Router uses a circuit breaker pattern and falls back to OpenRouter (DeepSeek V4 Flash). Removes the old `x-intelligence-level: High` header routing — OpenRouter is now purely a fallback, not a direct routing rule. - **Router** — circuit breaker tracks Sidecar failures in memory. On Sidecar failure, attempts to recover llama-server via Sidecar up to 3 times. After 3 failed attempts, circuit opens → routes to OpenRouter. Circuit resets on successful Sidecar interaction. - **Router** — removes `x-intelligence-level: High` header logic entirely. Adds OpenRouter as fallback backend with `OPENROUTER_API_KEY` from `.env`. ## Acceptance criteria - [ ] Circuit breaker tracks Sidecar failures (in-memory counter) - [ ] Up to 3 auto-recovery attempts via Sidecar before opening circuit - [ ] Open circuit → routes request to OpenRouter (DeepSeek V4 Flash) - [ ] Successful Sidecar interaction resets circuit breaker - [ ] Old `x-intelligence-level: High` header routing removed - [ ] `OPENROUTER_API_KEY` read from environment - [ ] Tests: Sidecar down triggers circuit breaker, fallback to OpenRouter, circuit reset on recovery ## Blocked by - #4 (Automatic model detection and switch) ## User stories covered 7

doru added the

type:afk

triage:ready

labels 2026-06-15 02:48:58 +03:00

doru referenced this issue

2026-06-15 02:48:58 +03:00

7. Systemd service deployment #8

doru referenced this issue from a commit

2026-06-15 04:13:40 +03:00

Epic: Model Switching via Sidecar — Issues #4-#7 + #8 deployment

Sign in to join this conversation.