Per-key budgets and circuit breakers (CBUDGET): how it works and when to use it
Per-key budgets and circuit breakers (CBUDGET), meters spend per tenant and can hard-block upstream calls past a circuit-breaker threshold, so a runaway loop hits a wall before the invoice. Here's how Crowkis does it and why it matters for cost and safety.
Production LLM traffic is deeply repetitive, and repetition is exactly what a bill is made of. Per-key budgets and circuit breakers (CBUDGET) is how Crowkis meters spend per tenant and can hard-block upstream calls past a circuit-breaker threshold, so a runaway loop hits a wall before the invoice.
How it works
Crowkis meters spend per tenant and can hard-block upstream calls past a circuit-breaker threshold, so a runaway loop hits a wall before the invoice. It runs inside one Redis-compatible engine, so it composes with semantic caching, agent memory, and the other intelligence layers instead of being a separate service you wire together.
CBUDGET GET TENANT acme
Why it matters
Repetitive LLM workloads are where the money is, and semantic caching can cut costs up to 60-70% on repetitive workloads. Per-key budgets and circuit breakers (CBUDGET) is part of what makes that reuse safe rather than reckless, the difference between a cache you trust in production and one you audit after every incident. Community edition ships at full power, free to run.
Infrastructure earns the critical path one boring, verifiable feature at a time.