Skip to content
⚠️ Before You Start
Risk: Medium
Duration: 1-2 days
Last Validated: Not yet validated (draft)
Kind: How-to
Risk Note: Medium (too tight blocks real work; fail-open lets spend run away)

Enforce per-run, per-user and per-tenant budgets with Redis counters

Draft outline: not validated

Drafted with AI from official sources on 16 Oct 2026. Do not use on a live system until it is written, dry-run by someone other than the author and signed off (planned 2026-12-09). Author: Sandeep Velu · dry-run: Yasar · sign-off: Sivaraj. Work items: AGT-11-T01, AGT-11-T02, AGT-11-T03, AGT-11-T04.

1. Summary

When building any agent that serves more than one user or tenant, or after a cost incident shows ceilings were not enforced.

Risk: Medium (too tight blocks real work; fail-open lets spend run away). Typical duration: 1-2 days.

2. When to use and when not to

Use it when: When building any agent that serves more than one user or tenant, or after a cost incident shows ceilings were not enforced.

Do not use it when: to be written by the author.

3. What you must know first

  • INCRBY is a single O(1) command and creates the key at 0 if missing
  • A key created by INCRBY has no expiry until you set one; EXPIRE with NX needs Redis 7.0 or later
  • Check the budget before a model call with an estimate and charge actual usage after
  • Decide fail-closed or fail-open per tier for Redis outages, and record it
  • Step limits and token limits catch different failures; enforce both

4. Prerequisites and access

  • Redis instance reachable from the agent with TLS and Entra or managed credentials
  • Budget values agreed in the budget policy (PB-AGT-11-04)
  • Token usage per model call available from the response

5. Steps

Outline of the steps; the author adds the commands and a check after each one.

  1. Define key names for run, user and tenant scopes and their time windows.
  2. Before each model call, read the counters and refuse if the estimate would exceed a limit.
  3. After each call, charge actual tokens to all three counters in one pipeline.
  4. Set expiry on first write so windows reset.
  5. Return a clear message to the user when a ceiling is hit.
  6. Emit a metric and trace attribute when a budget blocks a run.
  7. Unit-test limits, window resets and the Redis-down path.
  8. Verify in staging that a scripted loop stops at the run ceiling.
  9. Rollback: switch the middleware to observe-only mode by configuration.

Draft snippet: Charge tokens to run, user and tenant counters

import os

import redis

r = redis.Redis.from_url(os.environ["REDIS_URL"])  # TLS URL; credentials from managed identity or Key Vault

LIMITS = {"run": 20_000, "user": 200_000, "tenant": 5_000_000}  # tokens, from the budget policy
TTL = {"run": 3600, "user": 86400, "tenant": 86400 * 31}


class BudgetExceeded(Exception):
    pass


def charge(run_id: str, user_id: str, tenant_id: str, tokens: int) -> None:
    keys = {
        "run": f"budget:run:{run_id}",
        "user": f"budget:user:{user_id}",
        "tenant": f"budget:tenant:{tenant_id}",
    }
    pipe = r.pipeline(transaction=True)
    for scope, key in keys.items():
        pipe.incrby(key, tokens)
        pipe.expire(key, TTL[scope], nx=True)  # needs Redis 7.0+
    results = pipe.execute()
    totals = dict(zip(keys, results[0::2]))
    for scope, total in totals.items():
        if total > LIMITS[scope]:
            raise BudgetExceeded(f"{scope} budget exceeded: {total} > {LIMITS[scope]}")

Snippets to add:

  • snippets/agent/budget.py: Budget middleware with three scopes (python)
  • snippets/tests/test_budget.py: Tests for limits, resets and Redis outage (python)

6. Verify it works

In the sandbox, a second engineer runs a scripted looping agent and a heavy user against the middleware. Pass: the loop stops at the run ceiling, the heavy user is blocked at the user ceiling, counters reset after the window and the Redis-down test follows the agreed fail mode.

7. Known issues and fixes

Symptom Likely cause Fix
Counters never reset Expiry not set on keys created by INCRBY Set EXPIRE with NX after the first increment
Runs blocked during a Redis blip Fail-closed with no retry Retry once with a short timeout, then apply the agreed fail mode
Users exceed limits by a little Charge happens after the call Pre-check with an estimate and accept small overshoot, or lower the limit

8. Monitoring and alerts

Signal Alert when Action
Budget blocks per tenant More than 10 blocks in 1 hour Check for loops or a tenant needing a higher tier
Redis errors from the budget middleware Any for 5 minutes Check Redis health; confirm the agreed fail mode is active

9. AWS and open-source equivalents

On AWS, ElastiCache or MemoryDB for Redis provides the same counters. Open-source gateways such as LiteLLM offer per-key budgets that can complement code-level ceilings.

10. Validation record

Pending. Planned dry-run scenario: In the sandbox, a second engineer runs a scripted looping agent and a heavy user against the middleware. Pass: the loop stops at the run ceiling, the heavy user is blocked at the user ceiling, counters reset after the window and the Redis-down test follows the agreed fail mode.

11. Sources