Rate Limiting Without Storing Who You Are

API rate limiting is how a service decides that a caller has asked for too much, too fast, and the standard implementation creates a privacy problem nobody mentions: the limiter key is almost always a user ID, an IP address, or an API key stored in plain text.

That store is a log of who used what and when, sitting in Redis, usually outside whatever retention policy covers your actual database.

The ChatFuse limiter holds none of it. Here is the contract.

What is forbidden from the ChatFuse limiter?

Raw tenant IDs, user IDs, session IDs, API keys, tokens, cookies, emails, IP addresses, prompts, content, and request URLs. None of them may be a limiter key, and none may appear in limiter logs or metrics either.

The log and metric half matters as much as the key half. Plenty of systems hash the key and then log the raw subject next to the decision for debugging, which recreates the thing the hash was there to avoid.

How do you throttle someone without storing who they are?

ChatFuse derives a fixed width digest and stores only that. One function is the sole production path for building a storage key, and it does 5 things in order.

The scope, which is defined in code rather than taken from the request, is normalized and bounded. The subjects are length framed through canonical serialization. That serialization is then keyed with HMAC-SHA-256 using the service secret. The namespace and environment are folded in. What lands in Redis is a fixed width digest.

Reversing it requires the secret. Redis holds counters against opaque keys, which is all a rate limiter ever actually needed.

Why does length framing matter?

Because concatenating strings to build a key is how you get collisions and unbounded cardinality at the same time.

Join two fields with a delimiter and a subject containing that delimiter can impersonate a different pairing. Build keys from a path or query string and a caller can mint unlimited distinct keys by varying it, which both defeats the limit and fills your keyspace. Framing each subject by length before serializing removes both, because an oversized or delimiter carrying input can no longer produce a key that means something else.

This is the kind of detail that looks like paranoia until the first time someone finds it.

What stops a new endpoint from skipping all this?

A ChatFuse inventory script that fails closed, run as a gate rather than trusted to review.

The script scans for limiter constructors and calls, direct Redis rate counters, 429 branches, canonical response owners, browser gate and bypass owners, the retry header, recovery branches, and retry helpers. Every reference it finds has to belong to exactly one reviewed path group, and both the count and a sorted SHA-256 fingerprint of the references have to match the manifest.

A new file is unclaimed and fails. A changed occurrence in an already reviewed file changes the fingerprint and fails. You cannot add a quiet 429 somewhere and have it slip through, and you cannot silently alter an existing one.

Is a fingerprint gate not just annoying?

It is annoying, and that is the trade. Anyone touching a rate limited path has to update the manifest, which forces a human to look at the change.

The alternative is a rule in a document that nobody greps for, which is how throttling drifts. We would rather have a build failure with a clear cause than a policy everyone agrees with and nobody enforces.

What does a ChatFuse client see when it gets limited?

A 429 with a retry header, from one canonical response owner rather than assembled per endpoint.

One owner means one shape, so the browser has exactly one thing to recognize. When a same origin proxy sits in front of the backend, the status and the retry header propagate through rather than being flattened into a generic error, which is how a limited request ends up looking like an outage to the user.

Does ChatFuse limit concurrency as well as request rate?

Yes, and for a streaming AI product like ChatFuse the concurrency cap is the one that carries more weight.

Requests per minute is the wrong unit when a single request can hold a connection open and stream for a minute. Ten concurrent streams cost far more than 10 sequential requests that each finish in 200ms. So streams are capped separately from request rate, because the resource being protected is different.

Does rate limiting change my credits or plan?

No. Throttling and billing are deliberately separate. This contract explicitly does not touch billing entitlements, credit balances, or provider semantics. Being briefly limited is not a charge and does not consume anything.

Keeping them apart matters because merging them creates a system where an abuse control can silently cost someone money.

What does the browser do when it hits a limit?

It waits the amount it was told to wait, and it does not decide that number itself.

The failure mode worth designing against is a client that treats a limit as a transient error and retries straight away. That turns one limited caller into a loop hammering the endpoint, which is the exact traffic the limiter existed to stop, now generated by your own frontend.

So the retry interval comes from the server and the browser honors it. One gate owns that behavior rather than each call site implementing its own backoff, because a dozen call sites means a dozen slightly different recovery behaviors and only some of them are correct.

The recovery branches for 429 and 409 are also part of the inventory the gate scans, so a new call site that handles a limit its own way fails the build rather than shipping.

Is API rate limiting the same as abuse prevention?

No. Rate limiting is a volume control and it is one input to abuse prevention. Secondary guardrails handle patterns that are not about volume, like a caller staying under every threshold while doing something they should not. We wrote about the adversarial side of that in AI guardrail testing.

Can I still be identified if the secret leaks?

If someone has the service secret they can test whether a specific known subject produces a specific key, so the derivation is not a substitute for protecting the secret. What it does remove is the standing plain text record. A dump of the limiter store on its own tells an attacker the shape of your traffic and nothing about who generated it.

Our wider position on standing access is in zero trust AI data security.

Where can I read what ChatFuse holds?

Every category of data we store, and why, is on the security page rather than summarized here. Start free if you want to try the product, or read zero trust AI data security first.