API Rate Limiting Without Storing Who You Are

API rate limiting without PII decides a caller is going too fast without saving who they are.

Standard setups have a privacy problem everyone ignores. They almost always use a user ID, an IP, or a plain text API key as the limiter. That turns the store into a record of who did what and when. It lives in Redis, which normally sits outside the retention rules for your main database.

The ChatFuse limiter doesn't keep any of it. These are the rules it runs on.

What is forbidden from the ChatFuse limiter?

We never use raw tenant or user IDs, session identifiers, API keys, tokens, cookies, email addresses, IPs, prompt text, content, or request URLs as a limiter key. They can't go into limiter logs or metrics either. This rule covers both what you pick for the key and what you write down about it. Both parts are equally important.

Many platforms run the key through a hash and then write the exact subject to a log right beside the outcome. They're literally putting the sensitive data right next to its protection, which is the precise thing the hash was built to stop.

How do you throttle someone without storing who they are?

A fixed width hash gets built from the request, and that's the only thing we store next to a counter. One function handles making the storage key, and it runs 5 steps in a row. You can't reverse it without our service secret. All Redis keeps are counters next to those hashes, which covers everything a rate limiter does.

The function starts by normalizing a scope we set in code. Then it length frames the subjects with canonical serialization. Next, it hashes that result with HMAC-SHA-256 using our secret. After that, it adds the namespace and environment. The final output that goes into Redis is that fixed width hash.

Why does length framing matter?

Length framing is needed when you build keys out of strings. Without it, you get both collisions and an endless number of keys. A subject that includes the delimiter can pretend to be a different pair. And a key made from something like a path lets someone create as many new keys as they want.

Prefixing every subject with its length stops both issues. A caller can't just change a path to create unlimited distinct keys and fill your keyspace. An input that's too long or has the delimiter in it can't produce a key that stands for something else.

It feels like paranoia until someone actually finds this problem.

What stops a new endpoint from skipping all this?

A gate script in ChatFuse that fails closed, not a reviewer. It searches the whole codebase for any rate limiting reference it can find. Every single one must be part of a path group that's already been reviewed. The total number of references and their sorted SHA-256 fingerprint have to match the manifest exactly. If they don't, the build stops.

The code checks for rate limiter setup and use, direct Redis calls for counting, 429 error handling, the things that own the official response, browser gate logic and its bypasses, the retry header itself, plus branches for recovery and retry helper functions.

A new file won't have a claim, so it gets flagged. If you change something in a file that already passed review, that shifts its signature and it gets flagged too. You can't sneak a new 429 check in without it being seen, and you can't tweak an old one on the quiet.

Is a fingerprint gate not just annoying?

It's frustrating, but we signed up for it. If you're working on anything that hits a rate limit, you have to change the manifest. That makes a person review your code. The other way is to write a rule in some document that no one ever searches for, and that's exactly how rate limits start to slip.

We'd rather see the build break for an obvious reason than have a rule that everyone likes but no one actually follows.

What does a ChatFuse client see when it gets limited?

A client hitting their rate limit gets a 429 from ChatFuse, and it comes with a retry header. This reply comes from a single owner, not from all the different endpoints on their own. With one source, you get one consistent shape. A browser can recognize that.

Even if a proxy is sitting between the user and our backend, the status and the header still get through. They don't get flattened into a generic error page. That flattening is what makes a user think there's a full outage, when it's really just a rate limit.

Does ChatFuse limit concurrency as well as request rate?

Yes, and for a streaming product like ChatFuse, the cap on simultaneous streams is the one that really matters. Requests per minute isn't the right way to count when a single request can tie up a connection for a full minute. That's why we limit streams apart from requests, since they're protecting 2 different things.

10 streams at once cost a lot more than 10 requests that run one after another and each take 200ms. We cover why streaming changes how you measure a chat product in LLM streaming latency.

Does rate limiting change my credits or plan?

No. We keep throttling and billing completely separate. The contract is crystal clear that it doesn't deal with your billing rights, your credit balance, or how providers work. So if you get rate limited for a bit, it doesn't count as a charge and you don't lose any credits.

If we combined these 2 things, we'd build a system where stopping abuse could secretly burn through someone's money.

We explain how credits work over at AI credits vs tokens.

What does the browser do when it hits a limit?

The server gives it a number, and it sticks to that number exactly. It doesn't pick its own timing. This single spot in the browser handles the retry logic, so it's in one place. That means you don't get a dozen different call sites all trying to do their own thing with slightly different results.

When a client sees a limit and just retries right then, that's the scenario we have to stop. You don't want a single call turning into a loop that pounds the endpoint. That's the very traffic the limiter is supposed to prevent, but now it's coming from inside your own system. If you had a bunch of different backoff methods, most of them would probably be wrong.

And because the gate checks all the recovery paths for 429 and 409 responses, any new code that tries to handle a limit on its own won't pass the build. It doesn't get to go out.

Is API rate limiting the same as abuse prevention?

No. Rate limiting just controls how much gets through. It can't catch everything. You also need a second layer of rules that spot bad behavior, even when it doesn't break any volume limits. Think of someone who still manages to stay under every single threshold while doing something they shouldn't.

We covered that kind of deliberate attack in AI guardrail testing, and how we set up a system to hit our own product on a regular schedule in automated AI red teaming.

Can I still be identified if the secret leaks?

Someone with the secret can check if a subject makes a specific key. So this derivation step doesn't mean you can stop protecting that secret. But it does get rid of the plain text log that just sits there. If an attacker grabs the limiter store, all they see is your traffic pattern. They learn nothing about who made which key.

Our full take on standing access is over at zero trust AI data security.

Where can I read what ChatFuse holds?

All the data types we keep and why we keep them are explained on our security page instead of being repeated here. You can start free to use the product, or you might want to read about zero trust AI data security first. With ChatFuse you get over 100 AI models from Google, Meta, OpenAI, Anthropic and others.