jordanirabor.com / notes / pairwise-idsSide B · MMXXVI
Vancouver, Canada49.2827° N, 123.1207° W
← The Index

Notes · Privacy

Pairwise pseudonymous IDs, the hard way.

OpenID Connect gives you two kinds of subject identifier and about a paragraph of guidance on the harder one. public means every client sees the same sub for a user, which makes cross-service correlation trivial: two apps compare notes and they know you're you. pairwise means each client (strictly, each sector identifier) gets its own opaque sub, so that comparison yields nothing. The spec sketches example derivations, hashing a sector identifier with the local account ID and a salt, and then quietly leaves everything that matters to the implementer: what the salt is, where it lives, how it rotates, and what happens when any of those answers change under real users.

At ConsentKeys, pairwise subjects aren't a compliance checkbox, they're the product. Which meant the derivation had to clear four constraints at once:

  1. Deterministic. The same user at the same app must get the same sub forever, across deploys, migrations, and database restores. No stored mapping table as the source of truth.
  2. Uncorrelatable. Two colluding clients, comparing their full user lists, learn nothing. This has to hold even if one of them can make unlimited queries.
  3. Fast under load. The derivation sits on the token-issuance hot path. It runs on every authorization, not once at signup.
  4. Recoverable when something goes wrong. If a salt or key is ever suspected compromised, there has to be a rotation story that doesn't silently re-identify every user at every app.

There are three routes an implementer will seriously consider, and each fails a constraint if you push on it. A random-mapping table, where you generate an opaque ID per (user, app) pair and store the mapping, is the intuitive one, and it violates the first constraint's spirit immediately: the table is the identity system now, it grows with every pair, and a restore that loses a row loses a user. Encrypting the local subject per client (an AES-SIV style construction) is deterministic and tableless, but it's reversible: the thing you hold can be decrypted back to the real identifier, which turns a key compromise into a full de-anonymization event rather than a correlation risk. And the spec's own sketch, hashing the sector identifier, the local ID, and a salt together, is directionally right but underspecified in exactly the places that hurt: one global salt means one secret whose compromise breaks uncorrelatability everywhere at once, and a bare hash gives you no principled story for domain separation or rotation.

The construction we settled on at ConsentKeys is HKDF with per-app salts. HKDF's extract step condenses a master secret into a pseudorandom key; the expand step derives a per-context output using an info parameter, which is where the sector identifier and the per-app salt go. The properties fall out of RFC 5869 rather than our own cleverness: outputs are computationally independent across contexts, so no number of (app, sub) pairs helps colluding clients link users without the master secret. And the per-app salt means the blast radius of any single leaked salt is one app's namespace, not the world.

The spec is also silent on the two questions that will actually consume your design review. First, rotation: any procedure that changes the master secret or a salt changes every derived sub downstream, which from a client's perspective is every user disappearing and being replaced by a stranger. Whatever your recovery story is, it has to preserve continuity deliberately (a dual-derivation window, a migration mapping, a re-consent flow) because nothing preserves it by accident. Second, the client that legitimately needs to correlate identities across two of its own apps. The spec's own answer is the right one: that's what the sector identifier is, one sector spanning both apps, correlation by declared design rather than by side channel. The dishonest version of the request, two sectors that want to compare notes after the fact, is precisely the thing the whole construction exists to refuse, and the refusal has to survive the customer asking nicely.

What I'd tell an implementer starting today: read what the spec doesn't say as a list of decisions you own. Every one of them (salt scope, storage, rotation, the recovery path) eventually fires in production, and the spec's silence won't be there to help you. The part that deserves your hardest review is not the derivation, which is a solved primitive, but the lifecycle around it. Deriving a pairwise ID is one line of HKDF. Keeping it stable for years, across rotations and restores and the customer's org chart changing under you: that's the system.