Catching the ring without breaking the customers inside it
A seven-figure flagged account turned out to be reserved-but-unused GPU capacity, not fraud. Reversals like that became the carve-out gates every bulk shutdown now passes through.
Business Analyst, AWS Payments & Fraud Prevention
Fraud calls reversed on legitimate accounts, including a seven-figure account proven to be reserved GPU capacity, then hardened into a standing control: bulk ring actions catch the abuse with ≈0 collateral, every wave ships with its legitimate-impact exposure priced for sign-off, and known false positives are catalogued so they're never re-flagged.
- Verification workflow Prove what the flag actually is before it counts as a verdict.
- Hard-identifier-only matching Bulk-action SQL matches on hard identifiers, never brittle heuristics.
- Legitimate-impact carve-out The named real accounts come out of the batch, not out of an apology later.
- Confidence pass Adversarial review, drop-the-top-signal sensitivity, leave-one-out stability.
- Exposure quantified The legitimate-impact exposure for the wave, written down.
- Stakeholder sign-off No batch action runs before the priced exposure is signed.
Show the data table
| Stage | # | What it checks |
|---|---|---|
| Verification workflow | 1 | Prove what the flag actually is before it counts as a verdict. |
| Hard-identifier-only matching | 2 | Bulk-action SQL matches on hard identifiers, never brittle heuristics. |
| Legitimate-impact carve-out | 3 | The named real accounts come out of the batch, not out of an apology later. |
| Confidence pass | 4 | Adversarial review, drop-the-top-signal sensitivity, leave-one-out stability. |
| Exposure quantified | 5 | The legitimate-impact exposure for the wave, written down. |
| Stakeholder sign-off | 6 | No batch action runs before the priced exposure is signed. |
Fraud systems flag legitimate accounts too, and the expensive failure is acting on those flags at scale. When you action a coordinated ring in bulk, the abusive accounts aren't the risk. The legitimate ones sitting inside the same signature are: a shared NAT, a placeholder fingerprint, a common BIN, and suddenly the candidate list has real customers in it.
- I reversed fraud calls on two legitimate accounts by proving what each flag actually was: one a seven-figure flagged-OPEX exposure, the other a shutdown that traced to a billing soft-decline rather than fraud.
- I then made false-positive aversion a standing control rather than a closing sanity check.
- Bulk-action SQL is restricted to hard-identifier matches, never the brittle heuristics that sweep up look-alikes.
- Before any wave, a legitimate-impact carve-out analysis names which real accounts the signature would catch and removes them before suspension.
- A confidence pass has to clear before anything ships: adversarial review, drop-the-top-signal sensitivity, leave-one-out stability.
- Every batch carries quantified exposure to stakeholders for sign-off before it runs.
- Known false-positive accounts are catalogued as first-class do-not-shut calibration anchors.
Rings still get closed. What changed is the order of operations: the evidence gets checked before the action instead of after the apology, and the gates run whether or not anyone is feeling careful that week. Precision stops being a virtue and becomes a step you can audit.
The startup loses its infrastructure over a spend pattern that was never fraud, and the next bulk action sweeps legitimate customers up with the ring. That manufactures a customer incident, burns trust at scale, and forces reactive reversals and apologies instead of preventing the harm.
The flag said fraud. The account was spending like fraud: a seven-figure OPEX exposure with barely any usage behind it. Shutting it down would have looked decisive and been wrong. The evidence said reserved-but-unused GPU capacity at a VC-backed startup, which is what a company stockpiling scarce compute looks like from the billing side. A second case in the same family: a shutdown that read as fraud enforcement traced back to a billing soft-decline. Both calls were reversed because I checked what the flag meant before treating it as a verdict.
Individual saves don’t scale, and both reversals taught the same thing: a flag is a hypothesis, not a verdict. Reversed once, each of those accounts became a do-not-shut anchor. The catalogue started there. Scaled up, that lesson is the whole argument for the gates.
So I built the second half as structure, and the choices in it are the substance. Hard identifiers instead of heuristics, because a heuristic match is a guess about a customer while an identifier match is a fact. A carve-out analysis before the wave rather than a review after it, because a name you read before suspension is a customer you still have. And the confidence pass exists because a cluster held together by one signal is one bad signal away from being a customer incident. The cluster has to survive losing its strongest link before anyone acts on it.
The piece I’m proudest of is the smallest. Those do-not-shut anchors are kept as first-class calibration cases, not closed tickets, so a legitimate customer who once looked like a ring member never gets re-flagged by the next iteration of the rule. The system has a memory for its own near-misses.
None of this was meant to slow enforcement down. It was meant to make the fast path one that already checked.