Notes on an incident from spring 2024, when I was global sysadmin at Infoblox.

Lockouts are the most boring ticket there is

Every IT queue is full of them. Someone mistyped a password, someone’s phone died and they cannot do MFA, someone came back from PTO and forgot which password they rotated to. You unlock, you move on.

That is also what a credential-stuffing attack looks like from the service desk. It arrives one ticket at a time, in the most ignorable format the queue has.

What caught my attention

The volume was unremarkable. What was off was who was getting locked out.

Normal lockouts are random. They scatter across the org, one in finance, one in sales, one in engineering, with no relationship between them. That is what unrelated user error looks like.

These clustered, and they clustered around people who worked together. When the victim list starts to look like a slice of the org chart instead of a random sample, something is selecting for those accounts. Randomness does not produce org structure.

So I escalated to the security lead and the SOC while it was still building, instead of working the tickets one by one.

Why it looked like that

Root cause was exposure, not an infrastructure weakness.

A chunk of the marketing organization had just come back from a large industry event. Their corporate email addresses had been harvested there (trade shows concentrate real, current, verified corporate addresses in one place) and those specific addresses were then run against the VPN with credential stuffing.

That is why the victim list looked like a department. It was a department. The attacker did not pick them for who they were internally; those were simply the addresses they had.

Over 15 users were locked out before it was mitigated. It got a formal incident number and a post-mortem.

The thing I still think about

Nothing detected this before I did. I want to be careful about that sentence, because it is the point of the post and not a brag.

There was tooling. There was a SOC. But the lockouts arrived through the support queue, and the support queue is a workflow system, not a security tool. It also happens to be where the earliest, most direct evidence of an identity attack shows up: real users, in real time, saying they cannot get in.

The gap was that “account lockout” is filed as a support event, and support events get resolved instead of analyzed. Every one of those tickets was individually closeable. The signal only existed in aggregate, and nothing in the workflow encourages anyone to look at the aggregate.

What I would tell someone on a service desk

Look at who, not only how many. Volume thresholds catch loud attacks. Shape catches quiet ones. A modest number of lockouts concentrated in one team is more interesting than a larger number spread evenly.

Correlate against the outside world. The unlock here was remembering there had been a conference. Recent public events involving your people (trade shows, a funding announcement, a press cycle, layoffs) change your exposure and explain victim lists that otherwise look arbitrary.

Escalate on pattern, not on certainty. I did not know it was an attack. I knew the distribution was wrong. Waiting until I could prove it would have cost the SOC the time they used. Being wrong would have cost a slightly embarrassing message; being late would have cost every additional account they got through.

Your queue is a sensor. It is staffed by people who talk to users all day and know what normal looks like. That intuition is usually the first thing to notice when normal changes.

The most useful security habit I picked up was asking why these people, instead of clearing the ticket.