“Is anyone changing something?”
The site suddenly stopped taking bookings, and what I found was a script, not a person.
(Illustrative scenario, not a real incident)
That day, the website suddenly stopped taking bookings.
Customers came in, clicked, and nothing happened.I typed one line into the group chat:“Is anyone changing something?”
No one said yes.
A whole day of business simply slipped away.
I used to think that when a site breaks,it's usually one of us slipping up.Later I learned the thing to worry about mostisn't “someone is changing things,”but “no one is changing anything, and things changed anyway.”
I spread out the logsand noticed something strange:there was almost no “person” in them.
Only a few seconds between one action and the next.First a quick round of tries across several regions,then all the firepower focused on one place.
That was not a person at a keyboard.That was a script.
A hacker is like the most diligent entrepreneur:costs close to zero, the bill charged to you,never picky about customers, never on vacation, never tired.
So we changed one way of thinking:it's easier to limit people than to limit machines.Let services be operated by people wherever possible,and block unfamiliar robots first.
Laozi said: “The great affairs of the world must begin with the small.”Whether a company runs into real troubleis often hidden inside one quiet“it's probably nothing.”
Not every outage is a hacker.But every outagedeserves a second look.
For every anomaly you can't explain, first check the logs for who did what, and from where. Block unfamiliar automated traffic first, and let through only your own robots.