“Just get it running first”
The “just get it running first” AI agent held the keys to the entire database.
(Illustrative scenario, not a real incident)
This forty-person online grocerbuilt a customer-service AI agentto look up orders and update contact details.
During development, to save effort,they simply gave it a database admin account.“Just get it running first.We’ll tighten the permissions later.”
After launch, everyone got busy with other things.That “later” never came.
One day, a customer typeda carefully crafted message into the chat box.The agent did as it was told,and pulled up orders that weren’t this customer’s.
The review afterward found:the agent needed only two permissions,“read orders” and “update contact details,”yet it could have dropped entire tables.
Every word a customer typesshapes what the agent does.Whatever an admin can do,in theory, it can do too.
This kind of problem has a name:excessive agency.Too many functions, too many permissions, too much autonomy.
Laozi said: “Know the masculine, keep to the feminine.”To know you can do a great deal,and still choose to stay where you belong.
So it is with people.So it should be, even more, with agents.
- List what the agent actually needs to do.
- Give it a dedicated account with only those permissions.
- Leave deletions and refunds for a human to confirm.