Just asking whether it’s there
He only wanted to know if a variable existed; every secret got dumped into the AI chat.
(Illustrative scenario, not a real incident)
Engineer Xiao-An asked his AI assistant for a small favor:“That password I just set—did it get saved?”He wanted one answer: yes, or no.
Following the official docs, the AI ran a standard command.The command didn’t just answer “yes.”It printed every password on that service,line by line, in plain text.
Database passwords, third-party keys,all sitting in the chat log.And chat logs get saved,get sent to the AI company’s servers,and may get screenshotted and shared.
Nobody stole anything.Everyone was just following the rules.
Later someone suggested:other vendors’ tools mask passwords by default;to see them, you have to run one more command yourself.
What you see is a simple “is it there?”What you don’t seeis that, to answer it,the whole ring of keys was dumped on the table.
Keys used to be shown only to people,and people who saw them would remember to put them away.Now AI is watching too,and every word it sees stays behind.
Laozi said: “Know when to stop, and you will meet no danger.”Know where to stop, and nothing goes wrong.
Answer only what was asked.
Have the AI look up secret names, not values, and choose commands that mask secrets by default. Once a secret shows up in a conversation, treat it as leaked and rotate it right away.
The clever learn from their own mistakes.The wise are inspired by the mistakes of others.