[ Chapter 4 · Recognizing and managing AI risks at work ]
Data privacy and leaks
4 min read
Forget science-fiction risks. One of the most common AI incidents in real organisations is mundane: an employee pastes confidential material into an external AI tool to save time. A contract to summarize, source code to debug, meeting notes to clean up, a customer complaint to answer. In that instant, the data has left the organisation and now sits with a third party, under that third party's terms.
Why it happens to smart people
Nobody thinks of pasting as "sending". The chat window feels like a private scratchpad, the way a calculator or a notes app is. It is not. It is a form that transmits everything you type to another company's servers, where it may be retained, logged, reviewed for abuse, or, on some consumer plans, used to train future models. The feeling of privacy is a design accident, not a fact.
What counts as sensitive
- Personal data: anything identifying customers, colleagues, or candidates: names, emails, HR records, health details. In Europe this is GDPR territory, with legal obligations attached.
- Confidential business information: source code, financials, strategy, unreleased products, incident details, anything covered by an NDA.
- Credentials and access: passwords, API keys, internal URLs, security configurations. Never, in any tool, under any plan.
- Third-party material: your client's data or your partner's documents carry their confidentiality obligations with them.
The fix is rarely "never use AI". Often the same task works with the sensitive parts removed or replaced: "summarize this contract" becomes "summarize this contract template with names and amounts replaced by placeholders". You get the productivity; the data stays home. Chapter 5 turns this into a routine.