How generative AI works, roughly

You do not need to understand neural networks to use AI safely, but one mental model is worth installing, because it predicts almost everything these tools do well and badly.

Training happened in the past, on a snapshot of text. That has three practical consequences you will meet constantly:

1. Knowledge cutoff: the model's built-in knowledge stops at its training date. Unless the tool explicitly searches the web, it does not know yesterday's news, current prices, or the latest version of a regulation.
2. Fluency is not accuracy: the model was optimized to produce text that reads well. Polished, confident prose is its default register, applied equally to correct and incorrect content.
3. Garbage in, gospel out: whatever assumptions your prompt contains, the model tends to accept and build on. Ask "why is our competitor's product worse?" and it will oblige, whether or not that is true.

Where your text goes

When you use a chatbot, your prompt travels to the provider's servers and is processed there. What happens next depends entirely on the tool and the plan. Consumer versions of many tools may keep conversations and use them to improve their models unless you opt out. Enterprise versions typically promise not to train on your data and add contractual protections. Same brand, very different rules. This single difference is why your organisation cares which tools and which accounts you use, and it is the subject of Chapter 4.

One more consequence worth knowing: because models learn patterns from their training data, text that appeared frequently online is what they reproduce most confidently. Niche, recent, internal, or specialized knowledge is exactly where they are weakest, and unfortunately that describes a lot of real work.