Bias: a real case

Models learn patterns from historical data. When that history contains human bias, the model learns the bias as if it were just another pattern, and then applies it at scale, consistently, and with a straight face.

Two details make this case the perfect teaching example. First, nobody programmed the bias; the model found it in the data, which is exactly how modern AI failure works. Second, one of the world's most sophisticated engineering organisations could not be confident the bias was fully removed, and walked away from the tool instead.

What this means for your work

• Be most cautious exactly where AI touches decisions about people: hiring, evaluation, credit, access to services. This is also where regulators are most demanding; the EU AI Act treats many recruitment and employee-management uses as high-risk, with duties phasing in over the coming years.
• Bias is invisible in any single output. One polished CV summary looks fine; the pattern only appears across hundreds. That is why individual users cannot "spot check" bias away, and why organisations must test such systems deliberately.
• If you use AI to summarize or rank anything about people, keep a human decision-maker genuinely in charge, not just formally in the loop. In places this is stricter than the letter of the law; it is the standard that keeps your decisions defensible.

Bias also appears in subtler forms: models trained mostly on English text understand other languages and cultures less well, and content generated "by default" tends to reflect majority perspectives. Awareness of that is part of reading AI output critically.