Why teachers need to understand it first

Generative AI is not a search engine, and it is not a calculator. The way it works decides what it is good at and where it goes wrong, and both of those bear directly on whether it can be used in class with confidence. Rather than treat it as a mysterious black box, it is better to understand one simple principle first, which you can then draw on every time you judge whether an answer can be trusted.

Large language models are really predicting the next word

At the core of today's generative AI is the large language model. It learns the patterns between words from a vast amount of text, and what it does in operation is really one thing: based on the words that came before, it predicts the next most likely word, generating a sentence one word at a time. It has no database it can look up for right or wrong, and it does not understand a question the way a person does; it is working out which combination of words most resembles the answer a person would write.

This one feature explains almost all of its strengths and weaknesses. Because it is imitating human writing, what it produces reads smoothly, is well organised and sounds natural; and because what it is after is resemblance rather than correctness, it can write wrong content just as smoothly, with nothing to back it up.

What it is good at, and what it is not

Look at the strengths and weaknesses separately and you will not misjudge how to use it.

  • Good at: rewriting and polishing text, generating plenty of examples and variations, summarising long passages, explaining a concept another way, and drafting a first version for someone to revise
  • Not good at: giving accurate facts, figures and quotations; handling multi-step calculations that need rigorous reasoning; judging whether information is true; and answering questions about the latest or obscure information
  • A useful dividing line: for any task about how something is expressed, AI is usually reliable; for any task about whether a fact is right, verify it

Why it gets things wrong so fluently

The one point a teacher most needs to remember is that AI can give a wrong answer with great confidence. What it generates is the text that most resembles an answer, and the text that most resembles an answer is not necessarily the right one. Unless it is specially set up to, it will not say 'I am not sure', and instead writes a guess as firmly as a fact. This is often called AI hallucination, and it is not an occasional fault but a natural result of how the technology works.

So if you ask AI about a writer's life, where a formula came from, or the details of a news story, it may answer very convincingly while getting the year, the name or the number wrong. The more precision a thing needs, the more it has to be checked.

Three pointers for classroom use

  • One, use it for expression, do not take its facts on trust. Letting AI help rewrite comments, generate practice or explain a concept is fine; for content that involves specific facts, the teacher checks it before using it.
  • Two, teach students to see through it with you. Rather than ban it, take students through the moments when AI gets things wrong; verifying beats blocking.
  • Three, get comfortable with it yourself before bringing students in. A teacher's understanding of AI is one of the things the programme wants to build; once you use it easily in lesson preparation and admin, classroom use follows naturally.