AI Terms Glossary: Tokens, Prompts, RAG, Hallucinations and More, Explained

AI coverage is full of words that sound like they were designed to keep ordinary readers out: tokens, parameters, RAG, embeddings, hallucinations. In truth, each of them names a simple idea, and once you have the ideas, the jargon loses its power — including the power some sellers and scammers use to make ordinary products sound magical. This glossary explains the essential terms in plain English, with each definition tied to how the technology actually behaves. Keep it beside our plain-English guide to how modern AI works and our explainer on what a large language model is.

Token

A token is the small, machine-readable unit that text is broken into before a model processes it. IBM’s LLM explainer describes tokenisation as the process of breaking text down into tokens — smaller units such as words, subwords or characters — which standardises language so rare and novel words can be handled consistently. A common word might be one token; a long or unusual word might be several. Tokens explain two things users bump into daily: why AI services have usage limits measured in tokens rather than words, and why a model can stumble over unusual spellings or rare names — they arrive as unfamiliar token pieces. When a service says a model accepts a certain number of tokens, it is telling you the size of the desk the model can spread your documents across.

Prompt, context window and parameters

A prompt is simply the input you give the model — your question or instruction, plus any text you include with it. Prompt engineering is the practice of shaping that input: IBM notes it is the easiest and fastest way to get domain-specific responses from a general-purpose model, because modifying the prompt changes what the model generates without any retraining. The context window is the maximum number of tokens the model can see and use at once when generating text; newer models have windows of hundreds of thousands of tokens, enough for entire research papers or long conversations, but anything outside the window is effectively invisible to the model in that moment. Parameters are the model’s internal settings — the billions of learned weights that control how it processes data and makes predictions. Where tokens are the words on the desk, parameters are the trained judgement doing the reading: they were set during training and do not change because you had a nice chat. That is why training, not chatting, is where a model learns.

Embedding, transformer and inference

An embedding is what a token becomes inside the model: a list of numbers — a vector — that places the token in a kind of meaning-map, where words used in similar contexts sit close together. IBM explains that the goal is for the model to learn semantic associations, so that “bark” sits nearer “dog” than “tree” in a passage about dogs, based on the surrounding words. The transformer is the architecture that does this work, using self-attention to weigh how much each token should attend to the others around it; it is the design every modern LLM is built on. Inference is the name for using a trained model: you give it a prompt, it predicts one token at a time until the reply is complete. Training builds the model; inference is the model answering you. The distinction matters because inference is fast and cheap compared with training, and because during inference the model does not learn — it applies what training taught it, nothing more.

Fine-tuning, RLHF and temperature

Fine-tuning is additional training that adapts a general model to a specific task or field — IBM gives the example of a foundation model fine-tuned on legal questions and answers to create a chatbot for the legal field. RLHF, reinforcement learning from human feedback, is a fine-tuning method where humans rank model outputs and the model is trained to prefer the outputs people ranked higher; IBM notes it is often used for alignment, making outputs more useful, safe and consistent with human values, and for adjusting style. Temperature is a generation setting that controls randomness: IBM describes it as controlling how random the text an LLM generates is during inference, with sampling settings balancing creativity against coherence. Low temperature gives you the model’s most predictable answer — right for factual extraction. High temperature gives variety — useful for brainstorming, risky for facts. None of these change what the model knows; they change how its knowledge is adapted, preferred and expressed.

RAG and hallucination: the pair that decides trust

RAG stands for retrieval-augmented generation. IBM defines it as a method for connecting a pretrained model with external knowledge bases, enabling more relevant responses at a higher level of accuracy: the system retrieves relevant information, passes it into the model’s context window, and the model generates its answer using that material — without retraining. It is how a company chatbot can answer from today’s policy documents instead of last year’s training data. A hallucination is the failure mode RAG helps reduce: the model generates a statement that is fluent and confident but fabricated — a citation that does not exist, a date that never happened. Hallucinations are not the model lying; it has no intent. They are the predictable result of a system that predicts likely language: when the pattern runs ahead of the facts, the output still sounds like an answer. Knowing this term protects you twice — you verify important AI answers, and you stay immune to the fear covered in our guide to AI myths and scams, where fluent fakes are the whole business model. One more term belongs here: an agent is a model connected to tools and goals so it can act, not just answer — defined fully in our agents versus chatbots explainer.

What this means

A glossary might look like the driest article in this series, but it may be the most useful one, because vocabulary is leverage. Every term above converts a moment of confusion into a question you can ask. A vendor says their tool “uses embeddings and RAG” — now you can ask: RAG over which documents, updated how often? A service advertises a giant context window — now you know that means it can read more at once, not that it remembers you or knows more. A headline warns that a model “hallucinated” — now you know that is a known, mechanical failure mode with known mitigations, not evidence the machine is scheming. And when a scammer or salesman leans on jargon to rush you, the terms give you the calm to slow down and translate. The deeper pattern is that none of these words describes magic. Tokens are text cut small. Parameters are settings set by training. A prompt is your input; inference is the answer being built; RAG is looking something up before answering. Plain ideas, dressed in technical names. Learn the names once, and AI coverage, product pages and sales pitches all become readable — which is exactly what this whole series exists to give you.

Bottom line

Jargon is a fence, and every term in this glossary removes a plank. Once tokens, RAG and hallucinations are plain ideas, vendor claims and scary headlines lose their power over you.

Read Next

• What Is a Large Language Model? LLMs Explained in Simple Words
• AI Myths and Scams: What to Believe and What to Avoid

Sources

• IBM Think — What Are Large Language Models (LLMs)? (accessed 6 November 2026)
• IBM Think — What Is Machine Learning? (accessed 6 November 2026)
• Google Cloud — What Is Artificial Intelligence (AI)? (accessed 6 November 2026)