ProjekinAja. Agentic AI Stage 1 · Foundations Lesson 1.3
● Stage 1 · Lesson 1.3

The LLM Engine — What It Is and Isn't

Before agents, the engine. A large language model is a next-word prediction machine — astonishingly capable, and not at all what most people think it is.

⏱ ~13 min read 🧠 The brain inside 🔬 Demystifying

🎯 By the end you'll understand

What a large language model fundamentally does (predict the next token, over and over), why that simple trick produces such fluent, reasoning-like behavior, what it genuinely is not (a knower of truth, a memory, a browser, a doer), the few practical properties worth knowing — tokens, context window, temperature, training cutoff — and why all of this becomes the strengths and limits of every agent you'll ever build.

KEY INSIGHTThe one idea to take away

A large language model does exactly one thing: it predicts the most likely next chunk of text, given everything it has seen so far. Everything else is that, repeated.

It looks like the model is thinking, planning, remembering, and reasoning. Under the hood it is doing something far humbler and far weirder: it reads the text so far and answers a single question — "what word probably comes next?" — then it adds that word, reads everything again, and asks the same question. Fluent essays, working code, step-by-step reasoning: all of it falls out of that one move done thousands of times.

🔑 Hold onto this

The LLM is the brain in the agent formula you met in Lesson 1.1 — LLM + tools + loop, pointed at a goal. But the brain by itself can't do anything. It only produces text. Understanding what that text-machine can and can't do is the difference between trusting an agent wisely and being burned by it.

CORE CONCEPTWhat it actually does: predict the next token

Forget the word "intelligence" for a minute. Here's the mechanism, with nothing hidden.

You hand the model some text — say, The capital of France is. The model converts that into a list of tokens (roughly: word-pieces — more on that below), looks at the whole sequence, and outputs a giant probability table over its entire vocabulary: Paris at 92%, located at 3%, a at 1%, and so on for tens of thousands of possible next tokens.

Then it picks one — usually a likely one — appends it to the text, and does the whole thing again with the now-slightly-longer text. The capital of France is Paris → predict next → maybe . or , or a. One token at a time, left to right, until it predicts a special "stop" token or hits a length limit.

📖 Token

A token is the unit an LLM reads and writes — typically a word-piece, not a whole word. cat is one token; unbelievable might be three (un, believ, able); a space and punctuation count too. A rough rule of thumb in English: ~1 token ≈ ¾ of a word, so 1,000 tokens is roughly 750 words. Models think in tokens, are billed in tokens, and have limits measured in tokens.

That's genuinely the whole loop at the lowest level. There is no separate "understanding" step, no internal database it queries, no little reasoner sitting beside the predictor. There is only: read the text so far → predict the next token → repeat. Everything impressive an LLM does is an emergent side effect of getting very, very good at that one prediction.

CORE CONCEPTWhy a guessing machine sounds so smart

If it's "just" predicting the next word, why does it feel like talking to something clever? Because of what it had to learn in order to predict well.

The model was trained on an enormous amount of human text — books, code, articles, conversations, documentation. Training means: show it a passage with the next token hidden, let it guess, measure how wrong it was, and nudge its billions of internal numbers (parameters) to be a little less wrong. Do that across trillions of tokens.

Here's the key move: to predict the next word well, the model is forced to absorb the patterns behind the words. To finish The boiling point of water at sea level is with 100, it has to have soaked up a fact. To continue a half-written function correctly, it has to have internalized how code is structured. To answer "if all bloops are razzies and all razzies are lazzies, are all bloops…" it has to have picked up the shape of logical inference from millions of examples.

💡 The honest framing

The model didn't memorize answers — it learned the statistical regularities of human knowledge and reasoning well enough to reconstruct plausible continuations. That's why it can handle questions it never saw verbatim. It's not retrieving; it's generating what a knowledgeable person would likely say next.

This is also why people argue endlessly about whether it "really reasons." For our purposes the answer is practical, not philosophical: it produces reasoning-shaped output that is often correct and sometimes confidently wrong. Treat it as a brilliant, fast, well-read pattern-matcher that has never once checked whether what it's saying is true.

SEE ITThe next-token predictor, laid bare

Pick a prompt below. The bars show candidate next tokens with rough probabilities. Press Predict next → to let it pick the top token, append it, and show the next set of guesses. Watch how a sentence gets built one token at a time — that's the entire engine.

🔮 Interactive · Next-Token Predictor

Probabilities are hand-authored and illustrative — they show the idea, not a real model's exact numbers.

Text so far:
Predicted next token (ranked):

Notice three things. First, the model is never "sure" — it has a distribution, and the top guess is just the tallest bar. Second, picking the top token every time is one valid strategy; nudging it to pick slightly-less-likely tokens is what temperature does (below). Third, nowhere in here did it "look up" Paris — it simply learned that Paris overwhelmingly follows that phrase in human text.

WEAKNESSWhat an LLM is not

Most disappointment with AI comes from expecting the engine to be something it isn't. Four corrections that will save you a lot of grief:

1. It does not "know" what's true. It predicts plausible text. Plausible and true overlap a lot — but not always. When the most likely-sounding continuation happens to be false, the model states it with the exact same confidence as a fact. We call this hallucination.

✗ Hallucination is not a bug you can fully patch out

A model invents a plausible-but-false citation, API, statistic, or quote because fabricating it and recalling it feel identical to a next-token predictor — both are just "likely text." It has no built-in sense of "I don't actually know this." This is why agents that act need guardrails and verification (all of Stage 3): a confident hallucination that triggers a real tool call is how you get a wrong email sent or wrong code shipped.

2. It has no memory between calls — by default. Each request is a blank slate. The model only "remembers" what you put in front of it this time. The illusion of memory in a chat is mechanical: the app re-sends the whole conversation back to the model on every turn. Stop sending it, and the model has no idea you ever spoke. Real persistent memory is something we add on top (Lesson 2.4).

3. It can't browse, run code, or act on its own. The raw model only emits text. It cannot fetch a webpage, query your database, or send a message. When an AI product "searches the web" or "books a table," that's tools wired up around the model (Lesson 2.2) — the agent layer doing the acting, with the LLM only deciding what to do.

4. It is frozen in time. A model's knowledge stops at its training cutoff — the date its training data ends. Ask about events after that and it will either say it doesn't know or, worse, confidently make something up. Anything genuinely current has to come in through tools or the prompt, not from the model's memory.

📖 Context window

The context window is the maximum amount of text — measured in tokens — the model can "see" at once: your instructions, the conversation, any documents, and the reply it's generating, all counted together. Everything outside the window simply does not exist to the model on this call. It's the model's working memory, and it has a hard ceiling. (We dig into living within it in Lesson 2.3.)

PRACTICALThe few knobs and limits worth knowing

You don't need the math. You do need a working feel for five terms that come up constantly:

TermPlain meaningWhy you care
Tokenthe word-piece unit the model reads/writes (~¾ of a word)limits and cost are measured here
Context windowhow much text it can see at once, in tokensthe model's working memory — exceed it and old text falls out
Temperaturehow adventurous the next-token pick is: low = pick the safe top token, high = roll the dice on less-likely oneslow for facts/code, higher for brainstorming and variety
Training cutoffthe date the model's knowledge endsanything newer must come from tools or your prompt
Parametersthe billions of tunable numbers learned in trainingloosely tracks capability — but bigger isn't automatically better (see below)

On temperature specifically, in plain terms: remember the bars in the demo. At temperature 0 the model always takes the tallest bar — deterministic, repeatable, boring-but-reliable. Crank temperature up and it sometimes reaches for the second or third bar, producing more varied (and more error-prone) text. It is the single dial between "give me the same correct answer every time" and "surprise me." For agents doing real work, you usually want it low.

▸ A useful mental image

Picture the model as a fluent improviser with a fixed-size desk. The desk (context window) holds only so many pages; anything pushed off the edge is forgotten. It improvises beautifully from what's on the desk, never fact-checks itself, and its general knowledge was learned long ago (training cutoff). To give it fresh facts, you put a new page on the desk — you don't change what it knows.

CONTEXTWhy this matters for every agent

Here's the payoff, and the reason this lesson sits in the Foundations stage. In the agent loop you met in Lesson 1.1 and will dissect in Lesson 2.1, the LLM is the decision-maker. At every turn of the loop, the agent shows the model the goal and what's happened so far, and the model predicts the next token — which, dressed up as a structured action, becomes "call this tool with these inputs" or "we're done."

An agent is only as good as the engine driving it. The LLM's strengths become the agent's strengths; its limits become the agent's limits — now with the ability to act on them.

So trace the consequences:

Every clever thing in the rest of this course is, at bottom, a way of compensating for what a next-token predictor can't do on its own — and amplifying what it does brilliantly. Hold the engine clearly in your head and the whole architecture of agents stops looking like magic and starts looking like sensible engineering.

WEAKNESSCommon mistakes to avoid

✗ "It understands what it's saying, like a human does."

It produces text that reads as understanding. There's no inner experience, no checking against reality, no intent — just an extraordinarily good next-token prediction shaped by human text. Useful? Immensely. Understanding in the human sense? No. Calibrate your trust accordingly.

✗ "It looks things up when it answers."

The raw model retrieves nothing. It generates from learned patterns. Any real "looking up" is a tool bolted on around it (web search, a database). If a system isn't wired with such tools, treat every fact, citation, and number as a confident guess until verified.

✗ "Bigger / newer = always smarter."

More parameters and newer models often help, but not always for your task. A smaller, faster, cheaper model with the right tools and a tight prompt routinely beats a giant one used carelessly. Capability is real; "bigger is automatically better" is marketing. Match the engine to the job.

KEY INSIGHTWhere this connects in your course

  • Chatbot vs agent (1.1) — this is the brain in LLM + tools + loop. Now you know what that brain is really doing.
  • Why now? (1.4) — the engine getting reliable enough at tool use and reasoning is exactly why agents took off when they did.
  • The agent loop (2.1) — every turn of the loop is one round of next-token prediction, reframed as a decision.
  • The context window (2.3) — the "fixed-size desk" limit we introduced here, and how to work within it.
  • Reasoning & planning (2.6) — how we coax better thinking out of a next-token predictor.

The engine is the foundation under everything. Keep the one sentence — "it predicts the next token, repeatedly" — and you'll never again be fooled into thinking the agent is smarter, or dumber, than it actually is.

GLOSSARYKey terms introduced

LLM (large language model)
An AI trained on vast text to predict the next token; the reasoning engine inside an agent.
Token
The word-piece unit a model reads and writes; ~¾ of a word in English. Limits and cost are counted in tokens.
Prediction
The model's single core act: outputting a probability distribution over the next token, then picking one.
Context window
The max amount of text (in tokens) a model can see at once — its working memory, with a hard ceiling.
Hallucination
Confident, plausible-sounding output that is false, because the model predicts likely text, not verified truth.
Training cutoff
The date a model's training data ends; it has no built-in knowledge of events after it.
Temperature
A setting controlling randomness in next-token choice: low = safe/repeatable, high = varied/creative.
Parameters
The billions of tunable numbers adjusted during training; loosely tracks (but doesn't guarantee) capability.

ACTIVE RECALLCheck your understanding

Answer in your own words first, then reveal.

1In one sentence, what does a large language model fundamentally do?›

It predicts the most likely next token given the text so far, then appends it and repeats — one token at a time. Fluent reasoning, code, and answers are all that single move done many times over.

2Why does an LLM hallucinate, and why can't you fully patch it out?›

Because it predicts plausible text, not verified truth. Fabricating a fact and recalling one feel identical to a next-token predictor — both are just "likely continuations." It has no built-in sense of "I don't know," so confident nonsense and confident truth come out the same way.

3An LLM, by itself, can't remember past chats, browse, or act. So how do products seem to do all three?›

Those are added around the model. "Memory" is the app re-sending the conversation each turn (and later, real persistent memory); "browsing" and "acting" are tools wired up by the agent layer. The raw model only emits text and only decides what to do — the tools and loop do the doing.

4Why does understanding the engine matter so much for working with agents?›

Because the LLM is the decision-maker in the loop, its strengths and limits become the agent's strengths and limits — now able to act. A hallucination becomes a wrong action; no memory means state must ride in the context window every loop; the training cutoff is why agents need tools. The whole architecture is built to amplify what the engine does well and compensate for what it can't.

You've met the brain. Next: why all of this exploded right now →