AI at work · course 33 of 105

An AI Chatbot Under the Hood

See the working parts of a chatbot as numbers you can check: counting tokens, how temperature reshapes the next-word odds, how a context window fills up, and the words for a hallucination.

4 modules

Try a question

Not on sale yet. When sales open: $4.99 for this course, paid once, or every course for $9.99 a month. Prices

Preview

What it teaches

The working parts of a chatbot, as numbers and quoted definitions you can check. What a token is and how to count tokens by rule (with one tokenizer, tiktoken’s o200k_base); the odds a model gives its next-word candidates (softmax) and what temperature does to them, worked out to the percent; how a context window adds up (system prompt, tool definitions, the chat so far, your new message and the reply you want room for) and what drops out when it is full; and the words for a hallucination (a definition test, and the three groups of citations in a study of 115 references from ChatGPT-3.5). The AI asks 3 questions, places you, shows a short card per module, then gives you one question at a time.

Not a prompt-writing course, and not a fact-checking course: it teaches the vocabulary of hallucination, and it can’t tell whether a real AI answer is true.

How long

33 items (8 concept questions, then 8 token, 8 probability and 9 context-and-hallucination items). About 12-30 rounds for a confident student up to about 45-60 for a beginner, plus a 12-item final: 1-2 sessions (estimate, not timed with a real student; simulated students took 14 rounds as a confident student and 51 as a beginner). The tutor gives a one-line save code at the end of each session.

From the answer key · M1, row T1

How many tokens is this text? “Thank you for your help with the project.”

Think first, then show the answer

Answer9

WorkingThe o200k_base pieces: “Thank” “ you” “ for” “ your” “ help” “ with” “ the” “ project” “.” = 9.

The wrong answer it catches8 → counted the words only; the full stop is a token too.

Counting rules for plain English (not a law for all text): each ordinary word, with the space in front of it, is one token; each punctuation mark is one token; a number is cut from the left into groups of up to 3 digits, one token per group, and a space right in front of a number is a token of its own. Examples: “Hello” “,” “ world” “!”; “I” “ have” “ ” “3” “ cats” “.”; “202” “6”.

The modules

4 modules, drilled one item at a time.

M0
The words
The words: token, context window, next-token prediction, temperature, softmax, hallucination, and why one tokenizer’s count isn’t every model’s
M1
Tokens
Count the tokens in a short text by the card’s rules, and estimate tokens from bytes
M2
Probabilities and temperature
Turn a model’s scores into percent odds, and see what temperature does to them
M3
Context windows and hallucinations
Add up a context window and see what drops out, and label the kinds of hallucination

Before it teaches, it asks

1
What do you use AI chatbots for most (writing, work, code, nothing yet), and have you ever hit a length limit or met the word 'token'?
2
How many tokens is this text? “What is the capital of France?”
3
A model’s context window is 8,000 tokens. In this request the system prompt is 600 tokens, the conversation so far is 5,900 tokens and your new message is 700 tokens. You want room for a reply of up to 1,000 tokens. Is the total over or under the window, and by how many tokens?

What’s exact, and what isn’t

Show

Straight from the course’s own notes.

Token counts (M1): The keys of the six counting items are counts from tiktoken 0.14.0's o200k_base encoding (the one tiktoken maps gpt-4o to, checked by script), computed by script and also checked against the card’s counting rules so a student can reach each by hand. This is one tokenizer: other models count differently. Anthropic’s glossary (read 2026-10-13) says its newer models “use a newer tokenizer that produces approximately 30 percent more tokens for the same text than earlier models”. The practice texts are plain English with numbers and punctuation; rare words, other languages, code and emoji are not drilled. The two estimate items use the tiktoken README’s “about 4 bytes” per token (read 2026-10-13) as plain arithmetic, not as a count.
Probabilities (M2): The numeric keys are computed by script in 60-digit decimal arithmetic, rounded half-up to a whole percent; the script refuses any item whose answer lies within 0.05 percentage points of a rounding edge, and the same code reproduces both numeric examples on Wikipedia’s softmax page (read 2026-10-13). “Greedy” and “sampled” are this course’s names for always taking the top-scored word and for drawing a word at random with the probabilities. Anthropic’s glossary notes that even temperature 0 isn’t fully deterministic in its API (quoted on the card).
Context windows (M3): The window sizes and message sizes in the questions are made up for practice; no real model’s window is claimed. What counts toward a window is from Anthropic’s context-window page (read 2026-10-13). The drop items apply the “first in, first out” idea from that page’s note on chat interfaces as an exercise rule; real apps differ.
Hallucination words (M3): The definition test is Google’s glossary definition (CC BY 4.0, read 2026-10-13); the citation items use the three groups in the study sentence quoted on Wikipedia’s page (read 2026-10-13). The scenarios are made up, and each question states the facts that decide the label. The thinnest-keyed part: the study sentence names the “fabricated” group without defining it, so those questions say plainly that no such reference exists.
Honest limits: 36 quotes are checked word for word on saved pages by a script that does not re-fetch the live pages. It is not connected to OpenAI, Anthropic or Google. The course can’t judge a real AI answer.
Not in it yet: Other tokenizers and languages, top-p and top-k, how models are trained, retrieval and embeddings, prices per token, and how to prevent or fix hallucinations.