Tokens
Updated 2026-08-04
Questions this answers
- What's a token, and why does everything get measured in them?
- Why am I billed per token instead of per word?
- What does "context window" actually mean?
- Why does the AI forget things from earlier in a long chat?
The Fix
A token is the chunk of text a model reads and writes in. Roughly, one token is about four characters, or three-quarters of a word, so "unbelievable" might split into un / believe / able. Everything the model sees gets broken into tokens first, because underneath it works in numbers rather than letters. Both what you send and what it writes back get counted.
Providers bill per token, input and output, so a long document you paste in runs up more than a quick question. And every model has a context window, the most tokens it can hold at once, from about 128,000 on older models to a million on the newest. When a chat runs past that window, the oldest tokens drop out of view, which is why a long conversation starts forgetting what you told it near the start.
When to Use It
Tokens matter the moment cost or length enters the picture. If an AI bill looks high, it's counting tokens, so trimming what you paste in and asking for shorter answers brings it down. If a model "forgets" earlier instructions in a long session, you've likely run past its context window, and the fix is a fresh chat or a quick recap of what matters.
You don't need to count them by hand. It's enough to have a feel for it: more text in and out means more tokens, which means more money and more pressure on the window. When you're doing something long or repetitive, that instinct is what tells you to start fresh or tighten things up.