A token in AI is the unit a language model reads and writes. Models do not process text word by word; they break it into tokens, roughly word fragments. A short English word is often a single token, while longer or rarer words split into several. When people ask what are tokens in AI, the short answer is: the pieces the model actually sees. Everything a model takes in and everything it produces is counted in these units.
Tokens matter because of the context window, the fixed amount of text a model can consider at once. Context windows are measured in tokens because tokens are the model's native unit. Everything competing for the model's attention, the instructions it was given, the conversation so far, and any documents it retrieved, must fit inside that window together. That is the practical meaning of AI tokens: the budget a model spends on paying attention.
For a CX buyer, this is more than trivia. A support conversation is token-hungry. A long email thread, the knowledge articles retrieved to answer it, and the customer's account and order context all consume the same window. When that budget fills with the wrong material, answers degrade. This is why what the system chooses to retrieve matters more than raw window size: a model with an enormous window fed stale or irrelevant content still answers badly, while a model given a compact, correct picture of this customer and this policy answers well. AI tokens, explained properly, are less about capacity and more about curation.
What consumes tokens in a support conversation
| Source | What it contains | Why it grows |
|---|---|---|
| Thread history | every prior message in the conversation | long threads, repeated back-and-forth |
| Retrieved knowledge | help-center articles, policies, macros | broad or unfocused retrieval |
| Customer and order context | account data, order state, past tickets | rich integrations, long histories |
| Instructions | what the system is told to do and not do | complex rules and edge cases |
Aide, the agentic AI platform for customer experience, treats the context window as something to spend deliberately. Rather than stuffing the window with everything available, it grounds each answer in the right retrieved context: the specific policy, the live order state, the relevant slice of history for the request at hand. The result is an answer based on what is true for this customer, not on whatever happened to fit. See what Aide is for how that works in practice.
Frequently asked questions
- How many tokens is a word?
- As a rule of thumb, one token is about three quarters of an English word, so 100 tokens is roughly 75 words. Short common words are often one token each; long, rare, or technical words split into several.
- Why do AI tokens matter for customer service?
- Because support conversations are long and context-heavy. Thread history, retrieved knowledge, and customer data all share one token budget, so answer quality depends on what the system puts in the window, not just how large the window is.