Token
Term 244 of 268 · Technology
In one sentence
A token is the smallest unit of text a language model processes: a piece of a word, a whole word or a symbol. LLMs read, generate and charge by tokens, not by words or characters.
Reviewed by Juan Manuel Garrido
Co-founder of VantegrateLinkedIn
Token definition
A token is the basic unit a language model (LLM) uses to read and produce text. It is not exactly a word or a letter: it is a fragment, which can be a whole short word ("house"), part of a long word ("inter" + "national"), a space, a number or a punctuation mark. Before processing any instruction, the model breaks the text down into tokens through a process called tokenization, and it always works on that list of units.
The token matters in practice for two reasons. First, it defines the technical limits: every model has a context window measured in tokens, which is the maximum amount of text (input plus output) it can handle in a single interaction. Second, it defines the costs: AI providers charge by the number of input and output tokens, not by words. Understanding this unit is key to designing AI agents and automations that are both accurate and cost-effective.
A rule of thumb that Google publishes for its Gemini models: 1 token is about 4 characters, and 100 tokens are about 60 to 80 English words (Google). Spanish tends to use somewhat more tokens than English because of its accented characters and longer words.
How a token is formed
A token comes out of a process called tokenization, where an algorithm (typically BPE, short for Byte Pair Encoding) splits the text into the most frequent units of a fixed vocabulary learned during training. Common words are usually a single token; rare words, proper names and technical terms are broken into several pieces. For example, "WhatsApp" may become two or three tokens, while "the" is just one. Each token is then converted into a number (a vocabulary ID), which is what the model actually processes when it computes embeddings and predicts the next token.
This explains a central trait of LLMs: they generate text token by token. The model does not write a complete answer in one go; it predicts the most likely token, adds it and predicts the next one again, repeating the cycle until it finishes. That is why answers appear to be "typing themselves" on screen, and why a long answer takes longer and costs more than a short one.
Why tokens matter for your business
For any applied AI project, the token is the economic and operational unit. There are three concrete reasons:
- Cost: a model's bill is calculated by multiplying the tokens consumed by their unit price, with different rates for input and output (output is usually more expensive). A wordy prompt or a long conversation history inflates the spend without the user noticing.
- Context limit: if the sum of input and output tokens exceeds the model's context window, the text is truncated or the call fails. This forces you to summarize, trim or use techniques such as RAG to stay within the limit.
- Latency: more output tokens mean more generation time. An agent that answers on WhatsApp with huge paragraphs feels slow; a concise one answers fast.
A concrete example
Consider a consumer goods company in Argentina that deploys a customer service agent on WhatsApp. Each conversation includes: the system instructions (about 800 fixed tokens), the history of the latest messages (another 1,200), the customer's question (50) and the generated answer (200). That adds up to about 2,250 tokens per interaction. If the agent handles 10,000 conversations a month, that is 22.5 million tokens a month. Cutting the system prompt from 800 to 400 tokens and shortening the history can cut the cost in half without losing answer quality, simply by designing with tokens in mind.
Common mistakes
The first is confusing tokens with words or characters: estimating costs on the assumption that "1 word = 1 token" underestimates the spend, since even in English 100 tokens yield only 60 to 80 words. The second is ignoring the accumulated history: in a long conversation, every turn resends everything that came before, so token consumption grows nonlinearly. The third is overusing few-shot examples: adding many examples to the prompt improves accuracy but multiplies the input tokens on every call. Well-done prompt engineering balances accuracy and token economy.
Tokens vs other units
It is common to mix up tokens with neighboring concepts. This table separates them:
| Unit | What it measures | Who uses it |
|---|---|---|
| Token | A text fragment for the model | The LLM, internally and for billing |
| Word | A unit of human language | People, when they write |
| Character | An individual letter or symbol | Text editors, field limits |
| Parameter | An internal weight of the trained model | Describes the model's size |
The practical takeaway: when you plan an AI automation, always estimate in tokens, not in words. Calculating the average tokens per interaction and projecting it to the monthly volume gives you a realistic cost and lets you optimize the design before putting the agent into production.
FAQs about Token
What is a token in artificial intelligence?
What is a token in artificial intelligence?
A token is the smallest unit of text a language model processes. It is not exactly a word or a letter but a fragment: it can be a whole short word, part of a long word, a space, a number or a punctuation mark. Models first break all the text down into tokens and then work on that list of units to read and generate answers.
How many words are in a token?
How many words are in a token?
As a rule of thumb, Google estimates for its Gemini models that one token is about 4 characters, and 100 tokens are about 60 to 80 English words. Spanish tends to consume somewhat more tokens than English because it has longer words and uses accented characters, which are sometimes split into extra pieces. Common words are usually a single token, while rare or technical ones are broken into several.
Why do AI models charge by tokens?
Why do AI models charge by tokens?
Providers charge by tokens because that is the actual unit the model processes, not human words. The cost is calculated by multiplying the number of input and output tokens by their unit price, and output is usually more expensive than input. Charging per token accurately reflects the computational work, since generating long text consumes more resources than generating short text.
What is the difference between a token and a context window?
What is the difference between a token and a context window?
The token is the unit of measure and the context window is the container measured in that unit. The context window defines how many tokens, input and output combined, the model can handle in a single interaction. If the text exceeds that limit, it is truncated or the call fails. In other words, the token is the brick and the context window is the maximum size of the building.
How can I reduce token consumption in an AI agent?
How can I reduce token consumption in an AI agent?
You do it with several techniques: shortening the system instructions without losing clarity, limiting the conversation history to recent messages, asking for concise answers, avoiding adding too many examples to the prompt, and using retrieval (RAG) instead of loading entire documents into the context. Measuring the average tokens per interaction and projecting it to the monthly volume helps you spot where you are overspending.
When is the token the wrong metric for making a decision?
When is the token the wrong metric for making a decision?
When what you are evaluating is whether the use case is worth it, not how to optimize it. Token spend is usually marginal compared with the cost of a bad answer: trimming context to save tokens leads to retries, escalations to a person on the team and errors that end up costing far more. Tokens do not measure quality or effort either, because two answers of the same length can be worth very different things. Measure resolution and satisfaction first, and only then consumption.
An AI agent that already knows how to do this
We implement AI agents on the CRM you already use, for sales, collections and support. Tell us which process eats your day and we will tell you straight whether an agent solves it.
Related terms
- Context WindowThe context window is the maximum amount of information (measured in tokens) that a language model can process at once, adding up the user's question, the instructions, the attached data and the response it generates.
- LLM (Large Language Model)An LLM (large language model) is an artificial intelligence system trained on huge volumes of text that predicts and generates natural language. It is the engine behind chatbots, automated writing and AI agents that can understand and respond in human language.
- EmbeddingsEmbeddings are numerical representations (vectors) that encode the meaning of a text, image or data point, so that similar concepts end up close together in space. They let AI measure semantic similarity and search by meaning, not by exact words.
- Prompt EngineeringPrompt engineering is the practice of designing, structuring and refining the instructions given to a generative AI model so it returns accurate, useful and reliable answers, by adjusting the context, format, examples and constraints.
- Tool CallingTool calling is a language model's ability to invoke external functions (query a database, call an API or run an action) so it can operate on real systems instead of only generating text. It turns a model that talks into an agent that acts.
- WorkflowA workflow is the ordered sequence of steps, rules and approvals that defines how a business process moves from start to finish, automating repetitive tasks so they run on their own based on predefined conditions.
Related questions
AI Agents
What AI agents are when applied to sales, collections and support, and how they are implemented on the CRM you already use.
How AI Agents solve itFive AI products that work on top of the CRM you already use. They don't replace your system: they add the layer you do by hand today.
