Instant (no download)
Exact (GPT-2)
GPT-4o / o1 · GPT-4o, GPT-4o-mini, o1
Ejemplos:
Text & documents
Numbers & maths
Data & formats
Security
Development & DevOps
Artificial Intelligence
Finance
Health & Wellness
Productivity
Games & Entertainment
Multimedia & design
Business
How to use
What a token is and why count them
A language model doesn't read letters or words: it first splits the text into tokens, small pieces (a short word, part of a long word, a symbol, a space…). As a rough rule, in English 1 token ≈ 4 characters ≈ ¾ of a word; in Spanish and in code there are usually somewhat more tokens per word.
Counting tokens matters for two reasons:
• Context window — each model accepts a maximum number of tokens (input + output). If you go over, the text is cut off or the request fails.
• Cost — APIs charge per token, not per word. Knowing how many there are tells you how much you'll pay and whether it fits.
How to use it (step by step)
1. Pick the mode at the top: Instant (default) or Exact.
2. In Instant, choose the encoding / model in the dropdown.
3. Type or paste your text into “Text to tokenize” (or click an Example: Short, Paragraph or Code).
4. The count updates on its own once you stop typing; at the top you see Tokens, Characters and Chars / token.
5. Below, the Visualization tab shows the split and Costs and context estimates the price per model. Everything happens in your browser.
Instant vs Exact mode
There are two ways to split, both BPE (Byte Pair Encoding) and 100% local:
• Instant (default) — uses the gpt-tokenizer library with OpenAI's real encodings. It downloads nothing and responds instantly. It's the exact count for GPT/o1 models and the one you should use almost always.
• Exact (GPT-2) — downloads the real GPT-2 tokenizer (~5 MB from Hugging Face, cached the first time) and tokenizes with it. Useful only if you work specifically with GPT-2.
Neither is the tokenizer of Claude or Gemini (those families use others): for them the number is a close estimate, not exact.
Encodings and which model they map to
The Instant-mode dropdown chooses which encoding is used to count. Each is shared by several models and has its own context window:
• o200k_base — GPT-4o / o1 — the newest; used by GPT-4o, GPT-4o-mini and o1. 128K-token context. It's the default option.
• cl100k_base — GPT-4 / GPT-3.5 — for GPT-4, GPT-3.5 Turbo and the embeddings models. 128K context.
• p50k_base — Codex / davinci-002 — older models: Codex and text-davinci-002/003. 8K context.
• r50k_base — GPT-3 / GPT-2 — the oldest: davinci (GPT-3) and GPT-2. 2K context.
Pick the one that matches your model: the same text gives a slightly different token count depending on the encoding.
How to read the result
Above the text you'll see three figures:
• Tokens — the count that really matters for context and cost. It's the real number even if the visualization is trimmed.
• Characters — length of the text in characters (includes spaces and line breaks).
• Chars / token — average characters per token. The higher it is, the more “efficient” the text (fewer tokens per letter); code and languages with many accents tend to lower this figure.
Visualizing the split
Each colored box is one token; the colors just alternate to tell the boundaries apart. Notice how a long word breaks into several pieces and how spaces usually stick to the start of the next token. To avoid overloading the browser, the view shows at most the first 500 tokens; if your text is longer a notice appears, but the count and cost are for the full text.
Privacy
All the splitting and counting happen in your browser: the text never leaves your machine. In Instant mode nothing is downloaded; in Exact mode only the GPT-2 tokenizer (~5 MB) is fetched once and then cached. You can safely paste prompts or snippets containing internal data.
