Instant (no download)
Exact (download)
GPT-4o / o1 · GPT-4o, GPT-4o-mini, o1
Ejemplos:
How to use
What a token is and why count them
A language model doesn't read letters or words: it first splits the text into tokens, small pieces (a short word, part of a long word, a symbol, a space…). As a rough rule, in English 1 token ≈ 4 characters ≈ ¾ of a word; in Spanish and in code there are usually somewhat more tokens per word.
Counting tokens matters for two reasons:
• Context window — each model accepts a maximum number of tokens (input + output). If you go over, the text is cut off or the request fails.
• Cost — APIs charge per token, not per word. Knowing how many there are tells you how much you'll pay and whether it fits.
How to use it (step by step)
1. Pick the mode at the top: Instant (default) or Exact.
2. In Instant, choose the encoding / model in the dropdown.
3. Type or paste your text into “Text to tokenize” (or click an Example: Short, Paragraph or Code).
4. The count updates on its own once you stop typing; at the top you see Tokens, Characters and Chars / token.
5. Below, the Visualization tab shows the split and Costs and context estimates the price per model. Everything happens in your browser.
Instant vs Exact mode
There are two ways to split, both BPE (Byte Pair Encoding) and 100% local:
• Instant (default) — uses the gpt-tokenizer library with OpenAI's real encodings. It downloads nothing and answers right away. It is the exact count for OpenAI models.
• Exact (download) — fetches the real tokenizer of GPT-2, Qwen3 (multilingual) or SmolLM2 (mostly English): only its vocabulary, 2 to 10 MB, once and cached. It is for comparing: the same Spanish text splits into many more pieces with a tokenizer built for English.
None of them is Claude's or Gemini's, which are not public: for those the number is an estimate, not exact.
Encodings and which model they map to
The Instant-mode dropdown chooses which encoding is used to count. Each is shared by several models and has its own context window:
• o200k_base — GPT-4o / o1 — the newest; used by GPT-4o, GPT-4o-mini and o1. 128K-token context. It's the default option.
• cl100k_base — GPT-4 / GPT-3.5 — for GPT-4, GPT-3.5 Turbo and the embeddings models. 128K context.
• p50k_base — Codex / davinci-002 — older models: Codex and text-davinci-002/003. 8K context.
• r50k_base — GPT-3 / GPT-2 — the oldest: davinci (GPT-3) and GPT-2. 2K context.
Pick the one that matches your model: the same text gives a slightly different token count depending on the encoding.
How to read the result
Above the text you'll see three figures:
• Tokens — the count that really matters for context and cost. It's the real number even if the visualization is trimmed.
• Characters — length of the text in characters (includes spaces and line breaks).
• Chars / token — average characters per token. The higher it is, the more “efficient” the text (fewer tokens per letter); code and languages with many accents tend to lower this figure.
Visualizing the split
Each colored box is one token; the colors just alternate to tell the boundaries apart. Notice how a long word breaks into several pieces and how spaces usually stick to the start of the next token. To avoid overloading the browser, the view shows at most the first 500 tokens; if your text is longer a notice appears, but the count and cost are for the full text.
Privacy
All splitting and counting happen in your browser: the text never leaves your computer. Instant mode downloads nothing; Exact mode downloads the vocabulary of the tokenizer you choose once, and it stays cached. You can safely paste prompts or snippets with internal data.
