AI Token Calculator

What an LLM Actually Counts, and Why It's Not Words

Large language models don't process text word by word — they process it as tokens, sub-word chunks produced by a byte-pair-encoding style tokenizer. A word like "calculator" might be one token, while a rarer word or a piece of code might split into three or four. Because the exact split depends on the specific tokenizer a model uses (tiktoken for OpenAI, SentencePiece-style tokenizers elsewhere), there's no universal exact count without running the actual tokenizer — but a well-established rule of thumb gets close enough for planning purposes.

The Formula

This calculator averages two independent estimates, both standard approximations for English text:

The 0.75 figure is the flip side of the commonly cited "100 tokens ≈ 75 words" rule used in OpenAI and Anthropic documentation for rough English-text estimates.

Where This Calculation Matters

• Budgeting API usage before writing code — getting a ballpark token count for a prompt or document before wiring up an actual API call and tokenizer library.
• Context window planning — checking whether a document will fit inside a model's context limit before attempting to send it.
• Cost estimation — converting a block of text into an approximate token count as the first step toward estimating API cost with a per-model pricing calculator.
• Prompt engineering — getting quick feedback on how "expensive" (in tokens) a prompt template is as it's being drafted.

Rule-of-Thumb Word-to-Token Scaling

This is an estimate for typical English prose. Code, non-English text, and text with unusual formatting or many rare words can tokenize quite differently from this ratio.

How to Use This Calculator

1. Paste your text into the input box.
2. Select Calculate to get an estimated token count, along with the underlying character and word counts.

Related Calculations

Once you have a token estimate, price it out with the OpenAI Token Cost Calculator or the Claude Token Cost Calculator.