TL;DR: A token is roughly three-quarters of a word. Models charge separately for tokens you send in and tokens they send back, and every message in a chat resends the whole conversation. That last fact is why long chats get expensive.
What a token actually is
Language models do not read letters or words. They read tokens — chunks of text produced by a splitting algorithm. Common words are usually one token. Longer or unusual words get split: “unbelievable” might become “un”, “believ”, “able”. Punctuation and spaces count too.
The rough conversion everyone uses in English: 1 token ≈ 0.75 words, or about 4 characters. So 1,000 tokens is roughly 750 words, and a 2,000-word article is about 2,700 tokens.
Why you are billed twice
Every API call has two meters:
- Input tokens — everything you send: your question, your system prompt, any pasted document, and the entire prior conversation.
- Output tokens — what the model generates back. Almost always priced higher than input, often several times higher, because generating is more expensive than reading.
The thing that surprises people
Chat models are stateless. They do not “remember” your conversation — the application resends the whole history with every new message. So a 40-message conversation is not 40 small requests. Message 40 pays for messages 1-39 all over again.
This is why a long chat about a 50-page PDF can cost more than you expect: that PDF is being re-sent, and re-charged, on every turn. It is also why the size of a model’s context window matters — the window is the ceiling on how much can be resent at once.
Practical consequences
- Start new chats. When the topic changes, open a fresh conversation. You are dropping the accumulated input cost.
- Do not paste what you do not need. Pasting an entire report to ask about one table charges you for the whole report, every turn.
- Ask for the length you want. “In under 150 words” is a direct cost control on the expensive meter.
- Summarise and restart. For long projects, ask for a summary of the conversation, start a new chat, and paste the summary. This is the single biggest saving available.
Subscriptions vs API
Flat-rate subscriptions (ChatGPT Plus, Claude Pro) hide all of this behind usage limits — you feel tokens as “you have hit your cap,” not as a bill. API access exposes the real meter. Most people should stay on subscriptions until they are automating something; the crossover point is roughly when a workflow runs unattended. The subscription-versus-usage question is the whole subject of my AI spend audit.
Other languages cost more
Tokenizers were optimised for English. The same sentence in Arabic, Hindi or Thai can use two to three times as many tokens as its English equivalent — meaning the same task genuinely costs more. It is a real and under-discussed inequity in AI pricing, and worth knowing if you work across languages.
If you want the next layer down on how models handle what you give them, the RAG explainer covers how documents get in without paying to resend them every time.
About the author
Shahid Saleem is the founder and editor of PickGearLab. He tests AI tools in the real world – writing, automation, content – and writes up what actually worked. Based in Dubai.
One practical AI tutorial. Every Monday.
Workflows like this one — straight to your inbox. Free. Unsubscribe in one click.
Subscribe free →
