What Is Token?
A token is the basic unit of text that LLMs process. One token is roughly 3/4 of a word in English.
When you send text to an LLM API, the text is broken into tokens before processing. A token can be a whole word, part of a word, or even a single character. In English, one token averages about 4 characters or 3/4 of a word. The sentence 'Hello, how are you?' is about 6 tokens.
Tokens matter because LLM pricing is based on them. When a provider charges '$5 per million tokens,' they mean the total tokens in your input (what you send) plus the tokens in the output (what the model generates). Input tokens are typically cheaper than output tokens because the model does more computational work generating responses.
Context windows are also measured in tokens. A model with a '1M token context window' can process roughly 750,000 words of input in a single request. Larger context windows cost more per request but let you process longer documents without splitting them up.