Have you ever wondered how ChatGPT understands and processes the instructions you provide? An important aspect of this is tokens.
ChatGPT has won the hearts of society and there are many who really try to make the most of this tool, bringing out all its advantages. However, a really important part of his personality lies in the tokens.
Explain to you that these are word fragments that are generated when ChatGPT processes the input. They are the basic unit that these OpenAI models use to calculate the length of a text. They are groups of characters, which sometimes align with words, but not always.
Specifically, it depends on the number of characters and includes punctuation marks or emojis. This is why the token count is often different from the word count.
They can include trailing spaces and even subwords, and are language dependent. For example, Spanish words tend to have a higher token-to-character ratio, making the API more expensive to implement for languages other than English.
To give you an idea of how tokens work, here are some general rules of thumb so you can finally understand it:
1 token — 4 English characters or almost a word 100 tokens— 75 words 1 or 2 sentences — 30 tokens 1 paragraph — 100 tokens 1,500 words — 2,048 tokens— 5.4 pages 3,000 words — 4,096 tokens— 10.8 pages 6,000 word s — 8,192 — 21.6 pages
The ChatGPT model GPT-3.5 could handle 4,096 tokens or about 8,000 words, but GPT-4 beat those numbers up to 8,192 tokens and the 32,768 tokens is, for now, only offered to a select few test users.
What happens if you go over the limit?
Imagine that you are using the free version of ChatGPT 3.5 with the limit of 4,096 tokens. If you enter a message that’s 4,000 tokens—about 3,000 words—that leaves your result — ChatGPT’s ‘completion’ or response tokens — with only 96 tokens left to use.
In general, there are two outcomes that happen if you exceed the token limit. The main one is simply an error message, telling you that you have exceeded the length limit and that you should try again. At this point the solution is simple. Copy and paste what you wanted to type as a message, go to another editor and reduce the size, then start a new chat and you’re done.
Another possible outcome is that it will simply start ignoring some of your previous explanations. This means that at some point you will start to “forget” the initial conversation. The problem? It will be quite difficult for you to know when this happens, since no warning message will appear.
But don’t worry too much about the total token limit, there are often creative ways to work within the limit, such as condensing the request or even breaking the text into smaller parts. In the end it is based on getting the most out of ChatGPT with what you have and well, there are not a few tokens that they offer starting.
