Key takeaways
- The context window is the model's working space for one exchange, counted in tokens.
- Everything competes for it: instructions, history, documents and the reply.
- Past the limit, earlier material is cut or condensed, and the model can lose details.
- Long-term memory is a separate layer that products add around the model.
What are tokens in AI
Tokens are the small units of text a language model reads and writes: whole short words, parts of longer words, numbers and punctuation.
A model does not see letters or pages; it sees a sequence of tokens. Context windows, usage limits and costs are all counted in them, which is why a long spreadsheet pasted as text can take far more of the window than it looks.
What happens when the context window fills up
When the context window fills up, older or less relevant material is dropped or condensed so that new text fits.
Products handle it differently: some cut the start of the conversation, some replace it with a summary, some refuse more input. Either way, the model answers only from what is still in the window. Even inside the window, a detail buried in a very long document can be overlooked.
Context window vs memory
The context window is short-term working space, while memory is information a product saves and brings back into later conversations.
A model itself remembers nothing between conversations; once a chat ends, its window is gone. Products add memory around the model: they save chosen facts, or fetch relevant documents through retrieval, and place them in the window when needed. That is how a tool can recall your positioning without you pasting it each time, and also why what it recalls depends on what was saved.
Why a model forgets earlier material in a long chat
A model forgets earlier material in a long chat because that material no longer fits, or no longer fits in full, in its context window.
In an illustrative case, a seller pastes a full RFP, three call transcripts and a pricing sheet into one chat, then asks a question about the first call. The answer misses what the buyer said, because the first transcript was cut or condensed to make room. A few habits avoid this.
- Give the model only what the task needs, not every file you have.
- Start a new chat for a new task, with a short summary of what matters.
- Put key facts, such as the buyer's name and the deadline, in your request rather than deep in a document.
- Check answers that depend on early material against the source.
Where Pebbles Ai fits
- FoundationAgent memoryAgent memory saves short facts you state or confirm at three levels, Your memory, Project memory and Company memory, and the assistants recall them in later chats.
- FoundationCompany IntelCompany Intel holds your company's business knowledge and brand voice for the assistants to draw on in any chat.
Common questions
Is a bigger context window always better?
Not always. A larger window lets you include more material, but answers can still miss details in long inputs, and longer inputs usually take more time and more usage.
What is context length?
Context length is another name for the size of the context window: the most tokens a model can handle in one exchange.
Does an AI assistant remember what I said last week?
Only if the product saves it. The model's context window starts empty with each new conversation, so recall across chats depends on a memory feature.
About this page. Written by the Pebbles Ai team for people who run sales, marketing and growth. Last updated .
