Why ChatGPT Answers Get Worse in Long Conversations — and When to Start a New Chat

Long conversations quietly erode the quality of AI responses. Picture this: you're citing a client quote and change the numbers three times along the way. By message forty, the old prices are still lingering in the chat history, and the model may blend old and new figures together. Errors like this often only surface when a wrong quote lands in a client's inbox.

The problem isn't that you need a stronger model or a better tool — it's knowing when to continue the current conversation and when to start over.

In the Same Conversation, Context Is a Key Asset

Continuing the same conversation isn't a mistake — provided the task and the underlying facts haven't changed.

If you're still working on the same task with the same set of facts, keeping the context is a win. The model remembers the constraints, format preferences, and background you gave it earlier, so you don't have to repeat yourself.

Another case where continuing makes sense is fine-tuning edits. Maybe you just want to shorten a paragraph or make the tone friendlier. Requests like these don't require rebuilding the factual foundation — just ask directly in the existing conversation.

The key criterion is: has the set of facts changed? If the facts haven't changed and the task hasn't changed, continuing the conversation is efficient.

When You Must Start a New Conversation

There are two clear signals that you should start fresh.

First, a topic switch. When you move from discussing one client to another, information from the old conversation becomes noise. The model may blend the previous client's quotes, terms, or background into answers about the new client.

Second — and more insidious — the answers start repeating themselves, contradicting earlier statements, or forgetting facts you explicitly provided. These are classic symptoms of context overload. The model's attention is diluted by a large volume of historical messages, and early key information gradually gets "drowned out."

Once these signs appear, correcting the model within the old conversation rarely helps, because the problem lies in the context itself, not in any single response.

Context overload refers to the point where accumulated conversation history dilutes the model's attention mechanism with a large number of historical tokens, causing the weight of early key information to decline. The Transformer architecture's attention computation can theoretically access the full context, but when the sequence grows too long, the attention resources allocated to each token get spread thin, and the model becomes more easily dominated by recent messages — leading to forgetting, repetition, or self-contradiction. This has nothing to do with the model's parameter capacity; it's the result of information competition within the context window. Even if the context window is large enough to hold all messages, the ability to effectively retrieve and use key information declines as length increases — which is why simply correcting within an old conversation often fails.

How to Cleanly Start a New Conversation

Starting over doesn't mean losing useful information. A simple three-step routine preserves the core facts:

Step one: have the model do a fact recap. Use an instruction like: "Summarize the facts we've agreed on in five bullet points. Only include what I've said." The value of this step is that it forces the model to compress the key information scattered across a long conversation into a single list.

Step two: verify the recap yourself. Check the numbers especially. Models are good at finding facts but less reliable at arithmetic. If a wrong number shows up in the recap, fix it manually.

Step three: open a new conversation, paste the recap at the top, and make your new request. This way the new conversation starts from a human-verified fact summary — avoiding context contamination while preserving essential information.

Fact recap works because it compresses the key constraints scattered across dozens of messages into a dense, short list — essentially manually rebuilding a clean, lean context starting point. Requiring the model to "only include what I've said" curbs its tendency to infer or fill in facts on its own, reducing hallucinations. Meanwhile, limiting the recap to five bullet points ensures the new conversation's initial context is short enough to avoid rapid overload again. The manual number-checking step cannot be skipped, because the model may preserve wrong numbers or introduce new arithmetic deviations during compression — and once a new conversation starts with wrong facts, all subsequent answers inherit that error.

You Must Do the Math Yourself

One piece of advice from the original deserves its own emphasis: models are strong at finding facts but weak at arithmetic reliability.

This means any total, summary, or calculation you're about to send out should be re-added yourself, even if the conversation looks perfectly fine. When numbers are revised repeatedly in a long conversation, the probability of error rises further.

The cost of this rule is low, but it prevents wrong numbers from reaching clients or colleagues.

The root of weak arithmetic reliability is that large language models fundamentally generate tokens based on probability rather than performing precise arithmetic operations. When numbers are revised multiple times in a long conversation, the model must simultaneously track multiple versions of the numbers and their relationships, further increasing the chance of error. Even if the model performs acceptably on a single simple addition, once multi-step calculations, unit conversions, or cross-paragraph summaries are involved, the error rate rises significantly. Therefore, any total, summary, or calculation intended for external output should be treated as a draft requiring human review — not a conclusion to be trusted as-is. The marginal cost of this habit is extremely low, but it effectively blocks wrong numbers from propagating downstream.

The Core Decision Logic

Simplify long-conversation management into a single question: Have the facts changed?

If the facts haven't changed and the task hasn't changed, keep using the same conversation and enjoy the efficiency that accumulated context brings. If the facts have changed, the topic has shifted, or the model starts forgetting and contradicting itself, decisively start a new conversation and use the three-step method to preserve the core information.

The model is not a memory bank, and the context window is not infinitely reliable storage. Managing conversations as tools with lifecycles is more realistic than expecting them to stay accurate forever.

One-line Summary

Long conversations degrade answer quality through context overload — judging whether the facts have changed and restarting at the right time is the key.

Paragraph Summary

In long conversations, context overload causes models to forget facts, repeat themselves, or contradict earlier statements — but the fix isn't a stronger model, it's judging when to continue and when to restart. When facts and tasks are unchanged, continuing the same conversation offers the efficiency of accumulated context; once the topic shifts or signs of forgetting and contradiction appear, start a new conversation decisively. The clean-restart three-step method: have the model summarize the key facts, manually verify the numbers, and paste the verified recap at the top of the new conversation. Also, models are strong at finding facts but weak at arithmetic, so key numbers must be recalculated yourself. Managing conversations as tools with lifecycles is more realistic than expecting them to stay accurate forever.