Can AI “Forget”? How Large Language Models Struggle With Negation

Large Language Models (LLMs), like ChatGPT and LLaMA, seem to possess incredible memory—storing vast amounts of information and retrieving it almost instantly. But what happens when they encounter negation, like “The cat is not black”? Do they process this correctly—or, like humans, are they prone to forgetting details when negation is involved?

A new study digs into this cognitive quirk, looking at whether LLMs exhibit Negation-Induced Forgetting (NIF), a well-documented phenomenon in human memory. If AI models “forget” information in a way similar to humans, what does that mean for their reliability? Let’s break it down.

What Is Negation-Induced Forgetting?

Imagine you’re watching a crime show and someone asks, “Did the suspect have a red car?” You correctly reply, “No, it was blue.” Later, when asked to recall details about the car, you struggle. It’s as if negating the incorrect information (red) caused your brain to store the whole event less effectively.

This is the essence of Negation-Induced Forgetting (NIF)—when negating an incorrect statement leads to poorer memory recall compared to confirming a correct statement. Studies show that people are more likely to forget details after responding with no than when affirming facts with yes.

Understanding if LLMs also experience NIF can offer insights into how they handle language-based memory tasks—and whether they might be prone to similar cognitive biases as humans.

Putting AI to the Test

A group of researchers set out to investigate whether popular AI models, including ChatGPT-3.5, GPT-4o-mini, and LLaMA-3-70B, exhibit NIF. They adapted an experiment originally designed for humans, where participants read a short story and were later asked verification questions like:

• “Did Montse drink coffee?”
• If the answer was yes, the participant affirmed a fact.
• If the answer was no, they negated incorrect information.

Then, participants were asked to recall the story as accurately as possible. The researchers performed the same test with the AI models, treating each chat session as a unique “participant.” Here’s what they found:

Some AI Models “Forget” Like Humans

The study uncovered some surprising results:

• ChatGPT-3.5 showed clear signs of NIF. It was notably worse at recalling details that had been presented in negated statements.
• GPT-4o-mini displayed a similar effect, but it was weaker.
• LLaMA-3-70B did not show NIF at all. It seemed unaffected by negation and recalled information consistently.

This means that some AI models genuinely struggle to correctly recall negated information—just like humans do! But why does this happen?

Why Would AI “Forget” Information?

Unlike our memory, which relies on brain structures like the hippocampus, LLMs use statistical patterns to process language. This means that negation could affect them in different ways:

1. Weaker Attention to Negated Information

LLMs use attention mechanisms to prioritize important parts of a text. When negation is introduced, it may shift the model’s focus away from the key detail, making it less likely to be recalled later.

2. Training Biases

Most AI models are trained on vast amounts of internet data. If negated facts are processed differently or appear less frequently, the model might have learned patterns that lead to weaker memory recall when negation is involved.

3. Compression of Information

To handle vast amounts of text, LLMs sometimes “compress” details, storing only the most essential parts. If a negated concept is seen as less informative, the model might deprioritize it, leading to unintentional forgetting.

Why This Matters

Negation is crucial in many real-world scenarios. If AI struggles with negation, it could lead to misinformation and faulty decision-making in critical applications, such as:

• Medical AI: If a chatbot forgets that a patient doesn’t have a certain allergy, it could recommend the wrong medication.
• Legal AI: If an AI summarizing court documents struggles with negating false claims, it might misrepresent crucial case details.
• Search & Research AI: If an AI cannot correctly process “This study did not find a link between X and Y,” it might spread false scientific conclusions.

Ensuring that AI models handle negation correctly is essential for their reliability in knowledge-driven tasks.

What Can Be Done to Improve AI’s Memory?

If some LLMs forget negated information, how can we fix this? Possible solutions include:

1. Better Training Data
   By exposing LLMs to more negation-based reasoning tasks, researchers can help models learn to correctly handle negation and recall details more accurately.

2. Improved Attention Mechanisms
   Modifying how LLMs allocate attention when processing negation may reduce their tendency to deprioritize negated facts.

3. Explicit Reinforcement of Negation
   Prompting techniques that remind an AI model to pay special attention to negation when summarizing or recalling facts may help minimize errors.

4. User Prompt Adjustments
   For everyday users, being very explicit in prompts can reduce errors. Instead of asking, “Did X happen?”, reframe it as, “Clearly state what happened and clarify what didn’t.”

Key Takeaways

🔹 AI can ‘forget’ negated information, similar to humans.

• ChatGPT-3.5 showed the strongest evidence of Negation-Induced Forgetting (NIF).
• GPT-4o-mini exhibited weaker signs of NIF.
• LLaMA-3-70B was unaffected by NIF, recalling both affirmed and negated information equally well.

🔹 Why does this happen?

• LLMs allocate different attention weights to negated statements.
• Statistical training patterns might treat negation differently.
• Compression mechanisms may deprioritize negated information.

🔹 Why does this matter?

• AI misinterpreting negation could lead to misinformation in medicine, law, and research.
• Reliable AI requires proper handling of negated facts.

🔹 How can we improve AI memory?

1. Optimizing training datasets with more negation-based examples.
2. Adjusting AI attention mechanisms to focus on negation more consistently.
3. Users can improve their prompts by explicitly asking AI to clarify what is and isn’t true.

Conclusion

As AI becomes a bigger part of our daily lives, understanding its memory quirks is key. This study shows that, much like humans, AI isn’t perfect at recall—especially when negation is involved.

While models like ChatGPT-3.5 struggle with Negation-Induced Forgetting (NIF), newer models seem to be improving. Future research will likely focus on making AI less forgetful when faced with negation—which is great news for anyone relying on AI for fact-checking, research, or decision-making.

Until then, if you’re using AI for critical tasks, remember: Just because it says “no” doesn’t mean it remembers what was actually true! 🚀

If you are looking to improve your prompting skills and haven’t already, check out our free Advanced Prompt Engineering course.

This blog post is based on the research article “Negation-Induced Forgetting in LLMs” by Authors: Francesca Capuano, Ellen Boschert, Barbara Kaup. You can find the original article here.