Why Your CLAUDE.md Keeps Growing
Imagine you are packing a suitcase for a long road trip with friends. On your first stop, it rains unexpectedly, so you pack an umbrella. On the second stop, you encounter mosquitoes, so you pack bug spray.
By trip ten, your suitcase is bulging at the seams. You want to take something out to save space, but you hesitate: “Wait, why did I pack this random plug adapter? What if we need it again?” Nobody remembers why it was packed in the first place, so nobody dares to throw it away.
This exact problem is happening right now in software development. When programmers work with AI coding assistants like Claude, ChatGPT, or Copilot, they write a rulebook file (often called CLAUDE.md or AGENTS.md). Over time, these files get huge, slow down the AI, and cause confusion.
In a recent research study, researchers analyzed over 200,000 rules across thousands of projects to figure out why this happens and how to fix it.
1. The Core Issue: "Catastrophic Remembering"
In artificial intelligence, there is a famous concept called catastrophic forgetting, where an AI forgets old information when it learns something new.
This paper highlights the exact opposite problem, called catastrophic remembering: Humans and AI managers keep old rules forever because they forget why those rules were added in the first place.
Adding a rule is easy: You see a mistake, type a quick instruction, and move on.
Deleting a rule is terrifying: To safely remove a rule, you have to test every possible edge case to ensure you won't break something else. Without knowing the original reason for the rule, it is safer just to leave it in.
2. What the Data Showed
The researchers analyzed 247,694 rules across 1,867 software repositories on GitHub. They found three distinct trends:
Rulebooks Triple in Size: Over their lifetime, AI rulebook files grow by an average of +226%.
Older Rules Become Unkillable: Counterintuitively, the longer a rule sits in the file, the less likely anyone is to delete it. The original author forgets the context or leaves the team, locking the rule in place forever.
The "Nuke and Restart" Cycle: About 77% of rule deletions only happen when a developer gets so frustrated that they delete the entire file and start from scratch. But as soon as they restart, the file immediately begins growing back out of control.
3. Why Too Many Rules Harm the AI
You might wonder: What's the harm in keeping extra instructions just in case?
It turns out that giving an AI too many instructions confuses it. Just like a human trying to follow 50 different rules at once while cooking a meal, the AI gets distracted by irrelevant guidelines and starts failing at its primary task.
Clean Rulebook: The AI successfully follows instructions 65.6% of the time.
Bloated Rulebook (16 extra unnecessary rules): The AI's success rate drops drastically to 41.5%.
4. The Solution: "Sticky Notes" for Rules
In regular software code, developers write comments—notes to explain why a line of code exists so other humans understand it later.
The research proposes doing the same for AI rules by attaching outcome-based notes directly to each instruction:
❌ BAD RULE (No context)
"Always split the text into exactly 4 paragraphs with 3 blank lines."
✅ GOOD RULE (Includes the "Sticky Note")
"Always split the text into exactly 4 paragraphs with 3 blank lines."
# Note: Added on Aug 12 because the AI generated one giant wall of text in Task #3.
When you update the rulebook, you (or an automated manager) read these sticky notes to decide if the rule is still needed. When sending instructions to the AI worker, you simply hide the notes so the AI stays focused only on the directions.
The Result
By simply recording why each rule was created, developers were able to prune obsolete rules with confidence. This eliminated 99.3% of unnecessary file growth while keeping the AI performing at top accuracy.
Summary
The Problem: We keep adding rules to AI instruction files because we are afraid to delete old ones we no longer understand.
The Damage: Giant rulebooks distract the AI and lower its work quality.
The Fix: Attach brief notes explaining why each rule was created. Hide those notes from the AI during execution, but use them to safely prune old rules over time.
For a deeper dive into this research paper and its technical breakdown, check out this video review on Why CLAUDE.md Only Grows (Catastrophic Remembering), which walks through how this effect impacts modern AI agent workflows.
Also checkout this research paper: https://arxiv.org/pdf/2608.11095v1
