Why Your AI Agents Randomly Become Dumb

Treat your AI behaviour like a project. Version control it.

The Problem

WTF! why is my AI suddenly dumb?

It worked perfectly yesterday.

If you build with coding agents, you’ve probably seen this. One day they’re brilliant. The next day they’re Ralph Wiggum.

At first it feels random. But it isn’t.

The Real Cause

When we say an AI is “good”, what we really mean is: It’s focusing on the right information.

AI behaviour is the result of context + attention.

Too much context:

hallucinations increase
token usage spikes
compaction happens earlier
agents spin their wheels

Too little context:

agents miss key information
behaviour becomes inconsistent

Eventually I realised:

AI doesn’t randomly get dumb.

It loses signal in noise.

Diagram: Why AI Suddenly Gets Dumber

Long chats. Too many files. Too many instructions. Too many tools.

Eventually the model spends more energy filtering noise than solving the problem.

My Early Attempts (And Why They Failed)

At first I tried solving this with directory structure. Every time Claude started behaving strangely I would:

make a new directory
strip my Claude.md
disable MCP servers
sometimes even switch computers

Essentially I was resetting the environment until things worked again.

The real problem was simple.

I had no visibility into what actually changed.

Where AI Behaviour Actually Comes From

AI behaviour is not just the model. It is a stack of configuration layers.

See content credentials
Diagram: The AI Context Pyramid

The key insight:

AI behaviour = global config + project config + live context filtered through attention.

If you cannot see these layers, the system feels unpredictable.

The Breakthrough

I realised something simple. We version control our code. But we don’t version control our AI behaviour.

So I started managing the files that control my agents using Chezmoi.

Chezmoi is normally used to manage dotfiles across machines. But it works extremely well for managing AI configuration.

What I Track

My Chezmoi repository tracks the core files that shape Claude behaviour.

Examples include:

~/.claude.json
~/.claude/settings.json
~/.claude/settings.local.json
~/.claude/CLAUDE.md

And the behavioural runtime layer:

~/.claude/hooks
~/.claude/skills
~/.claude/commands
~/.claude/agents
~/.claude/mcp-servers
~/.claude/scripts

Everything lives in: ~/.local/share/chezmoi — which is backed by a Git repository.

Detecting Behaviour Drift

The most important command in my workflow is: chezmoi diff

This shows configuration drift. In other words: what has changed about my AI behaviour since the last known good state?

Drift usually falls into two categories.

Runtime Noise

Things like:

telemetry fields
session IDs
token counts

These usually should not be committed.

Behaviour Changes

Changes to:

skills
hooks
MCP servers
configuration files

These require review before becoming truth.

See content credentials
Diagram: The Chezmoi Control Loop

This introduces back pressure into the system.

AI can propose changes. But humans decide what becomes truth.

My Runtime Workflow

When I launch Claude, my launcher script runs checks before the session starts.

It verifies:

Bitwarden secrets
MCP configuration
Chezmoi drift
verification results

Only then does it start the session. This prevents me from working in an unknown configuration state.

Signals Your AI Context Is Broken

Some common signals:

compaction happening earlier
hallucinations increasing
agents retrying solutions repeatedly
token usage spikes
tools conflicting with each other

Example: Two Playwright agents fighting over the same browser instance.

This is almost always context or configuration drift.

The Biggest Beginner Mistake

Many AI developers make the same mistake. They let the AI do everything.

But AI systems still need human infrastructure. Think of it like a CI pipeline.

AI proposes changes. Humans validate them. Git stores the truth.

Without that loop, behaviour becomes chaotic.

The Unexpected Benefit

The biggest surprise was how simple this solution is. Chezmoi solved my visibility problem almost immediately.

Now I can:

see exactly what changed
revert behaviour instantly
back up my AI configuration
export my agent environment

My AI setup finally feels predictable.

Final Thought

Treat your AI behaviour like a project.

Version control it.

Because your agent system is unique to you.

Next Issue

Next week I’ll break down: How I structure skills, hooks, and MCP servers to reduce context bloat in Claude Code.