AI is eating itself

and I say that as someone who uses it every single day

I love Claude. Not casually. Not the way you love a useful app. Not in passing. The way you love something that has genuinely changed how you think, how you build, how you move through hard problems. It has made me capable of things I could not have done alone, and I mean that seriously.

But I do not love it unconditionally. And I think that distinction matters more right now than a lot of the things being said about AI currently.

Because there is something happening underneath all of the productivity gains and the impressive demos and the think pieces about what this technology means for the future of work. Something structural. Something that is not being talked about enough in the spaces where most people are actually using these tools.

AI is eating itself. If you are not paying attention to how that is happening, you will not notice when it starts to affect the quality of your work.

what these models actually are

Before we get into the problem, we need to agree on what we are actually talking about.

Claude, ChatGPT, Gemini, Grok, every one of these frontier models you are using right now is not intelligent in the way the word implies. It is not thinking. It is not consulting a verified database of facts. It is a model trained on an enormous snapshot of what humans have chosen to put on the internet, up to a certain point in time, and it predicts what words should come next based on the patterns in that snapshot.

That is the whole thing. That is the whole magic trick right there.

The snapshot is remarkable. The patterns it contains are genuinely useful. But it is a snapshot of the internet. That is not the same thing as a snapshot of truth. The internet has always been a record of what people choose to publish, what algorithms choose to surface, and what engagement loops choose to reward. It has never been a neutral archive of verified reality. And AI is learning everything it knows from it.

This matters because the snapshot can be corrupted. It already has been.

the proof of concept nobody meant to run

In early 2025, DuckDuckGo’s AI started telling users that Donald Trump, the president of the United States, had died of rabies. He had not. Now is probably not the time for a joke, but I will say it was an understandable creative choice.

What happened was not a sophisticated attack. A loosely coordinated group of people on Reddit, united by a shared feeling about a certain public figure, seeded enough content into the places AI systems scrape that the model found pattern support for a false claim and stated it with complete confidence.

No budget. No infrastructure. No technical expertise. Just volume, patience, and the knowledge that these models absorb what the internet is saying.

They did it as a joke. What they actually did was prove a thesis. They demonstrated that you do not need to hack a model to corrupt its outputs. You just need to corrupt the data it is learning from. And they showed the rest of the world, including people who are not joking, exactly how that works.

Now scale that. A government, an organized group, a trillionaire with a long enough time horizon is not trying to make one AI say one funny wrong thing. They are trying to shift the baseline slowly. To make a population’s AI tools gradually more likely to distrust certain institutions, inflame certain divisions, accept certain framings as normal. Not dramatically. Not visibly. At the margins, over time.

This is not hypothetical. I am not trying to scare anyone. It is the logical next step of something that is already happening.

the model collapse problem

The adversarial version is the dramatic one, but there is a quieter version happening at the same time that is just as important.

Researchers call it model collapse. The short version: when AI models train on content that was itself generated by AI, the outputs degrade. Not immediately. Not dramatically. Gradually. The long tail of human variation gets squeezed out, the weird edge cases, the minority perspectives, the unusual phrasings, the niche knowledge. The model becomes more confident and less accurate. Each generation of training compounds that problem.

We are already past the point where anyone can reliably identify AI-generated text at scale. AI cannot even reliably identify AI-generated text at scale. Which means the training data for the next generation of models already contains significant AI output, with no reliable way to tell which is which. The generation after that will be worse. So on and so forth.

The snapshot is being contaminated. Some of it deliberately. Some of it just from the sheer volume of AI-generated content flooding every corner of the internet. Either way, the model is eating itself. Slowly. If you are not paying attention, you will not notice the quality shifting until it already has.

there are partial answers

This is not a dead end. My framing is not stop using AI. That would be a strange thing for me to say. My framing is use it with your eyes wide open.

The most honest partial answer is constrained, purpose-built models. A legal research tool that only pulls from verified case law. A medical model trained exclusively on peer-reviewed literature with human oversight baked into the output. A customer service model with a bounded, audited knowledge base. These are not as exciting as a frontier model that can do anything. But they are more trustworthy for the specific thing they are built to do, because the snapshot they are working from has been deliberately limited and checked.

I build constrained tools for my own work for exactly this reason. A pitch research tool that only pulls from a client’s live site. A site audit tool that scores against a fixed rubric I wrote and can update. The boundaries are the point.

The weakness is that even constrained sources can be corrupted over time. No source is perfectly clean. But a smaller, more audited snapshot is meaningfully harder to poison at scale than the entire internet.

what you put in and what gets put into you

The other partial answer is harder to package, but it is the one that actually scales.

People talk about AI reducing cognitive load. In some ways it does. But what AI actually does is shift cognitive load. The effort you used to spend on production you now spend on judgment, verification, and staying sharp about what the tool actually is. If you are not spending that effort, you are not getting a free pass. You are just not doing the part that matters most.

Here is what that looks like in practice for me.

I maintain documented rules for how AI works in my workflows, and I update them regularly, not once and never again. As models change, as my thinking changes, the instructions change with them. The model is only as good as what you have told it about you, and that is a living document, not a one-time setup.

I do not allow AI to handle human-facing communication without me fully in the loop. Emails to clients, social media posts, anything that goes out into the world under my name gets read, revised, and posted by me. Every time I do that, I am putting human-generated content into the world instead of AI-generated content. Does it make the work more intensive? Yes. That is the trade I have decided is worth making, because the compounding works both ways.

I prompt actively for the voices that are usually missing. When I ask AI to research something, I specifically instruct it to look for minority perspectives and underrepresented sources, not just the loudest or most-linked result. The default will always surface the default. You have to ask for something different to get something different.

I have a rule against AI personification in my own language, and I work to keep it out of my writing. I will get more into this in a future post, but the short version is that the way we talk about these tools shapes how we think about them, and how we think about them shapes how we use them. This piece is partly about what happens when the words we use become the reality we build. I try not to feed that loop.

I banned em dashes from my AI outputs years ago, before it became a reliable tell for machine-generated text. It started as a personal preference. Now it does double duty: a small, consistent signal that what you are reading came from a person. A person with occasionally questionable sentence structure, but a person nonetheless.

I also require a confidence score at the end of every AI response I receive. The model rates its own answer, one to one hundred, and explains its uncertainty if the score is low. Is AI judging AI a perfect solution? No. But at least we are making the model second-guess itself. That is the whole thing we are fighting for here.

I ask AI to use a clearly described placeholder rather than invent missing information. It sounds like a small rule. It is one of the most important ones. The confident tone is the problem. Training the tool to stop and flag uncertainty instead of filling in the gap is a direct counter to the failure mode this whole piece is about.

None of these are bulletproof. I have watched every single one of them fail at some point. The em dash slips through. The placeholder gets skipped. The research surfaces the same dominant voices anyway. These are not guarantees. They are pressure. Small, consistent, directional pressure against a very large default pulling the other way.

The tail is being consumed faster than most people are paying attention to. These are my attempts to slow it down, and what I can suggest others try.

The snapshot is not truth. Use it like the powerful, flawed, remarkable thing it is.