AI-generated content is taking over.

There’s no longer any need for human writers of any kind, some say.

But wait – isn’t Google going to penalize AI-generated content? They did announce their intention to de-rank content generated by bots. But how will they determine what’s AI and what’s not?

That’s a good question, and it leads us to the subject of today’s post.

The Beginning

If you’re in a hurry, I’ll save you some time by telling you this: AI content detectors do not work, period.

There is zero evidence of any kind that indicates otherwise.

How do I know this?

Well, it’s a long story.

I had a client who demanded my content register as 75% “human” on a (paid-for) AI detector tool.

No matter what I did, I could never get the thing to say my content was higher than 48% human. I had next to no instructions on how to get the content to pass this ephemeral test.

Eventually, I discovered that most of the time, AI-generated content came back as being more “human” than things I had written entirely free of any outside influence. For example, an FAQ section I wrote without any research was 99.9% AI, according to this tool.

In the end, I lost the client. After spending hours and hours making edits and changes, it made more sense to cut my losses and move on. I did the best I could.

I even asked for help from communities of copywriters. None of them had ever encountered this problem before, so I didn’t get much out of the exchanges. The best answer I got was to try asking the client to use a different tool, but that wasn’t an option.

This started me on a quest to learn the truth about so-called “AI detectors.” How do they work? How effective are they? And why do they think I am 99.9% robot?

The Research Proving AI Content Detectors Don’t Work

My research didn’t take long. A few simple searches reveal tons of examples, experiments, and case studies proving beyond a shadow of a doubt that AI detectors don’t work. The best they can do is be slightly better than random at determining human vs. AI content. And the vast majority of AI detectors can’t do better than random at all. 

That’s why OpenAI shut down its AI detection software. It wasn’t working. And who would want to put out a software product that doesn’t work? Especially when anyone can verify that fact with ease.

I don’t believe it to be a coincidence that the tool my client was using cost money. Imagine that: people pay for this software that has been proven to not achieve what it claims to.

Back to the matter at hand. There is no evidence that AI detectors work. But there is plenty of evidence that they don’t.

One of the most compelling pieces of evidence comes from a science fiction writer named Maria Korolov.

She took three steps to test out 13 different AI content detectors.

The first step was to input a short story she wrote herself. About 90% of the detectors said the text was “mostly human,” which she notes to be in line with OpenAI’s results. Their detector had about a 9% false positive rate.

That doesn’t seem so bad, right? Just wait, it gets worse.  

The second step involved asking ChatGPT to re-write some of her stories, and the third step involved editing the AI-written text. In both cases, the results were erratic. The edited AI text often came back as even less human than the purely AI-written stuff.

The third step involved putting 100% AI-written content into the detector. The results were haphazard, with a few bots accurately identifying the text as AI, and many of them saying the text was “mostly human” or “possibly AI.”

I find the first step of the detectors identifying her content as human to be interesting. I did not have the same luck, as mentioned earlier. Some of my original content did pass as being up to 90% human, but some of it also came back as 99.9% AI. Despite this, I did everything in my power to try to bring up the “human” score, to no avail. In some cases, this only made things worse. In a way, that aligns with Korolov’s results of the AI/human hybrid content registering as even more AI than anything else. If my original content was perceived as AI to begin with, then any changes I make from that point on can only make things worse.  

Using Bots to Make Things More “Human”

The author brings up a very good point relating to the use of these AI detectors. She notes that it’s not difficult for those creating AI content to then use additional tools to make sure that content passes for human:

“These tools suggest story ideas to you, generate the stories, fill them full of search-engine friendly keywords, then run them against all the detectors out there to make sure they pass.

Yup. The tools automatically run the stories through the detectors and modify them until they can pass for human in all the detectors, while still keeping the text sounding natural. I’m not going to list the tools here. The spammers — I mean, “content marketers” — already know what the tools are, and if they don’t, I don’t want to give them any ideas.

If you do want to use these detectors, keep in mind that the results seem to be almost random — and the bad guys have access to them, as well.”

– Maria Korolov via Metastellar.com

The methods these tools use for determining the “humanness” of content are, presumably, composed of a deterministic set of rules that programmers deem to be accurate. So it shouldn’t be too difficult to come up with a tool that can alter content to fit those rules.

Knowing the rules isn’t even necessary. As Korolov points out, all that’s required is a tool that will keep running the content through a detector and making changes as necessary until the “human” reading increases. I’m not a programmer, but it’s not difficult to see that creating such a tool might be even easier than creating a detector since all it requires is automating a process of trial and error.

The implications of this extend into the future. No matter how advanced these detectors become, there will always be a way to spoof them. In other words, there will never be a way to accurately assess whether or not something has been written by AI.

AI Content Detectors Don’t Work: Conclusion

The fact that AI content detectors don’t work isn’t an all-around bad thing. AI can be an extraordinary tool, and I’m not against using it in some cases. I find it particularly useful for brainstorming, planning and strategizing, and I know many others do, too.

What doesn’t make sense is requiring writers to meet the requirements of a flawed tool that accomplishes nothing.

If you want your content to sound human, the only way to go about that is by good old-fashioned editing. A human reading through the thing will continue to be the only way to determine if something passes for sounding like a human.

If there’s one bright spot in the fact that AI content detectors don’t work, that would be it.

Note: None of this was written by AI. Not a single word.

If you liked this post, please consider commenting and sharing it on social media.




Looking for a copywriter with extensive experience in Bitcoin/crypto and fintech?

I’ve worked with brands like BitPay, Cointelegraph, Business Insider, Newsweek, Blockworks, and SoFi.

You can see what past clients have said about me on my LinkedIn profile and samples of my work on my portfolio: https://bdncontent.com/writer-portfolio-2/.

Follow me on Twitter and Instagram at @bdncontent.

Send an email or DM for a free consultation on reaching your marketing goals.