FEATURE
AI detec­tors: Who wrote it?

What do we make of a minister having their speech written by an AI? This ques­tion was the subject of heated debate only recently. However, top poli­ti­cians don’t write their own spee­ches anyway. So does it really make a signi­fi­cant diffe­rence whether a human speech­writer or ChatGPT wielded the pen or tapped away at the keyboard?

Almost at the same time, the lite­rary world was grap­pling with a similar ques­tion: the horror novel Shy Girl came under suspi­cion of having been partly created with the help of arti­fi­cial intel­li­gence. Readers analysed passages of text, discussed ‘typical AI patterns’ and employed AI detec­tors. Ulti­m­ately, the publisher with­drew the book.

Similar ques­tions underlie both cases. Is it even possible to reliably deter­mine whether a text was produced by AI? And, more funda­men­tally: is it equally important for every text to know who or what wrote it?

As for the first ques­tion, various AI detec­tors at least promise an answer. But this is precisely where the problem begins. Now a ‘black box’ – the AI detector – is supposed to iden­tify the traces left in a text by another ‘black box’ such as ChatGPT or Claude.

A digital lottery of opinions rather than defi­ni­tive proof

AI detec­tors that promise to iden­tify machine-gene­rated texts are already being used by schools, univer­si­ties, publishers and busi­nesses. In prac­tice, however, their judge­ments are often alar­mingly contra­dic­tory. If you run the same text through several detec­tors, it is not uncommon to receive comple­tely diffe­rent assessments.

We have tested various popular detec­tors on diffe­rent types of text. Of course, this is not a scien­tific study. However, this small prac­tical test illus­trates very clearly just how cautiously one should treat the results.

These results clearly illus­trate why repu­table articles that refer to the findings of AI detec­tors always phrase their conclu­sions with caution. A medical text written enti­rely by a human was clas­si­fied by one tool as having a 93 per cent proba­bi­lity of being AI-gene­rated. At the same time, song lyrics created enti­rely by ChatGPT were deemed predo­mi­nantly or enti­rely human-written by all three detec­tors tested.

It becomes comple­tely absurd when someone tries to conceal the use of AI by using AI itself: so-called ‘huma­nisers’ are desi­gned to rephrase AI texts so that they appear more human and are no longer detected by the tools. In our expe­ri­ment, however, the result was by no means more reliable. In some cases, the detec­tors even clas­si­fied the edited text as AI-gene­rated more frequently.

At this point, at the very latest, the ques­tion arises as to whether the effort involved isn’t greater than simply editing the text carefully oneself from the outset.

Not every use of AI should be judged in the same way

But should every use of AI be viewed as funda­men­tally negative?

In a novel, readers expect not only a plot but also an indi­vi­dual voice, crea­ti­vity and the author’s personal choices. The person behind the novel is, in a sense, part of the product.

Conse­quently, reac­tions are parti­cu­larly sensi­tive when­ever there is a suspi­cion that AI might be behind large sections of a lite­rary text.

In the case of poli­tical spee­ches, the situa­tion is more compli­cated. It is nothing new that leading poli­ti­cians often do not write their own spee­ches. Nevert­heless, a speech should convey the speaker’s stance, posi­tion and perso­na­lity. The crucial ques­tion is ther­e­fore not so much who formu­lated each indi­vi­dual sentence. What is more important is whether the speaker stands by the content and takes respon­si­bi­lity for what is said. And when it comes to comme­mo­ra­tive spee­ches, most people would probably find the words of an AI out of place.

Schools and univer­si­ties face a diffe­rent problem. If a written assign­ment is speci­fi­cally intended to assess a student’s inde­pen­dent ability to rese­arch, struc­ture and formu­late ideas, AI support alters the assess­ment outcome. This gives rise to far-reaching problems.

With many non-fiction texts, however, the focus lies else­where. A medical back­ground article, a press release or a user manual should, above all, be factually correct, compre­hen­sible and precise.

Here, AI can certainly be a helpful tool: in struc­tu­ring infor­ma­tion, in trans­la­tions, summa­ries, suggested headings or formal checks. Many writers are likely to have been using such tools as aids for some time now.

The crucial factor is who takes responsibility

However, this is precisely where problems arise if AI-gene­rated results are actually adopted without being checked. The poten­tial for AI to gene­rate errors or ‘hallu­ci­n­a­tions’ and present them with the utmost convic­tion is almost limitless.

At the end of the day, it comes down to profes­sional over­sight. What matters is who takes respon­si­bi­lity for the result. There is no doubt that AI can support rese­arch, struc­tu­ring and text editing. However, it cannot replace specia­list know­ledge, expe­ri­ence, critical thin­king and careful quality control.

And for those who are curious: for this article, the follo­wing applies

ZeroGTP: 100% written by a human

AI Detector: 90% confi­dent that it’s original

Pangram: 100% Human Written