{"id":1425,"date":"2024-03-01T15:07:51","date_gmt":"2024-03-01T15:07:51","guid":{"rendered":"https:\/\/www.pk-pharmakommunikation.de\/blog\/we-ran-a-test-how-well-can-ai-create-texts\/"},"modified":"2024-03-04T06:25:26","modified_gmt":"2024-03-04T06:25:26","slug":"we-ran-a-test-how-well-can-ai-create-texts","status":"publish","type":"post","link":"https:\/\/www.pk-pharmakommunikation.de\/en\/blog\/we-ran-a-test-how-well-can-ai-create-texts\/","title":{"rendered":"We ran a test: How well can AI create texts?"},"content":{"rendered":"<p>Of course we did it last year too: we tested how good ChatGTP was at writing articles. Our verdict at the time was mixed. Yes, a home\u00adwork assign\u00adment on a not too current and diffi\u00adcult topic is defi\u00adni\u00adtely possible. The concerns of some teachers are justi\u00adfied. In our opinion, it was no longer suitable for a review article on a scien\u00adtific topic. Good for us \u2014 our job will stay with us for a while&nbsp;yet.<\/p>\n<p>But what left us feeling more than a little uneasy at the time was the way ChatGTP handled refe\u00adrences. For the most part, it simply made them up. No joke: these \u201crefe\u00adrences\u201d were made up from set pieces of real-life author names, titles, jour\u00adnals, etc. At first glance, it wasn\u2019t even noti\u00adceable. At first glance, it wasn\u2019t even noti\u00adceable, but only when it was checked \u2014 which is part of the job of a science journalist.<\/p>\n<p>The good news is that arti\u00adfi\u00adcial intel\u00adli\u00adgence is indeed evol\u00adving. The same query sent to ChatGTP today is answered with comple\u00adtely correct refe\u00adrences. Nevert\u00adheless, such missteps alert AI to the fact that we are giving it the benefit of the doubt \u2014 which may not be justified.<\/p>\n<p>Now one of the online spell-checking plat\u00adforms available on the internet has also \u201ccome up with AI\u201d (rechtschreibpruefung24.de). This is not prima\u00adrily intended to improve the grammar check, but rather to provide rewor\u00adding sugges\u00adtions. This looks like this, for example:<\/p>\n<p><strong>Original:<\/strong><\/p>\n<p><em>What left me feeling more than a little uncom\u00adfor\u00adtable at the time, however, was the way ChatGTP handled refe\u00adrences. For the most part, it simply made them up. No joke: these \u201crefe\u00adrences\u201d were put toge\u00adther from set pieces of real exis\u00adting author names, titles, jour\u00adnals, etc. At first glance, it wasn\u2019t even noti\u00adceable. At first glance, it wasn\u2019t even noti\u00adceable, but only when the checking started \u2014 which is part of the job for a science journalist.<\/em><\/p>\n<p><strong>Simpler:<\/strong><\/p>\n<p>At the time, I was parti\u00adcu\u00adlarly bothered by the way ChatGTP handled refe\u00adrences. For the most part, they were simply made up by combi\u00adning real author names, titles and journal names. I didn\u2019t notice this at first, only on closer inspection.<\/p>\n<p><strong>Formal:<\/strong><\/p>\n<p>What left an extre\u00admely unplea\u00adsant feeling, however, was ChatGTP\u2019s approach to refe\u00adrences. A large part of it was actually just cons\u00adtructed. These \u2018refe\u00adrences\u2019 consisted of frag\u00adments of real author names, titles and jour\u00adnals. This went unno\u00adticed at first and only became appa\u00adrent on closer inspec\u00adtion \u2014 some\u00adthing that is essen\u00adtial for a science journalist.<\/p>\n<p><strong>Alter\u00adna\u00adtive:<\/strong><\/p>\n<p>A very unsett\u00adling feeling arose at the time from the way ChatGTP handled refe\u00adrences. In fact, most of them were simply made up by thro\u00adwing toge\u00adther real author names, titles and journal names. This was not noti\u00adceable at first; only on closer inspec\u00adtion \u2014 which is important for a science jour\u00adna\u00adlist \u2014 did it become clear.<\/p>\n<p>&nbsp;<\/p>\n<p><strong>Our opinion on the result<\/strong><\/p>\n<p>Not bad in parts, but the meaning is slightly distorted in some places and some content is missing. In further tests with other texts, the actual meaning some\u00adtimes changed consider\u00adably in both the \u201cSimpler\u201d and \u201cAlter\u00adna\u00adtive\u201d versions.<\/p>\n<p><strong>Conclu\u00adsion:<\/strong> AI curr\u00adently seems to be a nice tool for complex and deman\u00adding texts. But please take a close look and keep your trust in the black box within narrow limits!<\/p>\n<p>P.S. Only the free versions were tested for our trials.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Of course we did it last year too: we tested how good ChatGTP was at writing articles. Our verdict at the time was mixed. Yes, a home\u00adwork assign\u00adment on a not too current and diffi\u00adcult topic is defi\u00adni\u00adtely possible. The concerns of some teachers are justi\u00adfied. In our opinion, it was no longer suitable for&nbsp;[\u2026]<\/p>\n","protected":false},"author":4,"featured_media":1423,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_et_pb_use_builder":"","_et_pb_old_content":"","_et_gb_content_width":"","wp_typography_post_enhancements_disabled":false,"footnotes":""},"categories":[20],"tags":[],"class_list":["post-1425","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-blog-en"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.1.1 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>We ran a test: How well can AI create texts? &#8226; PK Pharmakommunikation<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.pk-pharmakommunikation.de\/en\/blog\/we-ran-a-test-how-well-can-ai-create-texts\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"We ran a test: How well can AI create texts? &#8226; PK Pharmakommunikation\" \/>\n<meta property=\"og:description\" content=\"Of course we did it last year too: we tested how good ChatGTP was at writing articles. Our verdict at the time was mixed. Yes, a home\u00adwork assign\u00adment on a not too current and diffi\u00adcult topic is defi\u00adni\u00adtely possible. The concerns of some teachers are justi\u00adfied. In our opinion, it was no longer suitable for [&hellip;]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.pk-pharmakommunikation.de\/en\/blog\/we-ran-a-test-how-well-can-ai-create-texts\/\" \/>\n<meta property=\"og:site_name\" content=\"PK Pharmakommunikation\" \/>\n<meta property=\"article:published_time\" content=\"2024-03-01T15:07:51+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2024-03-04T06:25:26+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.pk-pharmakommunikation.de\/wp-content\/uploads\/Blog_Maerz_24.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"589\" \/>\n\t<meta property=\"og:image:height\" content=\"394\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Elke\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Elke\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"4 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/www.pk-pharmakommunikation.de\/en\/blog\/we-ran-a-test-how-well-can-ai-create-texts\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.pk-pharmakommunikation.de\/en\/blog\/we-ran-a-test-how-well-can-ai-create-texts\/\"},\"author\":{\"name\":\"Elke\",\"@id\":\"https:\/\/www.pk-pharmakommunikation.de\/en\/#\/schema\/person\/c09686d559e7fc144b0cde96970f2313\"},\"headline\":\"We ran a test: How well can AI create texts?\",\"datePublished\":\"2024-03-01T15:07:51+00:00\",\"dateModified\":\"2024-03-04T06:25:26+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.pk-pharmakommunikation.de\/en\/blog\/we-ran-a-test-how-well-can-ai-create-texts\/\"},\"wordCount\":573,\"publisher\":{\"@id\":\"https:\/\/www.pk-pharmakommunikation.de\/en\/#organization\"},\"image\":{\"@id\":\"https:\/\/www.pk-pharmakommunikation.de\/en\/blog\/we-ran-a-test-how-well-can-ai-create-texts\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.pk-pharmakommunikation.de\/wp-content\/uploads\/Blog_Maerz_24.jpg\",\"articleSection\":[\"Blog\"],\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.pk-pharmakommunikation.de\/en\/blog\/we-ran-a-test-how-well-can-ai-create-texts\/\",\"url\":\"https:\/\/www.pk-pharmakommunikation.de\/en\/blog\/we-ran-a-test-how-well-can-ai-create-texts\/\",\"name\":\"We ran a test: How well can AI create texts? &#8226; PK Pharmakommunikation\",\"isPartOf\":{\"@id\":\"https:\/\/www.pk-pharmakommunikation.de\/en\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.pk-pharmakommunikation.de\/en\/blog\/we-ran-a-test-how-well-can-ai-create-texts\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.pk-pharmakommunikation.de\/en\/blog\/we-ran-a-test-how-well-can-ai-create-texts\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.pk-pharmakommunikation.de\/wp-content\/uploads\/Blog_Maerz_24.jpg\",\"datePublished\":\"2024-03-01T15:07:51+00:00\",\"dateModified\":\"2024-03-04T06:25:26+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/www.pk-pharmakommunikation.de\/en\/blog\/we-ran-a-test-how-well-can-ai-create-texts\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.pk-pharmakommunikation.de\/en\/blog\/we-ran-a-test-how-well-can-ai-create-texts\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.pk-pharmakommunikation.de\/en\/blog\/we-ran-a-test-how-well-can-ai-create-texts\/#primaryimage\",\"url\":\"https:\/\/www.pk-pharmakommunikation.de\/wp-content\/uploads\/Blog_Maerz_24.jpg\",\"contentUrl\":\"https:\/\/www.pk-pharmakommunikation.de\/wp-content\/uploads\/Blog_Maerz_24.jpg\",\"width\":589,\"height\":394},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.pk-pharmakommunikation.de\/en\/blog\/we-ran-a-test-how-well-can-ai-create-texts\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.pk-pharmakommunikation.de\/en\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"We ran a test: How well can AI create texts?\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.pk-pharmakommunikation.de\/en\/#website\",\"url\":\"https:\/\/www.pk-pharmakommunikation.de\/en\/\",\"name\":\"PK Pharmakommunikation\",\"description\":\"\",\"publisher\":{\"@id\":\"https:\/\/www.pk-pharmakommunikation.de\/en\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.pk-pharmakommunikation.de\/en\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.pk-pharmakommunikation.de\/en\/#organization\",\"name\":\"PK Pharmakommunikation\",\"url\":\"https:\/\/www.pk-pharmakommunikation.de\/en\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.pk-pharmakommunikation.de\/en\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/www.pk-pharmakommunikation.de\/wp-content\/uploads\/pk-logo.svg\",\"contentUrl\":\"https:\/\/www.pk-pharmakommunikation.de\/wp-content\/uploads\/pk-logo.svg\",\"width\":566,\"height\":160,\"caption\":\"PK Pharmakommunikation\"},\"image\":{\"@id\":\"https:\/\/www.pk-pharmakommunikation.de\/en\/#\/schema\/logo\/image\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.pk-pharmakommunikation.de\/en\/#\/schema\/person\/c09686d559e7fc144b0cde96970f2313\",\"name\":\"Elke\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"We ran a test: How well can AI create texts? &#8226; PK Pharmakommunikation","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.pk-pharmakommunikation.de\/en\/blog\/we-ran-a-test-how-well-can-ai-create-texts\/","og_locale":"en_US","og_type":"article","og_title":"We ran a test: How well can AI create texts? &#8226; PK Pharmakommunikation","og_description":"Of course we did it last year too: we tested how good ChatGTP was at writing articles. Our verdict at the time was mixed. Yes, a home\u00adwork assign\u00adment on a not too current and diffi\u00adcult topic is defi\u00adni\u00adtely possible. The concerns of some teachers are justi\u00adfied. In our opinion, it was no longer suitable for [&hellip;]","og_url":"https:\/\/www.pk-pharmakommunikation.de\/en\/blog\/we-ran-a-test-how-well-can-ai-create-texts\/","og_site_name":"PK Pharmakommunikation","article_published_time":"2024-03-01T15:07:51+00:00","article_modified_time":"2024-03-04T06:25:26+00:00","og_image":[{"width":589,"height":394,"url":"https:\/\/www.pk-pharmakommunikation.de\/wp-content\/uploads\/Blog_Maerz_24.jpg","type":"image\/jpeg"}],"author":"Elke","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Elke","Est. reading time":"4 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.pk-pharmakommunikation.de\/en\/blog\/we-ran-a-test-how-well-can-ai-create-texts\/#article","isPartOf":{"@id":"https:\/\/www.pk-pharmakommunikation.de\/en\/blog\/we-ran-a-test-how-well-can-ai-create-texts\/"},"author":{"name":"Elke","@id":"https:\/\/www.pk-pharmakommunikation.de\/en\/#\/schema\/person\/c09686d559e7fc144b0cde96970f2313"},"headline":"We ran a test: How well can AI create texts?","datePublished":"2024-03-01T15:07:51+00:00","dateModified":"2024-03-04T06:25:26+00:00","mainEntityOfPage":{"@id":"https:\/\/www.pk-pharmakommunikation.de\/en\/blog\/we-ran-a-test-how-well-can-ai-create-texts\/"},"wordCount":573,"publisher":{"@id":"https:\/\/www.pk-pharmakommunikation.de\/en\/#organization"},"image":{"@id":"https:\/\/www.pk-pharmakommunikation.de\/en\/blog\/we-ran-a-test-how-well-can-ai-create-texts\/#primaryimage"},"thumbnailUrl":"https:\/\/www.pk-pharmakommunikation.de\/wp-content\/uploads\/Blog_Maerz_24.jpg","articleSection":["Blog"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/www.pk-pharmakommunikation.de\/en\/blog\/we-ran-a-test-how-well-can-ai-create-texts\/","url":"https:\/\/www.pk-pharmakommunikation.de\/en\/blog\/we-ran-a-test-how-well-can-ai-create-texts\/","name":"We ran a test: How well can AI create texts? &#8226; PK Pharmakommunikation","isPartOf":{"@id":"https:\/\/www.pk-pharmakommunikation.de\/en\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.pk-pharmakommunikation.de\/en\/blog\/we-ran-a-test-how-well-can-ai-create-texts\/#primaryimage"},"image":{"@id":"https:\/\/www.pk-pharmakommunikation.de\/en\/blog\/we-ran-a-test-how-well-can-ai-create-texts\/#primaryimage"},"thumbnailUrl":"https:\/\/www.pk-pharmakommunikation.de\/wp-content\/uploads\/Blog_Maerz_24.jpg","datePublished":"2024-03-01T15:07:51+00:00","dateModified":"2024-03-04T06:25:26+00:00","breadcrumb":{"@id":"https:\/\/www.pk-pharmakommunikation.de\/en\/blog\/we-ran-a-test-how-well-can-ai-create-texts\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.pk-pharmakommunikation.de\/en\/blog\/we-ran-a-test-how-well-can-ai-create-texts\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.pk-pharmakommunikation.de\/en\/blog\/we-ran-a-test-how-well-can-ai-create-texts\/#primaryimage","url":"https:\/\/www.pk-pharmakommunikation.de\/wp-content\/uploads\/Blog_Maerz_24.jpg","contentUrl":"https:\/\/www.pk-pharmakommunikation.de\/wp-content\/uploads\/Blog_Maerz_24.jpg","width":589,"height":394},{"@type":"BreadcrumbList","@id":"https:\/\/www.pk-pharmakommunikation.de\/en\/blog\/we-ran-a-test-how-well-can-ai-create-texts\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.pk-pharmakommunikation.de\/en\/"},{"@type":"ListItem","position":2,"name":"We ran a test: How well can AI create texts?"}]},{"@type":"WebSite","@id":"https:\/\/www.pk-pharmakommunikation.de\/en\/#website","url":"https:\/\/www.pk-pharmakommunikation.de\/en\/","name":"PK Pharmakommunikation","description":"","publisher":{"@id":"https:\/\/www.pk-pharmakommunikation.de\/en\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.pk-pharmakommunikation.de\/en\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.pk-pharmakommunikation.de\/en\/#organization","name":"PK Pharmakommunikation","url":"https:\/\/www.pk-pharmakommunikation.de\/en\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.pk-pharmakommunikation.de\/en\/#\/schema\/logo\/image\/","url":"https:\/\/www.pk-pharmakommunikation.de\/wp-content\/uploads\/pk-logo.svg","contentUrl":"https:\/\/www.pk-pharmakommunikation.de\/wp-content\/uploads\/pk-logo.svg","width":566,"height":160,"caption":"PK Pharmakommunikation"},"image":{"@id":"https:\/\/www.pk-pharmakommunikation.de\/en\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/www.pk-pharmakommunikation.de\/en\/#\/schema\/person\/c09686d559e7fc144b0cde96970f2313","name":"Elke"}]}},"_links":{"self":[{"href":"https:\/\/www.pk-pharmakommunikation.de\/en\/wp-json\/wp\/v2\/posts\/1425","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.pk-pharmakommunikation.de\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.pk-pharmakommunikation.de\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.pk-pharmakommunikation.de\/en\/wp-json\/wp\/v2\/users\/4"}],"replies":[{"embeddable":true,"href":"https:\/\/www.pk-pharmakommunikation.de\/en\/wp-json\/wp\/v2\/comments?post=1425"}],"version-history":[{"count":2,"href":"https:\/\/www.pk-pharmakommunikation.de\/en\/wp-json\/wp\/v2\/posts\/1425\/revisions"}],"predecessor-version":[{"id":1427,"href":"https:\/\/www.pk-pharmakommunikation.de\/en\/wp-json\/wp\/v2\/posts\/1425\/revisions\/1427"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.pk-pharmakommunikation.de\/en\/wp-json\/wp\/v2\/media\/1423"}],"wp:attachment":[{"href":"https:\/\/www.pk-pharmakommunikation.de\/en\/wp-json\/wp\/v2\/media?parent=1425"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.pk-pharmakommunikation.de\/en\/wp-json\/wp\/v2\/categories?post=1425"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.pk-pharmakommunikation.de\/en\/wp-json\/wp\/v2\/tags?post=1425"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}