AI Chatbots like ChatGPT aren’t limited to just answering your questions. They can write human-like emails, cover letters, essays, poetry, blog posts, and a lot more.
AI writing is like a chameleon – it can blend in with any other type of writing so well that you’d be hard-pressed to tell the difference.
Unfortunately, this makes it incredibly difficult for us humans to detect when an AI has written something instead of a person. Sure, we could all take Sherlock Holmes’ advice and “eliminate the impossible,” but who has time for that?
What if I tell you that the above paragraph is written by an AI? Yes, I asked YouWrite to write a quick introduction for this post, which is what it came up with. Pretty convincing right? That’s the problem, it’s too convincing that anyone can present it as their own work to benefit from it.
AIs don’t just rephrase the content; they learn the topic and present it in a natural language using RLHF. This means the generated content will neither feel robotic to read nor can it be compared with data available online to catch plagiarism.
Catching AI-generated content isn’t easy, and I am not going to give you false hope that you can catch it with 100% accuracy. However, with a little bit of detective work and some help from AI writing detection tools, you can catch most works of AI.
Below you’ll find manual ways to detect AI plagiarism and some automatic tools too.
Manually Detect AI Plagiarism 🕵️
ChatGPT isn’t even a big problem when it comes to AI plagiarism. There are many alternative tools that are based on the same GPT-3 technology and are built for writing. These are very advanced and specifically made to create human-like content that is difficult to catch.
Automatic detection tools are a bit hit or miss depending on content type, length, and complexity. So at least currently, manually connecting the dots to figure out whether the write-up is AI-generated or not is the best way.
Below I have listed some common clues that you could look for to catch AI-generated content.
#1. Outdated Information
ChatGPT and other GPT-3 based AI had their knowledge cut at the end of 2021. Even though some AI bots have a built-in search engine to look for the latest information, they still rely on old data to write long-form content like essays or blog posts.
If you notice the write-up constantly talking about outdated data, specifically before 2021, then it could be AI-generated.
In the below screenshot, I asked YouWrite to tell me about the recent FIFA World Cup, and this is what it came up with.
It talks about the World Cup 2018 instead of the recent 2022 one. Even though the tool is updated to talk about recent events like the 2022 World Cup, for writing long posts, it still uses the outdated data it is built on.
#2. Uses the Same Sentence Structure for Descriptions
While writing descriptions of multiple products/apps, AI usually uses the same sentence structure that it edits to suit the description of a product. The descriptions usually start with the same word, like The, It, A, or the product’s name. The descriptions also have a dedicated focus that it repeats.
For example, below, I asked ChatGPT to tell me about “data entry jobs websites”. You can see all descriptions start with “A” and has a strict focus.
I also asked YouChat the same question, and it did the same thing and repeated the name of each website at the start, and the descriptions themselves look like a rephrased version of each.
If the write-up you are analyzing has a description of anything similar in nature, it’s a great way to catch AI writing.
#3. Uses Short Sentences
For most write-ups, AI will use one-liner sentences with a single comma at most. Maybe they try to minimize the chance of error, but they avoid writing long sentences that might require the use of colons, semicolons, or heavy horizontal lines.
I know this isn’t the best clue to catch AI writing, but it’s worth considering to further prove your suspicion.
#4. Can provide incorrect information
For complex questions that usually involve some kind of instruction, the AI can fail to understand the intention and provide incorrect information. I am not talking about outdated information here, they just confidently tell you the wrong information when they don’t grasp the topic.
For example, I asked YouWrite to tell me “how to root an Android phone without a PC”. It still showed me methods that involve the use of a PC at some point in instruction.
#5. Doesn’t Add a Personal Opinion
When talking about something, AI just talks about known facts. Unlike humans, it won’t talk about its personal experience with it. Although AIs can be instructed to create a hands-on write-up, they tend to avoid doing it by default.
If you notice the write-up is only talking about well-known facts, it’s a good chance it’s written by an AI. For example, while talking about an app, it will talk about its features instead of how smooth the navigation is or how actually to use the app.
#6. Brief Explanations
As AIs are limited by the knowledge currently available, they tend to write brief explanations and descriptions. In my experience, if they talk about a particular subject — such as Android rooting — they keep the explanation within 2-3 paragraphs max. And for things like apps or products, the description usually doesn’t exceed 1 paragraph.
Humans can write in-depth about something, including features, limitations, personal experience, pricing, etc. On the other hand, AI usually focuses on the main features or highlights, so the write-up is short too. If the instructor asks for more details, the chance of incorrect information significantly increases.
#7. Check the writer’s history
As a teacher or webmaster, if you have access to the writer’s previous work or assignments, it might be a good idea to compare it with the write-up you are suspicious about. It’s highly unlikely that the writer has a similar writing style to an AI. Even though AI can change writing tones, but styles it chooses are rather generic, and it can’t copy your tone.
Taking the example of the AI-generated paragraph in the intro of this post. Anyone who has read my previous posts can easily see that my writing style is entirely different compared to it.
You can just read 2-3 previous works of the writer and see if it matches the writing style of the suspicious write-up.
Tools To Detect AI Plagiarism
There are a bunch of tools available that can detect AI-written content. However, their accuracy is different from each other and can fail depending on the type of content and length.
I tested these tools using different types of content generated by ChatGPT, GPT-3 Playground, WriteSonic, Rytr, and YouWrite. They managed to detect all types of content generated by ChatGPT and GPT-3 Playground.
Although they had mixed results for dedicated writing tools, interestingly, they all failed to detect YouWrite, I am guessing it’s because YouWrite makes human-like mistakes on purposes to avoid detection.
I will recommend you use these tools alongside manually checking the content. First, analyze the content with one of these tools and then manually check for any clues to confirm the suspicion further.
Below I am listing the ones that offered the best results and are the easiest to use:
Note: For demonstration, I am going to use generated text from WriteSonic on “Benefits of Mangoes”.
#1. Content At Scale
I found the AI content detector by Content At Scale to be the most accurate for most of the tests I ran. It lets you scan up to 2500 characters per scan and makes a percentage prediction showing real or fake content density.
Content that is over 60% fake is safe to assume is written by AI content, at least most of it. In the below screenshot, you can see says 96% content is fake, which I fully generated with AI.
#2. GPTZero
If you want to see exactly what content is AI generated, then GPTZero is a good option. GPTZero doesn’t score content. Instead, it highlights the exact content that is probably generated by AI.
Although it does show perplexity and burstiness scores to get an idea of randomness in the text, more importantly, it supports 5000 characters per scan and can also upload files. Scanning large content greatly helps in improving accuracy.
#3. GPT-2 Output Detector
The creators of ChatGPT also offer a bot to detect their own work. As GPT-2 and GPT-3 are only different in the total data used for training, this GPT-2 detector works fine for GPT-3 based AIs too.
There is no limit on how much content it can scan, so it’s perfect for detecting very long AI-generated content. I also found its accuracy fine, especially for ChatGPT-generated content. However, it’s a bit lose in predictions, so only consider its results when it shows over 50% of the content as fake.
#4. Writer AI Content Detector
Writer AI Content Detector results are questionable when detecting content from AI writing tools, but it works great for ChatGPT-generated content. You can scan 1500 characters per scan and even add the URL of the content if it’s already published.
For my test AI-generated text, it said 94% of the content is human-generated, which is quite off compared to other tools. Nonetheless, it’s still worth using to detect ChatGPT content, especially if it’s already published.
#5. Draft & Goal
A simple yet powerful AI writing detection tool. In my experience, Draft & Goal offered similar results to Content At Scale, but it doesn’t have a word count limit so that you can scan long write-ups. However, its scan time increases based on how much content you scan.
For my example text, it says that 94% of the text is AI-generated.
My Thoughts 💭
I personally think AI help is actually a great way to remove writer’s block, get all the information in one place, and find inspiration. However, it should never replace human writing as AI is limited to the knowledge already known.
You may also explore some best plagiarism checking tools to prevent copying.
