Complete Guide to Spotting AI Chatbot Plagiarism

Complete Guide to Spotting AI Chatbot Plagiarism.

AI chatbots like ChatGPT are not limited to just answering your questions. They can write emails, cover letters, essays, poetry, blog posts, and much more.

AI writing is like a chameleon: it can mix so well with any other type of writing that it would be hard to tell the difference.

Unfortunately, this makes it incredibly difficult for us humans to detect when an AI has typed something instead of a person. Sure, we could all take Sherlock Holmes’s advice and “eliminate the impossible”, but who has time for that?

What if I tell you that the above paragraph is written by an AI? Yes, I asked YouWrite to write a quick intro for this post, which is what came up. Pretty convincing, right? That’s the problem, it’s too convincing that anyone can present it as their own work to benefit from it.

AIs don’t just rephrase content; they learn the topic and present it in a natural language using RLHF. This means that the generated content will not feel robotic to read nor can it be compared to the data available online for plagiarism.

Capturing AI-generated content isn’t easy, and I’m not going to give you false hope that you can capture it with 100% accuracy. However, with a little detective work and some help from AI typing detection tools, you can spot most AI jobs.

Below you will find manual ways to detect AI plagiarism and also some automated tools.

Manually detect AI plagiarism 🕵️

ChatGPT isn’t even a big deal when it comes to AI plagiarism. There are many alternative tools that are based on the same GPT-3 technology and are designed for typing. These are very advanced and made specifically for creating human-like content that is difficult to grasp.

Auto-detection tools are a bit hit and miss based on content type, length, and complexity. So, currently at least, manually connecting the dots to determine if the item is AI generated or not is the best way.

Below I’ve listed some common clues you might look for in capturing AI-generated content.

#one. outdated information

ChatGPT and other GPT-3-based AI saw their knowledge scaled back in late 2021. Although some AI bots have a built-in search engine to find the latest information, they still rely on old data to write long-form content like essays or blog posts.

If you notice that the article constantly talks about outdated data, specifically before 2021, then it could be AI generated.

In the screenshot below, I asked YouWrite to tell me about the recent FIFA World Cup, and this is what came up.

It talks about the 2018 World Cup instead of the recent 2022. Although the tool is updated to talk about recent events like the 2022 World Cup, to write long posts, it still uses the outdated data it’s based on.

#2. Use the same sentence structure for descriptions

While writing multiple product/app descriptions, AI typically uses the same sentence structure that it edits to fit a product description. Descriptions often start with the same word, such as The, It, A, or the name of the product. Descriptions also have a dedicated repeating focus.

For example, below I asked ChatGPT to tell me about “data entry job websites”. You can see that all the descriptions start with “A” and have a strict focus.

I also asked YouChat the same question, and it did the same thing and repeated the name of each website at the beginning, and the descriptions themselves look like a rephrased version of each one.

If the article you’re reviewing has a description of a similar nature, it’s a great way to pick up on AI writing.

#3. use short sentences

For most articles, AI will use single line sentences with a maximum of one comma. Perhaps they try to minimize the possibility of error, but avoid writing long sentences that may require the use of colons, semicolons, or thick horizontal lines.

I know this isn’t the best hint for catching AI typing, but it’s worth considering to further prove your suspicions.

#4. You may provide incorrect information

For complex questions that typically involve some form of instruction, the AI ​​may not understand the intent and provide incorrect information. I’m not talking about outdated information here, they just confidently tell you the wrong information when they don’t understand the topic.

For example, I asked YouWrite to tell me “how to root Android phone without PC”. He still showed me methods that involve the use of a PC at some point in the instruction.

#5. Does not add a personal opinion

When talking about something, the AI ​​only talks about known facts. Unlike humans, she won’t talk about her personal experience with him. Although AIs can be instructed to create practical writing, they tend to avoid doing so by default.

If you notice that the article only talks about well-known facts, it is most likely written by an AI. For example, when talking about an app, you’ll be talking about its features instead of fluid navigation or how to use the app.

#6. brief explanations

As AIs are limited by currently available knowledge, they tend to write brief explanations and descriptions. In my experience, if they talk about a particular topic, like Android rooting, they keep the explanation within 2-3 paragraphs at most. And for things like apps or products, the description usually doesn’t exceed 1 paragraph.

Humans can write in depth about something, including features, limitations, personal experience, pricing, etc. On the other hand, AI usually focuses on the main features or highlights, so the writing is also short. If the instructor asks for more details, the chance of incorrect information increases significantly.

#7. Review the writer’s story.

As a teacher or webmaster, if you have access to the writer’s previous work or assignments, it might be a good idea to compare it to the article you suspect. It is highly unlikely that the writer would have an AI-like writing style. Although AI can change typing tones, the styles it chooses are pretty generic, and it can’t copy your tone.

Taking the example of the AI-generated paragraph in the introduction of this post. Anyone who has read my previous posts can easily see that my writing style is completely different compared to him.

You can simply read 2-3 previous works of the writer and see if it matches the writing style of the suspected article.

Tools to detect AI plagiarism

There are a ton of tools available that can detect content written by AI. However, its accuracy is different from each other and it may fail depending on the content type and extension.

I tested these tools using different types of content generated by ChatGPT, GPT-3 Playground, WriteSonic, Rytr, and YouWrite. They were able to detect all kinds of content generated by ChatGPT and GPT-3 Playground.

Although they had mixed results for dedicated writing tools, strangely enough, they all failed to detect YouWrite, I’m assuming it’s because YouWrite makes human-like mistakes in order to avoid detection.

I will recommend that you use these tools in conjunction with manual content verification. First, scan the content with one of these tools, and then manually check for clues to further confirm the suspicion.

Here are the ones that gave the best results and are the easiest to use:

Note: For the demo, I’ll be using WriteSonic-generated text under “Benefits of Mangoes.”

#one. content at scale

I found the Content At Scale AI content detector to be the most accurate for most of the tests I ran. It allows you to scan up to 2500 characters per scan and makes a percentage prediction showing the density of real or fake content.

It is safe to assume that content that is more than 60% fake is written by AI content, at least for the most part. In the screenshot below, you can see that 96% of the content is fake, which I generated entirely with AI.

#2. GPTZero

If you want to see exactly what content the AI ​​generates, then GPTZero is a good choice. GPTZero does not rate the content. Instead, it highlights the exact content that the AI ​​is likely to generate.

Although it shows puzzlement and burst scores to get an idea of ​​the randomness in the text, more importantly, it supports 5000 characters per scan and can also upload files. Scanning large content helps a lot to improve accuracy.

#3. GPT-2 Exit Detector

The creators of ChatGPT also offer a bot to detect your own work. Since GPT-2 and GPT-3 are only different in the total data used for training, this GPT-2 detector also works well for GPT-3 based AIs.

There’s no limit to the amount of content you can scan, so it’s perfect for detecting very long AI-generated content. I also found its accuracy to be good, especially for content generated by ChatGPT. However, it does miss out on predictions a bit, so only consider your results when it shows more than 50% of the content as false.

#4. Writer AI Content Detector

Writer AI Content Detector results are questionable when detecting content from AI writing tools, but it works great for content generated by ChatGPT. You can scan 1500 characters per scan and even add the URL of the content if it is already published.

For my AI generated text test, it said that 94% of the content is human generated, which is pretty bad compared to other tools. However, it’s still worth using to detect ChatGPT content, especially if it’s already published.

#5. draft and goal

A simple yet powerful AI handwriting detection tool. In my experience, Draft & Goal delivered similar results to Content At Scale, but doesn’t have a word count limit so you can scan long essays. However, your scanning time increases based on the amount of content you scan.

For my example text, it says that 94% of the text is generated by AI.

My thoughts 💭

Personally, I think AI help is actually a great way to get rid of writer’s block, get all your information in one place, and find inspiration. However, it should never replace human writing as AI is limited to already known knowledge.

You can also explore some of the best plagiarism checker tools to prevent copying.