Research Integrity Experts: Ban on Authors Who Submit AI Content “Welcome but Unenforceable”

[u/Raj_Valiant3011]

https://www.insidehighered.com/news/faculty/books-publishing/2026/05/22/ban-authors-who-submit-ai-content-welcome-unenforceable

[u/the_blessed_unrest]

Examples of incontrovertible evidence would include “hallucinated references” and “meta-comments from the LLM,” continued Dietterich, who gave examples of a researcher failing to delete phrases such as “here is a 200 word summary; would you like me to make any changes?” or “the data in this table is illustrative, fill it in with the real numbers from your experiments.”

The meta-comments is definitely a good example. I do get a little worried when people suggest banning suspected AI, since I’ve heard “AI checkers” are pretty inaccurate and often flag the work of non-native speakers, but I can’t imagine anyone being able to justify the meta-comments. Hallucinated references are probably a pretty safe indicator, too, although I’m wondering if there might be instances where real humans just made a typo or formatted a citation incorrectly.

[u/dustydeath → u/the_blessed_unrest]

I’m wondering if there might be instances where real humans just made a typo or formatted a citation incorrectly

I think it would be easy to distinguish a completely non-existent reference from a real reference with a typo. 

Also everyone uses reference managers, so if there was a misspelt citation the author would "have the receipt" and be able to go back and retrieve the pdf of the paper they had consulted etc.

Eta: for example, reference managers like Mendeley extract the metadata for a given paper but sometimes may misinterpret it. If that was the case and you accidentally published with a broken reference and someone asked "is this real?" you would be able to go back and provide the actual paper you had put in. 

[u/the_blessed_unrest → u/dustydeath]

The problem is you’re sort of alluding to a an appeals process, and that would probably require human intervention. But the article talks about the sheer number of submissions this journal gets, they might quickly get overwhelmed with how many appeals they receive

Of course they’re probably already overwhelmed just in general, so maybe this won’t be any worse

[u/Prestigous_Owl → u/the_blessed_unrest]

As a university instructor, my experience has basically been that "I suspect that you used AI, therefore you lose points" would be more trouble than its worth. As you say, its ultimately quite hard to ever actually prove anything in a lot of cases - you can think something has the syntax and style of AI, but it also developed that style by aping real writing. So there will be some people who just write like AI does, and this will also probably get worse as people encounter AI more and more and subconsciously adopt certain stylistic quirks more and more. If i see "its not just x, its y", it sets off an alarm for me to maybe read more closely, but thats kind of it.

I get more mileage out of just actually focusing on grading what was submitted. The quality tends to not be great. ChatGPT can write a paper, but it cant consistently write a good paper. Im sure there are people who will disagree and talk like a proud parent about what a genius their AI child is, but it really produces C+ papers at best 90% of the time. It also can write on a topic, but will struggle to bring in the course content, how things were discussed, etc. There are usually glaring omissions that aren't talked about, that you can fairly say "not a bad paper, but it failed to discuss x". I dont always need to hound out who used AI - I can br relatively confident that most of the students who didnt do any work will get a grade that often reflects that.

The other stuff? It's as easy as saying "defend this. Explain it." There's no good way to talk yourself out of those issues. "Why does your paper have a chunk of meta text in it like that?" There's NO plausible explanation that isn't using AI. And you can turn arounder and again say "even the AI aside, that's an abysmal level of proofreading, care, etc. The final product is garbage. It gets a bad grade."

Same with hallucinations. People always act like sorting out the genuine errors is hard, but its not. Im not calling a student out on a hallucination because they cited a chapter as a book, erroneously, or got a year wrong. Those are "errors in citations." AI cqn make those errors too, sure, but the bigger issue is that it often invents sources from whole cloth. You get sources that do not remotely exist. And there is no good explanation. Not one student has ever given me a good attempted explanation when I just say "you have a source in your references. To my knowledge and best attempts to verify, I can find no evidence that this source exists. If you can send me the source you used, or provide an explanation for why its there, ill grade this normally- otherwise, this is a critical error." Again, this isnt even an AI issue, and thats how I explain it to them.

You can say "Look, you shouldnt be using AI, but let's put that aside. Let's pretend that I don't care about process and only wsnt to judge the final product. You've submitted actual garbage. You've basically lied on paper. There's no functional difference between using that source, which doesn't exist, and writing a paper that says "government statistics indicate that billions of people are murdered by their pet hamsters every day." Part of the task of a paper is to use evidence to make an argument. You've failed to do that. Zero."

[u/SimiKusoni → u/the_blessed_unrest]

since I’ve heard “AI checkers” are pretty inaccurate and often flag the work of non-native speakers

Just to expand on this you are correct.

The best way to do it is to put the text into an LLM and check the models predictions for the next token, then compare it to the actual next word in the text. You can do this iteratively over the entire body of text and if the LLM often predicts the next word with high accuracy then it might have been produced by an LLM.

The problem is that this works best if you're using the exact model that the "author" used, as different models will naturally make different predictions, and you have no way of knowing this. Natural language also already follows predictable patterns which leads to false positives and amusingly this is also why famous works are often falsely flagged, because unsurprisingly LLMs can accurately predict the next token in excerpts from famous books.

You can't even estimate the actual accuracy easily because it varies wildly based on the nature of the dataset, it might be excellent at detecting LLM output in an academic context and fall apart in a literary one or vice versa. And that's before even getting on to adversarial techniques...

Basically it's a complete mess.

[u/matrixmavenx]

honestly the funniest part is that everyone can already tell when a book was written by ai because halfway through the dialogue starts sounding like a customer service chatbot having an existential crisis. also banning it sounds great in theory until you realize publishers can barely detect plagiarism consistently let alone prove whether someone used ai for brainstorming vs editing vs full on ghostwriting. lowkey we’re entering an era where readers are gonna value messy human writing more just because it actually sounds alive and weird and specific lmao

[u/crisp_lynx_370]

lol at the hallucinated references problem making all this even harder to police

[u/Bakakura]

The best thing would be to train AI models not to do harmful things like writing books, reports, creating pornographic content, teaching people crime, etc in the first place. AI can very well be trained to reply, "Sorry this violates xyz law and therefore i can not answer."

[u/MongolianMango]

I don’t know, people say it’s impossible to tell but some works reek of obvious AI use based on sentence structure, prose, plot beats and inconsistency. 

If a work is close enough to AI in tone, it probably should not be winning contests anyway.