Many comments are posted on OpenAI's work towards a classifier that distinguishes generated text from human written text. According to OpenAI's own announcement (a) input texts were not taken from an adversarial source, i.e. no deliberate attempts were made to obfuscate AI authorship; (b) there was a false positive rate of 9%, i.e. for 9% of the texts the tool evaluated a human written text as being AI generated.

These two points make the tool currently unfit for an academic misconduct allegation, where the false positive rate has to be practically indistinguishable from zero, and where efforts to obscure the authorship will be encountered.

Indeed, there is a short video in a response to OpenAI's tweet, in which a researcher copies ChatGPT output, runs it through GPT-3 – and the AI author is no longer recognized.

Also, the true positive rate of only 26% (i.e. what fraction of AI generated text was recognized as not having a human author) is rather sobering. Though this is hardly surprising: AI generated text is optimized to appear natural.

I touch on the topic in previous analyses at https://sentientsyllabus.substack.com – and misconduct will be the focus of the next newsletter at https://sentientsyllabus.substack.com

#SentientSyllabus #ChatGPT #HigherEd #AI #Education #Plagiarism