‘…the AI did not emerge victorious — but even this mediocre performance is a stunning change from what AI was capable of just a few years ago. Until recently, chat bots were very obviously bots, producing incredibly simplistic, confused, incoherent text that no one would confuse for a competent human writer.’
An AI-written essay went unnoticed by six judges in an essay writing contest, but didn’t make it to the shortlist. A student-writing competition about climate change, sponsored by The Economist, had 2400 entrants. The judges were not told one of them was an AI-generated text, prepared by the GPT-2 algorithm.
- Four judges said ‘no’ and two said ‘maybe’
- None identified the essay as computer generated, it just didn’t meet the judges’ standards.
- A ‘no’ judge said, “I do not think it shows a strong understanding of existing climate policy nor of the scientific literature coming out of the IPCC.”
- From one of the two who thought ‘maybe,’ “It is strongly worded and backs up claims with evidence, but the idea is not incredibly original.”