‘…the AI did not emerge victorious — but even this mediocre performance is a stunning change from what AI was capable of just a few years ago. Until recently, chat bots were very obviously bots, producing incredibly simplistic, confused, incoherent text that no one would confuse for a competent human writer.’

An AI-written essay went unnoticed by six judges in an essay writing contest but didn’t make it to the shortlist. A student-writing competition about climate change, sponsored by The Economist, had 2,400 entrants.

The judges were not told one of them was an AI-generated text, prepared by the GPT-2 algorithm.

Four judges said ‘no,’ and two said ‘maybe.’ None identified the essay as computer generated; it didn’t meet the judges’ standards.

A ‘no’ judge said, “I do not think it shows a strong understanding of existing climate policy nor of the scientific literature coming out of the IPCC.” From one of the two who thought ‘maybe,’ “It is strongly worded and backs up claims with evidence, but the idea is not incredibly original.”


The Economist’s essay contest featured an AI submission. Here’s what the judges thought.
VOX | October 4, 2019 | by Kelsey Piper