“It’s a symbolic gesture,” said a senior tech executive at a media company, who requested anonymity to speak freely.

– Sara Guaglione

Digiday interviewed five publishers who implemented OpenAI’s software to block AI web crawlers from collecting material. The software was adopted by several prominent organizations.

One publisher, unnamed, says they have a hard time protecting their proprietary content from web scrapers.

The director of public policy at Guardian Media Group says it’s a step to “start demanding more control over other folks that are scraping for different purposes.”


Why publishers are questioning the effectiveness of blocking AI web crawlers | DIGIDAY | September 29, 2023 | by Sara Guaglione