What does it tell us that AI scrapers are ignoring the more intelligent way of scraping data despite all the indications towards it?
https://shkspr.mobi/blog/2025/12/stop-crawling-my-html-you-dickheads-use-the-api/
I don't think the answer is a statement about AI, but about the people behind it?