“WIRED was able to confirm that a server at the IP address Knight observed—44.221.181.252—will, on demand, visit and download webpages when a user asks Perplexity about the webpage, regardless of what the site’s robots.txt says.”
Fuck #Perplexity, obviously.
And great work, @robb!👏👏👏
https://www.wired.com/story/perplexity-is-a-bullshit-machine/
Follow
@matthiasott @robb I would say that if a user asks about a specific page it's allowed to access that page regardless of robots.txt, because it acts as a form of browser.
