look, we get it - it’s possible to scrape, index, proxy, and otherwise manipulate a huge amount of public-facing data in the fediverse.
that doesn’t mean you should. it’s bad. it violates the authors’ consent in many cases, and frankly it’s rude.
this isn’t your dev sandbox.
@djsundog It's not one's dev sandbox, but it *is* a massive trove of text put up on a publicly-accessible network, retrievable via unauthenticated clients. The author's consent is not technologically necessary to access it, and may or may not even be audited by the administrator of the service.
It *will* be indexed. It's only a matter of time. I appreciate the appeal to "Don't be that guy," but everyone with something to lose *must* understand that if it's really, really important to them that their toots not be searchable, a solution of "trust eight billion people not to build the thing" is completely untenable.
