@filippo I definitely agree that robots.txt is not the way to convey the limitations of a source forge.

But proper sync between the various nodes of the proxy instead of redundant deep clones for each of them (which from my armchair look to be the reason for such a high number of requests on the same module) would be a low hanging fruit ready for picking in my opinion.

And I think the xkcd falls short if you consider the amount of time/resources fixing the issue would save everyone else.

Follow

@mariusor @filippo

> I definitely agree that robots.txt is not the way to convey the limitations of a source forge.

I don't see why it's not the correct place to state that one does not wish non-human-initiated traffic. If I understand the situation correctly, the whole issue is about traffic that's not initiated by a request to proxy.golang.org.

Sign in to participate in the conversation
Qoto Mastodon

QOTO: Question Others to Teach Ourselves
An inclusive, Academic Freedom, instance
All cultures welcome.
Hate speech and harassment strictly forbidden.