"I would absolutely love to discover the original code review for this and why this was chosen as a default. If the PRs from 2011 are any indication, it was probably to get unit tests to pass faster."
This was a pretty interesting read.
https://withinboredom.info/blog/2022/12/29/golang-is-evil-on-shitty-networks/
@nyquildotorg Here you go, from the code's author: https://news.ycombinator.com/item?id=34181846
And John Nagle commenting on the tradeoffs and problems: https://news.ycombinator.com/item?id=34180239
And the rest of the comments go, in a roundabout way, into wondering why git-lfs is making write syscalls 50 bytes at a time in the first place, regardless of what the network stack does beyond that, because that's going to suck regardless of what the network stack does beyond that.
@danderson @nyquildotorg Gotta admit, I'm team TCP_NODELAY. Nagle has surprised people for generations, and "But I want to take 20k syscalls to transfer 1MB and have it look efficient" doesn't make me very sympathetic!
@sgf @danderson @nyquildotorg Nagle gets used to "fixup" a bunch of problems (eg silly window syndrome etc).
In general, there are two types of flow: elephants (bandwidth heavy) and mice (latency sensitive). You want nagle for the first class (keep overheads as low as possible for maximum throughput), and not for the second (keep latency low as possible).
@isomer @sgf @danderson @nyquildotorg
I would expect that elephants will buffer (and when they temporarily become mice they will either reshuffle things so that the bufferedwriter is out of the picture or they will simply keep flushing the writer at appropriate times). If that's the case then by disabling Nagle's algorithm we're wasting at most one packet each time the buffer is emptied (pessimistically we will emit one one-byte packet then). So, Nagle should be superfluous if we buffer with a buffer that's much larger than a packet or that's chosen to be a multiple of a packet's size. Am I missing some reason Nagle is useful?
@robryk @sgf @danderson @nyquildotorg that's all true, but you can't always keep your buffer full. Eg when reading data from disk. Esp for long fast networks.
Connections can often flip between mice and elephants. It's common to say "do you want this data?" then wait for a reply, then send the entire data. The first part is latency sensitive, the second part is bandwidth heavy.
@isomer @sgf @danderson @nyquildotorg
If the buffer is not full, the buffered writer will not write until it gets full. People who deal with buffered writers are used to flushing them at appropriate times (and there are all those funny affordances like stdio's "flush out when someone's reading in").
@robryk @isomer @danderson @nyquildotorg I've now got the dev party of my brain going "You could argue that Nagle is just a defense against badly-written programs that can't buffer properly", and the SRE part of my brain going "Yes! And we need defenses against badly written programs!".
Can we rename TCP_NODELAY to TCP_TRUSTME?
@sgf @isomer @danderson @nyquildotorg
Is that a defense that actually helps though? Without it, the issue presents itself always. With it, the other issue presents itself only sometimes.
~~~ preventing any network traffic from ever leaving also does that, so you don't want to ignore the latter completely. Surely there's a tradeoff where what you're saying is a correct reasoning, but it's hard to tell whether in any case we actually want _that_ tradeoff.
My point was mainly that "badly written because it sends small packets" is something that you can easily test, monitor for, etc. and that has rather nonconfusing symptoms.
@robryk Oh, to be clear, I dislike Nagle and think having it on by default as currently implemented is distinctly suboptimal. I'm just a lot more defensive about dealing with badly-written apps than I used to be ("If an app misbehaves, just fix it").
OTOH, I'm not sure what a good defense against a badly-written app would look like here!
@robryk Nagle prevents unnecessary small packets wasting network capacity: Maybe think of it as defending other network users from a badly-written app, not just the badly written app from itself?