🧵Sigh.
Everyone dunking on this tweet from Elon is wrong. There's nothing particularly wrong with it.
It's a common code problem Twitter might have (many smaller ops to fulfill one large request).
It's common service problem (not enough servers in some countries).
I'm not saying Elon is smart here, only that he's not particularly dumb.
Everyone believes Elon is dumb, not a real engineer, and is looking for confirmation of their bias.
The reality is that Elon really is an engineer, or at least enough of one to understand this issue.
What I think is going on here is probably the same confirmation bias. Elon was already convinced Twitter code is inefficient, so is on a witchhunt to find the most inefficient parts of the code to pretend Twitter's old engineers were stupid.
It's indeed a common issue he describes. RPC is designed to make the cost of procedure calls invisible. So you often end up with programmers ignoring the cost. Better code is optimized around making as few RPC calls as reasonable.
@ErrataRob If you mean have dependency chains with fewer RPCs in them, I agree.
If you mean fewer RPCs in total (even if they happen concurrently), then I'm confused. How is batching them at the application layer more efficient than causing them to be sent as separate requests at the RPC layer? One way or the other you need to have the client and server keep track of all outstanding requests/parts of batches: it only moves the responsibility for doing so between RPC infrastructure and application layer. Worse, batching at application layer creates problems for the infrastructure: you can't split such a request between two replicas of a backend, the loadbalancing infrastructure has to deal with requests that are variably expensive (depending on their contents), latency measurements are now a function of distribution over batch sizes, etc.
I've seen cases where batching did help performance, but they were either of the "we need these things to happen in order, but we have all the requests ready already" variety or very specialized setups (where for some reason keeping track of subrequests in application code was more convenient or where the subrequests were extremely short). I've seen more cases when someone did implement batching and created headaches for everyone maintaining the system later on, so this makes me extremely doubtful of arguments that point at missing batching as a problem in itself.
