Spoke with an engineer over the holidays who maintained that Meta is blameless for abuses on their platform. He asked me, mockingly, "should they stop scaling just because they can't moderate everything?"
The answer is YES. Scale is not just a technical issue. If you cannot control your system, stop growing it.
(btw this is also a good example of how #UX comes from more than just the buttons on the screen)
@PavelASamsonov this is a more complicated issue than you give FB credit for. If scale will allow X extra people to connect with an enjoy their families and friends, at the cost of Y extra people's suffering, where to draw the line with regards to Y is.. tough.
It seems you advocate an "if Y > 0 then halt everything" approach but that would deny X people the warmth and joy that comes from connecting to family and friends.
What the engineer was probably trying to get at is the idea that the market should be left to decide: if people feel the service is causing more harm than good, people by and large will stop using it. If people feel the service is causing more good than harm, they will continue (or start) using it. The latter would translate to increases in demand (need to scale) representing a democratized "this is good, keep doing it".
One might not like what the market is saying, or believe that it is a good/fair line to draw when balancing X:Y, but it's certainly a defensible one.
