"Do as little testing as reasonable."
That was my advice here recently to a friend.
He was concerned about the quality standards at work. "The users do find bugs!"
I asked him, "are the users complaining loudly about it broadly or churning at a high rate?"
He said, "I don't know. But users find bugs!"
I shrugged. "Users will always find bugs. That's not enough signal to tell you if you're testing enough. Do as little testing as reasonably possible."
It's really easy to spend endless amounts of time testing. So the priority should always be to ship features and limit the risk of what happens when things go wrong.
@raiderrobert Different bugs have different costs. At #Backblaze, we put a huge effort into testing that user data is stored correctly and durably. Much less testing on web site button placement.
Indeed!
Decades ago, when video game software was published via ROM cartridges and discs (CD-ROM, DVD-ROM, BD-ROM), they had to be tested to assure the games were playable to the end without any major bugs. Especially as games got more complex, there were often some minor cosmetic issues, but they were seldom bad enough to trigger a total recall or the destruction of a publisher.
@bwbeach fully agreed!
Different levels of risk require different levels of testing.