I'm thinking about getting a "new" used car. Unfortunately, tests of driver assistance / "self driving" systems don't seem to be any better than they were three years ago when I wrote https://danluu.com/car-safety/.

No one's doing anything resembling what a programmer would consider serious benchmarking and it doesn't look like anyone is going to either even though driver assistance / ADAS systems are becoming a more important part of safety over time.

Since writing that post, I've spent a lot of time talking to people who work on ADAS systems about the processes they use; how they handle bugs when they're found; when they use a shared ADAS vendor, which companies find the bugs in the vendor's system and which companies never notice, etc.

I believe this is the only way you can reasonably compare these systems today and I doubt this will change within a decade. For obvious reasons, no one who has this information is going to write it up,

so it's now nearly impossible for the general public to make judgments about automotive safety except in cases where the company repeatedly ships extremely shoddy software, e.g., Tesla with https://mastodon.social/@danluu/109514063516163667, https://twitter.com/kenklippenstein/status/1612848872061128704, etc., but Uber was the only other company putting something so obviously bad on the roads, so this doesn't give you much information.

FWIW, I'd say that BMW has the best methodology I know of among cars you can buy (I updated the post to reflect that), but

every company I know of that's selling a "shipping" car has a methodology that you'd consider "move fast and break things" if they were a software company; the spectrum runs from "nearly completely untested, relies heavily on vendor's components working when connected and never verifies while pushing vendors to add unsafe features to reduce cost or increase velocity" to "moves quickly and cuts corners that create significant risk".

Just for example, a semi-typical story is