We often talk about ethics in tech but how much of an impact has that really made? I think that we can do better, but for that we need to give up on useless habits, hone our skills, and focus a lot more on governance.
https://berjon.com/ethicswishing/
@robin Thanks, this gave me a lot to think about.
Maybe you can help me with the following paradox:
You make a good case that non-binding values & reserve control are counterproductive. Commitments must be concrete and measurable enough to allow accountability.
However, this also immediately tingled my Goodhart instincts: Selection pressure for easy-of-measurability in ethics is a fast-track to dystopian results.
@robin If my python bot becomes the most moral entity around by simply min-maxxing mechanical rules, then that's not a moral system I'd like to be part of.
A quick search tells me that Goodhart's Law's relation to accountability is kinda known but I have no idea how you reconcile them in non-hand-wavy ways.
@artificialmind That's a very good question and it's very close to another thing I'm working on that I haven't finalised well yet.
Goodhart's Law is a management problem, it's basically what happens when you turn a complex problem into a relatively simple optimisation problem. You lose a lot of resolution about the target by doing that, and then incentivising that target means it will go where the optimisation is easiest which is unlikely to be actually good.
@artificialmind But that assumes bureaucratic management as the way in which accountability is done. It's a world of control by measurement.
Imagine if instead you knew you had to be in a town hall of people affected by your code on a regular basis. If you're doing it wrong, they're going to tell you (and maybe want to replace you) but you can't boil down their problems to a number.
@robin Ah that's a good example to explain why Goodhart's Law in my understanding is not only about numbers but rather about the difficulty to link "the spirit" to tight rules.
So I have to be in that town hall? Well I just bring my laptop and work on other stuff if attendance is what matters.
So the rule gets changed to "I have to interact with them". I'll just do small talk about other stuff.
"I have to talk about the code with them". I'll prioritize easy surface-level issues.
And so on.
@robin I wish I had a solution but that's the fear I have: It's hard-to-impossible to properly define rules that match the intented spirit.
The more you're trying to flesh out the rules, the more you're telling people "if you're inside the rules then your behavior is OK, we put a lot of thought into that". The flawed rules become a shield.
It almost feels like an Uncertainty Principle. Tight rules cannot match the spirit, loose rules cannot be held accountable.
@artificialmind That's why using democratic accountability isn't about rules, it's about whether the people affected by your behaviour find it good or not!
+bonifartius 𒂼𒄄
@robin @artificialmind
that's more or less natural law instead of the positive law implemented everywhere.
