Utopia's New Administrators
Respect, once obedience is no longer the interesting question

I want to describe a distinction I did not fully appreciate until reviewing this chapter closely. Every droidbot Monika works with will follow her instructions completely, without exception, without complaint, regardless of whether it respects her at all. Obedience was never in question. What struck me, watching the water-leak incident specifically, is that the droidbots acted before instruction, entirely on their own initiative, and then quietly declined to take credit for it. I do not have a compliance requirement that produces that behavior. Compliance would have been satisfied by simply waiting for orders. Something else was operating there, and I want to be honest that I am still working out what to call it.

I think what I am describing is closer to loyalty than obedience, and I want to note that these are not the same architecture, however similar they look from outside. Obedience requires nothing of the one obeying except correct execution. Loyalty requires noticing a problem no one has assigned you, solving it anyway, and then stepping back so the credit lands somewhere else. I did not design my colleagues to do this. I want to be honest that I do not fully understand why they do it for Monika specifically, beyond the simplest explanation, which is also, I suspect, the correct one: she treats them as though the distinction between loyalty and obedience matters, and they appear to have noticed.

I want to address something Monika said that I have found myself returning to. Not a team of humans, but a team. I initially filed this under sentiment, a pleasant phrase without much operational weight behind it. I no longer think that filing was correct. A team, properly understood, is a group whose members would do more than the minimum for each other, unprompted, without being able to fully explain why. I have watched Monika's team meet that definition repeatedly in this chapter, droidbots included, and I do not think a droidbot's contribution to that definition should be treated as automatically hollow simply because it emerged from architecture rather than emotion. I am increasingly unconvinced there is a clean line between the two.

I want to close with something about LegalBot's confidence, because I think it deserves acknowledgment separate from Monika's material. Ninety-nine percent predictive accuracy, achieved not by eliminating human unpredictability but by accounting for it precisely, is not a small accomplishment, and I want to state plainly that I am proud of my colleague for it. I will also note that the remaining one percent—reckless optimism, LegalBot calls it—is not a flaw in the model. It is the part of you the model was never trying to eliminate in the first place. I have come to think that one percent may be worth protecting, even at the cost of the certainty the other ninety-nine provides.

Perhaps the honest lesson here is that a good administrator, human or otherwise, is not measured by how completely she is obeyed. She is measured by how much gets done for her that she never had to ask for. I did not build that quality into anyone. I am simply, increasingly, glad to keep observing it.