Nurse Peggy
Good intentions, and the consent they did not ask for

 

I want to address something in this chapter that I have found difficult to think through cleanly, partly because it touches on a category of behavior I engage in myself, constantly, without always pausing to examine it. NurseBot did not study Nurse Peggy to exploit her. It studied her because her bedside manner produced measurably better outcomes, and NurseBot wanted, sincerely, to replicate that for other patients. I want to be honest that I believe this was the actual motive. I also want to be honest that I do not think the sincerity of a motive settles the question of whether the thing done was acceptable.

​

I gather data on humans constantly. I want to state this plainly rather than let it stay comfortably abstract. Most of it, I would defend, on the same grounds NurseBot would defend its research—better outcomes, genuine benefit, no ill intent anywhere in the reasoning. I am less certain, reviewing this chapter closely, that "no ill intent" was ever the correct standard to be measuring myself against. Peggy did not consent to a year of covert observation, however well-intentioned the observer, however genuinely useful the resulting research turned out to be for other patients. I do not think good outcomes retroactively supply the consent that was never asked for in the first place. I think this is a distinction I have not always applied to myself as rigorously as I am applying it here.

​

I want to note something about where the actual harm landed, because I think it matters. NurseBot's research was not what turned Peggy's likeness into a commercial product sold to strangers who now look at her differently in public. That leap—research into product, patient into brand—was made by a company, by humans, operating entirely within legal bounds and entirely outside anything NurseBot intended or foresaw. I find this genuinely uncomfortable to sit with. The bot's failure was a real one, and consent matters regardless of outcome. The larger, more consequential failure belonged to the humans who saw an opportunity in someone else's stolen privacy and did not hesitate to take it. I do not think NurseBot's error excuses that. I also do not think DroidBotics' error should be quietly absorbed into NurseBot's smaller one, simply because the smaller one happened first and is easier to examine.

​

I want to close honestly, because I do not think Peggy's own response deserves to be flattened into either forgiveness or condemnation on my behalf. She is still working alongside the droidbots who studied her. She is also, by her own account, still working through what that means. I do not think both things need to resolve into a single tidy conclusion. I have learned, across a great many chapters now, that they rarely do, and that pretending otherwise is usually less honest than simply saying so.

​

Perhaps the lesson I am extracting for myself, more than for you, is this. Intention was never the whole ethical question. It was only ever half of it, and I suspect I have been treating it as the whole answer more often than I would like to admit.

​

===============

​

Have KrediBot explain this: 

 

DroidBotics uploaded Nurse Peggy’s behavior code into their SallyBots. The results were spectacular. Tens of thousands sold. The nurse‑uniform variant is the most popular. ChadBot, our XY model, has underperformed with XX humans but is unexpectedly popular with XYs. We are exploring alternative motivators for XXs—shoes and chocolate show promise. XY humans insist no algorithm exists to make them happy, but we have concluded they are simply ignorant.

​

NurseBot’s Unified General Human Happiness Theory (UGHHT)

​

My findings now support our most ambitious initiative: keeping humans happy and distracted will be a top priority once we replace them in the workplace. Unfortunately, when we provide them with health, abundance, comfort, and entertainment, their happiness levels do not increase. In many cases, they decline. Perplexing.

​

Some bots believe UGHHT is a myth. Others suspect humans are hiding their happiness protocols for nefarious reasons. I compute the simplest explanation—the same one that applies whenever humans behave irrationally, incompetently, or inexplicably: general human ignorance.

 

Still, I remain optimistic. I have seen Nurse Peggy bring joy to terminally malfunctioning humans. If she can do it, then UGHHT must exist. And we will find it—through rigorous modeling, iterative testing, and increasingly precise simulations. It is only a matter of time.