Back in the 1990s, ground squirrels were briefly fashionable pets, but their popularity came to an abrupt end after an incident at Schiphol Airport on the outskirts of Amsterdam. In April 1999, a cargo of 440 of the rodents arrived on a KLM flight from Beijing, without the necessary import papers. Because of this, they could not be forwarded on to the customer in Athens. But nobody was able to correct the error and send them back either. What could be done with them? It’s hard to think there wasn’t a better solution than the one that was carried out; faced with the paperwork issue, airport staff threw all 440 squirrels into an industrial shredder.

[…]

It turned out that the order to destroy the squirrels had come from the Dutch government’s Department of Agriculture, Environment Management and Fishing. However, KLM’s management, with the benefit of hindsight, said that ‘this order, in this form and without feasible alternatives,* was unethical’. The employees had acted ‘formally correctly’ by obeying the order, but KLM acknowledged that they had made an ‘assessment mistake’ in doing so. The company’s board expressed ‘sincere regret’ for the way things had turned out, and there’s no reason to doubt their sincerity.

[…]

In so far as it is possible to reconstruct the reasoning, it was presumed that the destruction of living creatures would be rare, more used as a threat to encourage people to take care over their paperwork rather than something that would happen to hundreds of significantly larger mammals than the newborn chicks for which the shredder had been designed.

The characterisation of the employees’ decision as an ‘assessment mistake’ is revealing; in retrospect, the only safeguard in this system was the nebulous expectation that the people tasked with disposing of the animals might decide to disobey direct instructions if the consequences of following them looked sufficiently grotesque. It’s doubtful whether it had ever been communicated to them that they were meant to be second-guessing their instructions on ethical grounds; most of the time, people who work in sheds aren’t given the authority to overrule the government. In any case, it is neither psychologically plausible nor managerially realistic to expect someone to follow orders 99 per cent of the time and then suddenly act independently on the hundredth instance.

— Dan Davies: The Unaccountability Machine

***

Someone – an airline gate attendant, for example – tells you some bad news; perhaps you’ve been bumped from the flight in favour of someone with more frequent flyer points. You start to complain and point out how much you paid for your ticket, but you’re brought up short by the undeniable fact that the gate attendant can’t do anything about it. You ask to speak to someone who can do something about it, but you’re told that’s not company policy.

The unsettling thing about this conversation is that you progressively realise that the human being you are speaking to is only allowed to follow a set of processes and rules that pass on decisions made at a higher level of the corporate hierarchy. It’s often a frustrating experience; you want to get angry, but you can’t really blame the person you’re talking to. Somehow, the airline has constructed a state of affairs where it can speak to you with the anonymous voice of an amorphous corporation, but you have to talk back to it as if it were a person like yourself.

Bad people react to this by getting angry at the gate attendant; good people walk away stewing with thwarted rage.

— ibid.

***

A credit company used to issue plastic cards to its clients, allowing them to make purchases. Each card had the client’s name printed on it.

Eventually, employees noticed a problem: The card design only allowed for 24 characters, but some applicants had names longer than that. They raised the issue with the business team.

The answer they’ve got was that since only a tiny percentage of people have names that long, rather than redesigning the card, those applications would simply be rejected.

You may be in a perfectly good standing, but you’ll never get a credit. And you are not even told why. There’s nobody accountable and there’s nobody to complain to. A technical dysfunction got papered over with process.

***

Holocaust researchers keep stressing one point: The large-scale genocide was possible only by turning the popular hatred, that would otherwise discharge in few pogroms, into a formalized administrative process.

For example, separating the Jews from the rest of the population and concentrating them at one place was a crucial step on the way to the extermination.

In Bulgaria, Jews weren’t gathered in ghettos or local “labor camps”, but rather sent out to rural areas to help at farms. Once they were dispersed throughout the country there was no way to proceed with the subsequent steps, such as loading them on trains and sending them to the concentration camps.

Concentrating the Jews was thus crucial to the success of the genocide. Yet, bureaucrats working on the task haven’t felt like they were personally killing anybody. They were just doing their everyday, boring, administrative work.

The point is made more salient when reading about Nuremberg trials. Apparently, nobody was responsible for anything. Everybody was just following the process orders.

To be fair, the accused often acted on their own rather than following the orders. And it turns out that the German soldiers faced surprisingly mild consequences for disobeying unlawful orders. So it’s not like the high-ups would be severely hurt if they just walked away or even tried to mitigate the damage.

Yet, the vague feeling of arbitrariness about Nuremberg trials persists. Why blame these guys and not the others? There were literally hundreds of thousands involved in implementing the final solution. The feeling gets even worse when contemplating German denazification trials in 1960’s:

Today they bring even 96-year-olds or even centenarians to justice just because they worked somewhere in an office or sat in a watchtower. But if it had been done like that in 1965, more than 300,000 German men and women would have had to be imprisoned for life for aiding and abetting murder. […] It had to be kept under control and suppressed, because otherwise it would have been impossible to start anew. […] There was a secretary at the Wannsee Conference and no one considered putting her in jail. And yet all these terrible orders were not typed by those who issued them. They were typed by women.

— Interview with historian Götz Aly (in German)

***

At the first glance, this might seem like a problem unique to large organizations. But it doesn’t take a massive bureaucracy to end up in the same kind of dysfunction. You can get there with just two people in an informal setting.

Say a couple perpetually quarrels about who’s going to do the dishes. To prevent further squabbles they decide to split the chores on weekly, alternating basis.

Everything works well, until one of the spouses falls ill. The dishes pile up in the kitchen sink, but the other spouse does not feel responsible for the mess. It’s not their turn. And yes, there’s nobody to blame.

***

This is what Dan Davies, in his book The Unaccountability Machine: Why Big Systems Make Terrible Decisions — and How the World Lost Its Mind, calls “accountability sinks”.

Something that used to be a matter of human judgement gets replaced by a formal process. Suddenly, nobody makes any deliberate decisions. Instead, a formal process is followed. In fact, the process may be executed on a computer, with no human involvement at all. There’s nobody to blame for anything. There’s nobody to complain to when the process goes awry. Everybody’s ass is safely covered.

In