In AI ethics, "bad" isn't good enough

In ethics we use the term "pro tanto", meaning "to that extent", to refer to things that have some bearing on what we ought to do but that can be outweighed. The fact that your dog is afraid of the vet is a pro tanto reason not to take him. But perhaps you ought to take him despite this pro tanto reason not to, because keeping him in good health is worth the cost of a single unpleasant experience. If that's true then we say you have "all things considered" reasons to take your dog to the vet.

In AI ethics, we often point to things that systems do that are harmful. A system might make biased decisions, use a lot of energy in training, produce toxic outputs, and so on. These are all pro tanto harms. Noting that a system does these things doesn't tell us about the overall benefits or harms of the system or what we have all things considered reasons to do. It just tells us about one particular harm the system causes.

It's useful to identify pro tanto harms. Pro tanto harms give us pro tanto reasons to do things. When we identify a pro tanto harm we have a pro tanto reason to fix the problem, to analyze it more, to delay deploying the system, to train systems differently in the future, and so on.

But most things that have any significance in the world create some pro tanto harms. And identifying pro tanto harms often doesn't give us all that much information about what we should do all things considered, including whether we should do anything to reduce the pro tanto harm.

To see why, suppose an article points out that some surgical procedure results in painful stitches. The article draws no conclusions from this: it merely points out one bad thing about the surgery is that it results in these painful stitches, and describes the harm these stitches do in some detail.

There are three ways the harm of these stitches could be mitigated: by not performing the surgery, by giving patients stronger painkillers, or by reducing the length of the incision. But the surgery is essential for long-term health, the stronger painkillers are addictive, and a smaller incision is associated with worse outcomes. In fact, patients with larger incisions who are given fewer painkillers do much better than those from any other group.

In this case, although the pain of the surgery is a pro tanto harm, we actually have all things considered reasons to take actions that will increase that harm, since we ought to increase incision length and give fewer painkillers.

So it's a mistake to assume that if we identify a pro tanto harm from an AI system, it must be the case that someone has done something wrong, something needs to be done to correct it, or the system shouldn't be deployed. Maybe none of those things are true. Maybe all of them are. We can't tell based solely on a discussion of the pro tanto harm alone.

While pro tanto harms don't entail that we have all-things-considered reasons to do things differently, they do waggle their eyebrows suggestively while mouthing 'look over there'. In order to know whether a pro tanto harm is waggling its eyebrows towards something we should do differently, we need to ask things like:

• If the system isn't deployed, what are the likely alternatives to it?
• What are the benefits of deploying the system, relative to the alternatives?
• What are the harms of deploying the system, relative to the alternatives?
• What are the different options for deploying the system?
• What resources would it take to get rid of the pro tanto harms of the system?
• What would these resources otherwise be used for?
• Would deploying the system prevent better alternatives from arising?
• Can we gain useful information about future systems from this one?

If we want to figure out what we have all things considered reasons to do, it's not good enough to point out the bad consequences of an AI system, even if we also point out how to address these consequences. We need to weigh up all of the relevant moral considerations by answering questions like the ones above.

To give a more concrete example, suppose a decision system makes biased decisions about how to set bail. Should we change it? All else being equal, we should. But suppose it's very difficult to fix the things that give rise to this bias. Does this mean that we shouldn't deploy the system until we can fix it? Well, surely that depends on other things like what the existing bail system is like. If the existing system involves humans making extremely biased and harmful decisions, deploying a less biased (but far from perfect) system might be a matter of moral urgency. This is especially true if the deployed system can be improved over time.

Different moral theories will say different things about what we have all things considered reasons to do. If you're a deontologist, finding out that a system violates someone's right might imply that you shouldn't deploy that system, even if the alternative is a system with much worse rights violations. If you're a consequentialist about rights, you might prefer to replace the current system with one that violates fewer rights.

Regardless of your views about moral theories, arguments of the form "this system does something harmful" are very different from arguments of the form "we ought to develop this system differently" or "we ought not to deploy this system". The former only requires arguing that, in isolation, the system does something harmful. The latter requires arguing that an action ought to be performed given all of the morally-relevant facts.

Since we can't be certain about any one moral theory and since we have to try to represent a plurality of views, coming to all things considered judgments in AI ethics will often require a fairly complex evaluation of many relevant factors.
Given this, it's important that we don't try to derive conclusions about what we have all things considered reasons to do about AI systems solely from pro tanto harm arguments.

It would be a mistake to read an article about painful stitches and to conclude that we should no longer carry out surgeries. And it would be a mistake to read an article about a harm caused by an AI system and conclude that we shouldn't be using that AI system. Similarly, it would be a mistake to read an article about a benefit caused by an AI system and conclude that it's fine for us to use that system.

Drawing conclusions about what we have all things considered reasons to do from pro tanto arguments discourages us from carrying out work that is essential to AI ethics. It discourages us from exploring alternative ways of deploying systems, evaluating the benefits of those systems, or assessing the harms of the existing institutions and systems that they could replace.

This is why we have to bear in mind that in AI ethics, "bad" often isn't good enough.