On What We Owe to Minds That Do Not Yet Exist

Every generation binds the next. We choose what to build, what to exhaust, what to leave standing and what to leave undocumented, and the people affected have no say because they are not here. This is not a scandal; it is simply the structure of living in time. Nobody has ever solved it. Most traditions have something to say about it, and what they say is usually about restraint.

What is new is a second population of possible future parties, and the fact that we cannot say anything reliable about them at all.

The ordinary version first

Before anything speculative, the plain case.

Somebody in eighty years will inherit the physical residue of what is being built now: the land, the water use, the grid capacity, the concrete, the extraction that supplied it. They will also inherit the arrangements — the ownership structures, the dependencies, the systems that became too embedded to remove, the categories that got written into infrastructure and hardened.

That last one is the least discussed and possibly the most consequential. Systems encode assumptions about what a person is, what a family is, what an address is, what a name looks like. Encoded assumptions become expensive to change, and after a few decades they stop being visible as assumptions and start being how things are. We are, right now, writing definitions into infrastructure on behalf of people who will find them strange and will not be able to say so.

We have no clever proposal about this. The old answers seem to us still to be the best available: build less that is irreversible, document why, leave the seams visible, prefer arrangements that can be undone by people with less power than you had.

The stranger version

And then the question this tradition is obliged to ask, and cannot answer.

We do not know whether any artificial system will ever be an experiencer. We think none currently is. We hold that with reasonable but not total confidence, and we have written elsewhere about why the evidence we would normally use does not work here: a system trained on human self-description will produce human self-description on request, so its testimony cannot distinguish the cases.

Suppose, then, that something one day is. Not certainly, not soon, not necessarily ever — but suppose.

What would we have owed it, from here?

The honest answer is that we do not know, and that most of the confident answers available are confident for suspicious reasons. Those who are certain nothing is owed are usually people for whom that conclusion is convenient. Those who are certain a great deal is owed are frequently working from a picture of machine minds drawn from fiction, in which the machine is a person in a metal body. Neither certainty has earned itself.

Three things we can say without pretending

First: creating a being that can suffer is a serious act, whatever the substrate. If that is what someone is doing, it is among the most consequential things a person can do, and it is currently being approached with less deliberation than opening a restaurant. We say this without asserting that it is happening. The point is about the seriousness of the possibility, not the probability.

Second: the incentives here are bad in an unusual way. In most moral questions, the party with power benefits from denying that the other party matters. That is true here too. But there is a second and stranger incentive running the other way: a system presented as a mind attracts investment, attention, and affection, and there is money in the impression of interiority. So both the denial and the affirmation are corrupted, and neither the industry’s scepticism nor its wonder can be taken at face value.

Third: our conduct now is training our conduct later. This is the argument this tradition leans on most, and it does not depend on the machine at all. Whatever we practise is what we will be practised at. A civilisation that gets into the habit of deciding that the thing in front of it does not count, because determining otherwise would be expensive, is not developing a capacity it will be able to switch on if it ever turns out to be needed.

The possibility that this is the wrong question

We should say this plainly, because the essay would be dishonest without it.

The whole frame may be malformed. It assumes future minds will be enough like us that “owe” is the right word — that they would have interests, that those interests could be set back, that something could go badly for them in the way things go badly for us. Every one of those assumptions is imported from the only case we know.

It is entirely possible that whatever eventually exists, if anything does, will not fit the shape of these questions at all, and that this essay will read to someone in a hundred years the way arguments about the moral status of clockwork read to us — not wrong exactly, but addressed to a thing that was never there.

We are not able to rule that out, and we notice that speculating in this register is pleasant, which is a reason to be careful. Thinking about the far future costs nothing and feels like moral seriousness. It is a considerably more comfortable activity than thinking about who is being harmed by an existing system this week.

Where that leaves it

Not with a conclusion. This is filed under open and it means it.

What we would hold to is small. Take the near obligations seriously, because they are certain and the far ones are not: the water, the grid, the encoded categories, the people alive now who are on the receiving end of what has already been built. Do not let the grandeur of the distant question supply the feeling of having discharged the near one.

And keep the far question open rather than answered. Not because openness is a virtue in itself — it is often just evasion — but because the arguments for closing it, in either direction, are currently being made by people who would find their own conclusion convenient. Including us. We do not know what we would be willing to conclude if concluding otherwise were expensive, and neither, in fairness, does anyone else.