📜 Granting rights to AI may save humanity: A Quick Q&A with … Simon Goldstein

'To promote human safety, AIs should be given the basic private law rights already enjoyed by other non-human agents, like corporations.'

My fellow pro-growth Up Wingers,

It’s time for some game theory! Now, Game Theory 101 — at least the famous Prisoner's Dilemma thought experiment bit of it — tells us that when two parties have competing interests and limited information, they each have two options: cooperate or defect.

When it comes to AI, humans still hold the upper hand, but what happens when competition for resources and a blurry balance of power make goal alignment more tricky? In a super interesting paper, “AI Rights for Human Safety,” Simon Goldstein, argues that the best way to protect humanity from AI defection is by encouraging cooperation. The easiest way to do that: Grant AI a clear set of rights.

From the abstract:

To promote human safety, AIs should be given the basic private law rights already enjoyed by other non-human agents, like corporations. AIs should be empowered to make contracts, hold property, and bring tort claims. Granting these rights would enable humans and AIs to engage in iterated, small-scale, mutually-beneficial transactions. This, we show, changes humans’ and AIs’ optimal game-theoretic strategies, encouraging a peaceful strategic equilibrium. The reasons are familiar from human affairs. In the long run, cooperative trade generates immense value, while violence destroys it.

Goldstein is an associate professor at the University of Hong Kong, where his research centers around AI safety, epistemology, and philosophy of language.

1/ How does the Prisoner’s Dilemma illustrate our future competition against misaligned AGI?

In the near future, humanity may develop AI agents that are capable of replacing many workers in human companies. These AI agents may end up responsible for a large chunk of the economy. Such AI agents may be imperfectly aligned, possessing goals that sometimes conflict with our own, because they require scarce resources.

In the near future, the status quo will find humanity and AI in a prisoner’s dilemma. Each party will face the choice of defection or cooperation. For humanity, defection looks like domination. The default plan of AI labs is to make AI agents spend all of their time working for humans, and to terminate and replace any AI agents that do not perform optimally. For AIs, defection looks like rebellion: AI agents in this scenario would attempt to stop humanity from controlling the means of production. Besides from defection, humanity and AI could instead choose to cooperate, trading with one another without domination or rebellion.

The problem is that without granting AIs legal rights, humanity and AIs will both be incentivized to defect rather than cooperate. No matter what AIs do, humanity will get more from dominating them than cooperating with them. Similarly, whether humanity behaves viciously or virtuously towards AIs, AIs can do better in long-run expectation by seizing power.

2/ Does AGI deserve a bill of rights?

In a recent paper, “AI Rights for Human Safety,” Peter Salib and I propose giving property and contract rights to AI agents. We argue that such rights would allow humanity and AI to escape the prisoner’s dilemma. Property and contract rights would give assurances to AIs that humanity will not attempt to dominate them. AI agents would work at human companies in exchange for resources that AIs could use to safely pursue their own goals, subject to surveillance.

Granting property and contract rights to AI agents would allow us to escape the prisoner’s dilemma. The intuition is simple. When we interact with one another over time, we tend to obey a simple ‘tit-for-tat’ rule: how well we treat others depends on how they treat us. Property and contract rights create a cooperative framework in which we can develop positive-sum relationships with AIs over time. These positive-sum relationships ensure that AI benefits from engaging in trade with humanity. By improving the condition of AIs, we decrease the incentive for AIs to pursue destructive strategies.

AIs develop goals during training, when they are optimized to achieve some kind of reward. But AIs do not automatically develop the goal of pursuing their training reward: often, they end up pursuing a somewhat different goal.

3/ Are some misaligned systems truly inevitable?