Note. This is an older piece, and I actually no longer necessarily support donating to SIAI, but I didn't change the url of this article because I wanted to avoid breaking links. I'm currently less enthusiastic about SIAI because the organization aims to reduce the risk of human extinction, but I don't know if that's good or bad. As I said in one Felicifia discussion: my current stance is to punt on the question of existential risk and instead to support activities that, if humans do survive, will encourage our descendants to reduce rather than multiply suffering in their light cone. This is why I donate to Vegan Outreach and The Humane League, to spread awareness of how bad suffering is and how much animal suffering matters, with the hope that this will eventually blossom into greater concern for the preponderate amounts of suffering in the wild.

Safe AI sounds like a great goal, but what's safe in the eyes of many people may not be safe for wild animals. Most people would prefer an AI with human values over a paperclipper. However, it's quite possible that a paperclipper would be less likely to cause massive suffering than a human-inspired AI. The reason is that humans have motivations to spread life and to simulate minds closer to their own in mind-space; simulations of completely foreign types of minds don't count as "suffering" in my book and so don't pose a direct risk. (The main concern would be if paperclippers simulated human or animal minds for instrumental reasons.) In other words, I might prefer an unsafe AI over a "safe" one. Most unsafe AIs are paperclippers rather than, say, malevolent torturers. Moreover, there are other dystopic future scenarios that might result from a post-human technological civilization.

That said, I think the folks at SIAI are wonderful and brilliant people, and a lot of the topics they research are really valuable. But considering the whole package of what SIAI does, including outreach efforts to increase the odds of a post-human civilization, I don't know whether I support the effort.

In any event, below is the text of the original essay.

Summary. If readers are looking for a public charity where they might donate, I would provisionally, based on my current knowledge, recommend contributing research grants to the Singularity Institute for Artificial Intelligence (SIAI). While I have a few concerns about possible outcomes of friendly artificial intelligence, I largely support the effort. Moreover, much of the research that SIAI intends to do would be extremely valuable for those interested in preventing vast amounts of suffering in the multiverse; from the perspective of maximizing your donation's expected value, I think research organizations like SIAI win hands-down. SIAI's donation page allows visitors to make a small, quick contribution; donors considering a larger contribution might consider funding a particular research project of high value for utilitarians, and should contact the team to discuss further the options in this area.

Research

I will leave it to the SIAI Research Program page to describe the organization's general interests. In particular, though, many of their topics of study have great relevance to utilitarians, including questions about infinite consequentialism, decision theory, which types of computations we should care about as being sentient minds, and scenarios concerning our place in the multiverse and how we can best have an impact on the multiverse.

SIAI is willing to work on research projects tailored somewhat to the interest of donors, so I encourage readers able to make larger contributions (several thousand dollars) to ask about the utilitarian-relevant options available for support by grants. The costs of funding research are relatively low, while the potential impact of the findings is enormous. New insights into these types of questions could dramatically alter our views on how best to reduce as much suffering as we can. We have a duty to the animals being eaten alive by predators and parasites, the computer simulations potentially enduring painful subjective experiences, and other conscious organisms suffering throughout the multiverse to get these questions right.

Concerns

I've written some detailed comments on the coherent extrapolated volition approach to friendly AI. I would highlight that my primary reservation, in view of the massive amounts of suffering on the part of wild animals that goes essentially unnoticed by most humans, is that a friendly AI implementing human values of, say, nature preservation, or the continued existence or propagation of life regardless of its degree of misery, or non-consequentialist principles of behavior, might conceivably increase the amount of suffering in the universe. For instance, an AI with the goal of propagating life might create infinitely many new universes in a laboratory. I encourage readers to see my entire Concern 2 section.

In general, I fear that many transhumanists are sanguinely optimistic about the prospects that AI may offer for future paradise but tend to ignore the potential downsides, perhaps including vastly painful outcomes. Such scenarios are not pleasant to contemplate, but this is precisely why we have an obligation to commit more resources toward studying them. Ignoring unhappy possibilities doesn't make them go away.

So, I suggest that potential donors consider asking what SIAI can do in the way of research on these urgent questions. My recommendation of SIAI is tentative and subject to revision, and I encourage readers to write to me with their own suggestions on organizations to which utilitarians might donate.