TESCREALism and the Race to Build an AI God

(1,500 words)

I’m happy to say that I just submitted a final draft of my article (somewhat boringly) titled “TESCREAL” to the Oxford Research Encyclopedia of Science, Technology, and Society. It’s very long — about 16,000 words — but it’s also very detailed and comprehensive. If you’d like to look over a penultimate draft, you can do so here.

In it, I explain what the TESCREAL ideologies are — Transhumanism, Extropianism, Singularitarianism, Cosmism, Rationalism, Effective Altruism, and Longtermism — and argue that the ongoing race to build God-like AI systems directly emerged out of the TESCREAL movement. Hence, if one wants to understand how we ended up with the current AI models flooding our information ecosystems with slop, junk, clutter, and garbage, one really needs some understanding of the TESCREAL ideologies.

Also: the amazing journalist Taylor Lorenz just published an hour-long video about the pro-extinctionist views that have become popular in Silicon Valley. Very relevant to the TESCREAL worldview. I think it’s a fantastic tour of the territory, and it draws from my work quite a bit at times! It’s thanks to you — everyone who supports this newsletter — that I’m able to work on these topics. :-) If you’d like to know more about Silicon Valley pro-extinctionism, see my newsletter series here. For a more academic treatment, see this article of mine recently published in the Journal of Value Inquiry.

I also just started writing my next book, with the tentative title Digital Deities: Silicon Valley’s Quest to Replace Humanity with AI. (Do you like it? Any suggestions for something better?) I’m going to ask the publisher if I can share drafts of chapters via this newsletter — hopefully I’ll get a thumbs up, because I’d really love to get your feedback while I’m in the midst of crafting the document. As of now, I think the overall structure of the book will parallel that of the Oxford Research Encyclopedia entry. Here’s a brief overview, if you’re curious!

Introduction. (One chapter.) This will offer a synoptic survey of the topic and an explanation of why it matters. The world is replete with bizarre, absurd, ridiculous, and potentially dangerous ideologies. Most of these aren’t worth talking about because they have no power or influence.

In contrast, the TESCREAL worldview — with its grandiose eschatological aim of establishing a cosmic utopia full of immortal posthumans spread throughout the cosmos by building an AI God or Digital Deity that will realize this future for us — has been embraced and promoted by some of the wealthiest and most powerful people on Earth, including Elon Musk, Peter Thiel, Sam Altman, and Marc Andreessen. Hopefully, this opening chapter provides a compelling case for why reading the book is worth one’s precious time!

Part I, The Vision. (Three or four chapters.) This will dive into the gory details of the alphabet soup of TESCREAL ideologies. I’ve described transhumanism — the idea that we should develop advanced technologies to radically reengineer humanity, thus creating a new “posthuman” species — as the “backbone” of the TESCREAL bundle. All the other ideologies are either variants of transhumanism or directly arose from the modern transhumanist movement. Hence, if you understand transhumanism, you’ve got a pretty good understanding of what the TESCREAL worldview is all about.

I’ll also explain how AI doomerism and AI accelerationism fit into the picture: these are just two camps within the broader TESCREAL movement. Both agree that we should build superintelligent machines as quickly as possible; the doomers just think we aren’t ready yet. Hence, we must “pause” or “stop” AGI from being built in the near future. Accelerationists think this is nonsense: the default outcome of AGI will be utopia rather than annihilation. Part I unpacks these issues, providing a solid foundation of shared knowledge upon which the rest of the book will be built.

Part II, The Gamble. (Probably four chapters.) This will offer a detailed look at how the big four AI companies — DeepMind, OpenAI, Anthropic, and xAI — all emerged out of the TESCREAL movement.

Demis Hassabis, the cofounder of DeepMind, got money to start the company from Peter Thiel after giving a talk at the 2010 Singularity Summit (video is below). Sam Altman is a transhumanist who the New York Times describes as being a “product” of the Rationalist and Effective Altruist (by which they mean “longtermist”) communities. The core Anthropic team is a bunch of EA-longtermists who used to work for OpenAI but came to believe that Altman wasn’t doing enough to ensure that AGI brings about utopia. And xAI was founded, of course, by Musk, a transhumanist who calls longtermism “a close match for my philosophy.”

I will claim that, if not for the TESCREAL ideologies, there would be no AGI race right now. These companies would never have existed, and we might not even have had the generative AI systems that are now destroying society. (The whole reason these “AI” systems were created in the first place was because researchers believed they might be the stepping stones to AGI — and thus a techno-paradise among the stars.)

Part II of the book will also provide a comprehensive mapping of the complex funding channels through which super-wealthy individuals, including folks like Peter Thiel and Jaan Tallinn, have financially supported TESCREAL-aligned organizations, think tanks, institutes, and companies. (A good resource here is Openbook.fyi. If you’re not familiar, it allows you to search organizations and wealthy donors within the TESCREAL ecosystem. Very useful, and good to be aware of!)

Part III, The Outcome: This is my opportunity to distill all the criticisms I’ve made over the past 5 years targeting the TESCREAL worldview. I’ll argue that the ongoing attempt to create utopia is extremely dangerous. The reason is that “utopia” can justify virtually anything: if the stakes are sufficiently large — literally trillions of “digital people” in a sprawling multi-galactic civilization billions of years from now — then there’s no line that can’t be crossed; there’s no moral norm that can’t be violated.

Indeed, it’s this utopian promise that enables the evil AI companies to “justify” — if only in their own minds — the fact that they’re destroying the educational system, the rule of law, and the free press. Sure, things might get worse for most people in the near term, but that’s a cost worth suffering given the utopian “benefits” that await once AGI arrives.

A great video on how generative AI is destroying the educational system. Highly recommended:

I’ll also argue that if TESCREALists were to successfully create utopia, this itself would be utterly catastrophic for humanity. The argument here is that “utopia” is an inherently exclusionary concept. Someone is always left out of utopia — otherwise it wouldn’t be utopia.

So, the question one must always ask when confronted with utopian proclamations is: who is excluded from enjoying the paradisiacal delights being promised? With respect to the utopia of TESCREALism, humanity itself would be excluded. There is no place for our species in a world run and ruled by digital posthumans. This is precisely why I’ve been screaming about Silicon Valley pro-extinctionism for so long — and why I’m thrilled that folks like Taylor Lorenz are starting to talk about it. Many of these people really do want to replace humanity with AGI or “superintelligence,” not centuries from now but within our own lifetimes.

Part IV, The Revolt: Finally, this will outline what we can do to fight back against the tech oligarchs and the TESCREAL fantasies that motivate them. Surveys consistently show that the public is fed up with AI. In a sense, my anti-generative-AI side has already won the public debate. What’s missing is an organized movement that can direct all this anger and frustration toward reeling in and regulating the AI companies. There are already some movements working on achieving this aim, such as Stop AI. But, hopefully, my book will inspire more people to take a stand against the reckless race to build a Digital God.

Part IV will also explore alternative visions of the future. My friend Monika Bielskyte likes to say, “You cannot design for, you must design with.” I completely agree, and hence will argue that we need to ensure that representatives from every nation, culture, tradition, religion, demographic, and group has a seat at the decision-making table. (Here I’m reminded of Altman once saying: “Because if I weren’t in on this I’d be, like, Why do these fuckers get to decide what happens to me?” Exactly, dude.) I do not know what the future should look like, but I do know how to find out: by ensuring that everyone has a say through an inclusive and democratic process.

What do you think? This is basically a very short summary of Chapter 1, which I wrote last Wednesday in a burst of energy. I’m really excited about this project — although I’m also so freaking tired of thinking about the topic (!!). But it needs to be written. At this rate, I could probably have the book completed in two months. That’s not entirely implausible, as I wrote my 2017 book Morality, Foresight, and Human Flourishing in something like three weeks! There aren’t many things in life that I’m good at, but I am fairly good at becoming hyper-focused on, and monomaniacal about, writing projects. :-)

If you get a chance to read my Oxford Research Encyclopedia article, I’d love to know your thoughts on that, too. As always:

Thanks for reading and I’ll see you on the other side!