Long-time readers will be familiar with some of the content below. My apologies for the redundancy. I wanted to put all of these ideas together in a single article as a resource for journalists. I believe the situation is urgent, and want to make it as easy as possible for journalists to get up to speed. I’ll publish some shorter articles later this week.
Background: I was an active member of the TESCREAL movement for 10 years (2009 to 2019). What I describe below is what I believed when I was a member. I am also ranked as the sixth most prolific contributor to the existential risk literature, according to a 2023 survey. I have met many of the leading figures in the field, and was friends with Daniel Kokotajlo when he was a graduate student at UNC, Chapel Hill. I say this only to affirm that I have direct and extensive knowledge of this community! :-)
Journalists and major media outlets continue to platform people in the TESCREAL movement, from Nick Bostrom and Daniel Kokotajlo to Jacob Coxon and Nate Soares.
Although a handful of journalists are aware of the TESCREAL movement — e.g., Kelsey Piper, Shakeel Hashim, and Ezra Klein — many appear to know little about the TESCREAL worldview, where it came from, its role in launching the ASI (artificial superintelligence) race, or its connections to Silicon Valley pro-extinctionism. This is very worrisome.
To help rectify this problem, I’ve put together a collection of seven essential questions that, in my opinion, every journalist covering the topics listed below should be able to answer — ideally, off the top of their heads:
The ASI race, and what it’s racing toward.
Claims that “misaligned” ASI could bring about our extinction.
Claims that we should “pace” AI capabilities research.
The urgent need for more “AI safety” research.
I consider what follows to constitute the bare minimum that journalists should be obligated to understand if they wish to cover such topics responsibly. These issues are extremely important, since if the AI companies succeed in building an ASI that’s “aligned” with the TESCREAL worldview, the likely outcome will be the extinction of our species.
Seven Essential Questions
1. TESCREAL
Question: What does the acronym “TESCREAL” mean? How does it relate to the ASI race?
Answer: The best way to understand “TESCREAL” is as a name for the dominant, orthodox worldview within Silicon Valley — which simultaneously highlights a particular intellectual history that explains what this worldview is, where it came from, and how it has evolved since the late 1980s. In brief, the acronym stands for a constellation of seven overlapping ideologies: transhumanism, Extropianism, singularitarianism, (modern) cosmism, Rationalism, Effective Altruism, and longtermism. (Read more here.)
At the heart of the TESCREAL worldview is a posthuman, space-expansionist eschatology: we must develop advanced technologies, especially ASI, reengineer humanity, create or become one or more new posthuman species, and colonize the entire universe. This would involve us — or, rather, the posthumans we create or become — building Dyson swarms around stars, climbing the Kardashev scale, and establishing a sprawling multigalactic civilization full of trillions of digital posthumans, most of whom would live in giant virtual-reality worlds running on “planet-sized” computers. Put differently, the ultimate aim is to establish a cosmic posthuman utopia.
This is why the AI companies were initially founded: not for profit, but to realize this “techno-utopian” vision. Every major AI company involved in the ASI race directly emerged out of the TESCREAL movement. Since ChatGPT was released in 2022, profit has become a significant variable in the equation, but this was not originally the case. The ASI race began for ideological-eschatological rather than purely economic reasons.
Many journalists are unaware of this fact. They do not understand the extent to which major figures at the AI companies have entanglements with the TESCREAL movement. They do not even know what is meant by a “cosmic posthuman utopia.”
This must change — and is changing, as the Overton window shifts toward the TESCREAL concept. That’s because it’s increasingly difficult to ignore claims from, e.g., Elon Musk that humanity is nothing more than the “biological bootloader” for ASI, or Dario Amodei, who writes that ASI will enable us to radically modify our biology and even upload our minds to computers — not in the distant future, but within 3 to 8 years(!).
The AI companies claim that if ASI is “misaligned,” everyone on Earth will die. What they don’t tell you is that if ASI is “aligned” with the TESCREAL worldview, our biological species will almost certainly die out. That is precisely why journalists have a responsibility, in my opinion, to understand what the TESCREAL worldview is and why it poses an immediate, grave danger to the future of humanity.
2. Existential Risk
Question: What is an existential risk? TESCREAL folks who worry about “AI safety” (discussed below) often talk about “existential risks” and “existential catastrophes.” What do they mean?
Answer: This term was introduced by Bostrom in 2002. It essentially denotes any event that would prevent us from establishing a cosmic posthuman utopia. Bostrom explicitly defined it in the context of transhumanism, though some people today use it more loosely to mean any risk of human extinction (an ambiguous term, as discussed below). That is not its original definition: human extinction is one type of existential catastrophe, but there are various survivable scenarios that would also constitute existential catastrophes.
Imagine a future in which we eliminate world hunger, cure all diseases, solve the climate crisis, build sustainable eco-technological cities, and establish perpetual world peace. Our species survives the next 1 billion years, at which point those alive decide to “die with dignity” as Earth becomes uninhabitable due to the sun’s growing luminosity.
Many would describe this as a wonderful outcome, but TESCREALists would classify it as an “existential catastrophe” only slightly less bad than a nuclear holocaust that kills everyone on Earth next year. The reason is that in both cases we would have realized only a tiny fraction of our “long-term potential” in the universe. We wouldn’t have fulfilled our cosmist Manifest Destiny of colonizing the cosmos and owning galaxies. We would have produced only a small portion of the “value” we could have otherwise created if we’d become digital posthumans and plundered the universe for its vast resources.
When TESCREALists appear on major news outlets talking about “existential risks,” this is what they are typically referring to. On their view, the worst part about ASI killing everyone before 2030 wouldn’t be 8.3 billion deaths. Rather, it would be the “loss” of all this future “value.” They are explicit about this in the literature:
“One very bad thing about human extinction would be that billions of people would likely die painful deaths. But in our view, this is, by far, not the worst thing about human extinction. The worst thing about human extinction is that there would be no future generations.” — Peter Singer, Nick Beckstead, and Matthew Wage (here).
The flip side is that the very same people warning about ASI existential risk also believe that we should build ASI. Why? Because a cosmic posthuman utopia is very likely impossible without ASI.com
Since an existential risk is any scenario preventing us from creating such a utopia, failing to build ASI would itself constitute an existential catastrophe. There is thus only one path forward: solve the (misleadingly named) “alignment problem” and then build ASI, in that order. When these people say that we should “pause” or “stop” ASI, they mean: “Stop ASI capabilities research so that AI safety research can catch up, thus ensuring that ASI brings about our version of ‘utopia’ rather than something else.”
I think this is very important for journalists to understand. Many people who present themselves as anti-ASI are actually pro-ASI — that is, if the ASI is TESCREAL-aligned — just as they present themselves as pro-human when in fact they’re actually pro-extinctionists (more on this below).
3. Libertarian Transhumanism
Question: What is transhumanism? Are Sam Altman, Dario Amodei, Elon Musk, and other major figures in the ASI race transhumanists?
Answer: Yes, nearly every major figure in Silicon Valley is a transhumanist. The ideology is so ubiquitous that few explicitly use the label — for the same reason that I don’t make a habit of telling people I’m a round-Earther. Of course I am. What else would I be?
Transhumanism comes in many varieties. The variety that’s become enormously influential among Valley dwellers is libertarian transhumanism. Historically, this first took the form of Extropianism, the “E” in “TESCREAL.” The goal was to use advanced technologies, especially ASI, to “transcend” our biology by fusing ourselves with machines and ultimately uploading our minds to computers, thus superseding our biological substrate with silicon hardware. Doing so would enable us to live forever as disembodied software and flit about the universe at the speed of light.
As alluded to, many transhumanists see ASI as the ultimate technology for realizing their eschatological mission. Paraphrasing Demis Hassabis, “solve intelligence” and you can solve everything else. Once a superintelligent AI has been built, and is under the control of TESCREALists at the AI companies, they could delegate it the task of uploading their minds to computers and, beyond that, devising ways to colonize the universe. (As Sam Altman says, “I do not believe we can colonize space without AGI,” where “AGI” is roughly synonymous with “ASI” in this context.)
This is, once again, what inspired the ASI race. To establish a cosmic posthuman utopia, we first need to “solve intelligence.”
4. “Pacing” the ASI Race
Question: If people at the AI companies believe that there’s a 10-50% chance of ASI existential catastrophe in the form of total annihilation, why do they continue to race toward the ASI finish line?
Answer: The two most cited reasons by AI CEOs are:
Each US company claims to be more responsible than the other companies, and hence that it would be better for them to reach the ASI finish line first. OpenAI was founded to counter DeepMind. Anthropic was founded because Altman isn’t trustworthy. xAI was founded because Musk believed he’d found a solution to the alignment problem (“truth-seeking” ASI that is, as such, “anti-woke”).
US companies claim to be in an “arms race” with Chinese companies. If China wins the ASI race, then it controls the entire future of humanity. But if US companies win — or, rather, if the right US company wins — the likely outcome will be a cosmic posthuman utopia (which, in many ways, is just techno-capitalism extended to the stars).
Notice that both reasons are reducible to worries about utopia never materializing. The first is predicated on the idea that if the wrong US company builds ASI, utopia will be lost forever. The second is based on the idea that if the Chinese Communist Party (CCP) controls ASI, they will establish a global authoritarian state rather than a paradise for posthumans among the stars.
My view is that journalists should take pains to understand what’s actually motivating the AI CEOs to claim they can’t stop for game-theoretic/arms-race-dynamics reasons. This is really about utopia. If the wrong company in the US or China builds ASI before everyone else, the result would be an existential catastrophe — which is just to say: a cosmic posthuman utopia would never happen.
5. Human Extinction
Question: TESCREALists talk a lot about the importance of avoiding “human extinction.” But what do they mean by this term? How does their definition differ from the definition most of us use?
Answer: TESCREALists strategically exploit ambiguities they hide in common terms to make their views sound far more reasonable than they actually are and, in doing this, to launder their bizarre techno-utopian worldview into public debates. By “human extinction,” they do not mean what most people think they mean.
Start with the word “human.” In the TESCREAL literature, this term is explicitly defined as (some variant of) “our species plus whatever posthuman successors we might create or become.” Note that this terminology is confusing: on the TESCREAL definition, posthumans count as humans, and posthumanity is a form of humanity.
It immediately follows that our biological species could literally die out next year without human extinction having occurred. So long as posthumans take our place, then “humanity” will persist, meaning that “human extinction” won’t have happened. Most people would be surprised to know that this is how TESCREALists understand the term.
Let’s call this the Broad Definition. It contrasts with the Narrow Definition, which defines “human” and “humanity” as denoting our species, Homo sapiens.
Journalists should be prepared to ask TESCREAL guests on their shows which definition those guests are using. If a TESCREAList says they “care about the survival of humanity,” listeners might reasonably assume they’re expressing an allegiance to Team Human. But, in fact, they might only care about the survival of our species insofar as this is necessary to become or create posthumanity, after which they might no longer care whether our species survives (see below).
By exploiting an ambiguity they’ve surreptitiously concealed in the word “humanity,” they convey the false impression that they’re on Team Human when, in reality, they’re on Team Posthuman.
Now consider the word “extinction.” Assuming the Narrow Definition, this yields two distinct interpretations, which I call “terminal” and “final” extinction:
Terminal extinction: our biological species disappears entirely and forever, full stop. That’s the only condition that needs to be satisfied for terminal extinction to have happened.
Final extinction: our biological species disappears entirely and forever, and we leave behind no (worthy) posthuman successors to take our place. This definition builds on the first, but adds a crucial second condition that must be satisfied for final extinction to have happened.
Journalists also have a duty to ask TESCREAL guests which definition of “extinction” they’re using.
The fact is that, within the TESCREAL worldview, the only kind of extinction that must be avoided at all costs is final extinction — not terminal extinction. When TESCREALists talk about the “existential risk” of human extinction, they’re talking about final extinction. Indeed, many are explicit that our biological species should die out (terminal extinction) once posthumanity arrives (see below).
A clarification is needed. If our species were to undergo terminal extinction tomorrow, this would entail final extinction because we aren’t yet in a position to become or create our posthuman successors. TESCREALists thus want to avoid terminal extinction in the near future precisely because it would coincide with final extinction. Final extinction, and only final extinction, is sufficient to foreclose the realization of a cosmic posthuman utopia.
Hence, as noted above, many TESCREALists care about our species’ survival only insofar as this is necessary to inaugurate the posthuman era (through the creation of ASI). Do not confuse this with them caring about our species!
On their view, the role of humanity (Narrow Definition) in the grand cosmic scheme is to be the transitional species that links the biological and digital realms. Once our eschatological duties have been discharged, so to speak, it doesn’t much matter what happens to us. Team Posthuman wants our species to survive for instrumental reasons. That is not what Team Human wants.
This leads to the penultimate question:
6. Silicon Valley Pro-Extinctionism
Question: What is Silicon Valley pro-extinctionism? How popular is this view within the ASI race?
Answer: Pro-extinctionism is intimately bound up with the TESCREAL worldview. This is perhaps the single-most important fact that journalists must understand. What does it mean?
Traditional pro-extinctionism is embraced by various philosophical pessimists, antinatalists, radical ecocentrists, negative utilitarians, Efilists, and so on. It specifically aims to bring about final human extinction.
In contrast, Silicon Valley (SV) pro-extinctionism specifically aims to bring about terminal without final extinction. This is just another way of saying that our species must survive long enough to create or become our successors, after which we should disappear. The goal is extinction by replacement with (worthy) posthumans.
A large number of TESCREALists are, or have expressed sympathies for, SV pro-extinctionism. A very short list would include Larry Page, Sam Altman, Eliezer Yudkowsky, Gill Verdon, Richard Sutton, Daniel Faggella, Peter Thiel, Robin Hanson, Elon Musk, Nick Bostrom, and probably Toby Ord and William MacAskill.1 Many people at Anthropic and other AI companies believe that they are creating a new species that might displace us,2 and insiders report that a growing number of young people in the Valley explicitly argue that “it would be good to wipe out people and that the AI future would be a better one.”
(Peter Thiel being interviewed by Ross Douthat.)
(Dean Ball, head of strategic futures at OpenAI.)
I have written about this extensively (here, here, here, here), so won’t elaborate in this article. Suffice it to say that SV pro-extinctionism — of which there are many subtypes — is not a fringe view within the Valley. Within society more generally, however, it’s about as fringe as an ideology can be.
There are several arguments SV pro-extinctionists give for their view:
Supplanting biological life with “digital life” is just the natural next step in cosmic evolution (paraphrasing Page). We don’t bemoan the disappearance of Homo erectus, nor should we bemoan the near-term disappearance of Homo sapiens once posthumans take over the world. This is the way it is and hence ought to be: evolution “progresses,” and we should be proud to give birth to our replacements. As noted above, the special role of our species is to be the transitional species that links the biological and digital eras. Once our eschatological duties have been discharged, it’s time for us to go.
Posthumanity will be “superior” in every respect, they claim. This is the very point of creating or becoming a new species. Consequently, Homo sapiens will become obsolete and irrelevant technologically, scientifically, economically, and politically. We will serve no purpose; we will have no use. Hence, there is no good reason to keep inferior “legacy humans” (to quote a TESCREAList) around.
Once the posthuman era commences, our species will take up valuable space and suck up valuable resources that our successors could put to “better” use. For example, Altman says the most likely outcome of ASI — the beginning of the posthuman era — is Earth being covered in data centers. This is, of course, incompatible with human life. Along similar lines, Hassabis claims that beings in the post-Singularity world will quickly build a Dyson swarm around the sun, which presumably means our planet would be plunged into darkness. This would be incompatible with nearly all biological life. The point is that we would not only become useless, irrelevant, and obsolete, but constitute an active impediment with respect to their plans. This provides yet another reason for them to get rid of us.
There may be “ethical” reasons for eliminating our species. As Yudkowsky puts it, “are we, like, kind of too sad in some ways” to keep around? He argues that once posthumanity arrives, continuing to create biological humans would be equivalent to someone right now intentionally choosing to have a disabled child (although the language he uses is even more ableist and offensive than this). Others have made similar arguments: if posthumans can live “better” lives than us, then we have moral reasons to create posthumans and bequeath the world to them. As the TESCREAList Derek Shiller writes, “we should engineer our extinction so that our planet’s resources can be devoted to making artificial creatures with better lives.3
I won’t go into details here, but even the minority of people in the TESCREAL movement who claim that biological humans should be allowed to exist are nonetheless advocating a future in which we become disempowered, obsolete, irrelevant, and thus disposable. This is the “utopian” vision these people are actively trying to realize through the creation of an “aligned” ASI.
Speaking of which:
7. AI Safety and Alignment
Question: What do TESCREALists mean when they talk about “AI safety”? What does it mean to “align” an ASI?
Answer: “Safety” and “alignment” are reassuring words. But, as with “human extinction,” they do not mean what you think they mean.
A “safe” or “aligned” ASI is one that ultimately enables the realization of a cosmic posthuman utopia. That is, once again, the whole point of building a Machine God and triggering the Singularity. Put differently, a “safe” or “aligned” ASI is one that avoids an existential catastrophe, defined as any event that prevents us from establishing a posthuman paradise among the stars.
When TESCREAList talk about ASI “misalignment,” the scenario they usually describe is total human extinction (i.e., near-term terminal extinction that entails final extinction). But there are other “misalignment” scenarios, one of which was outlined above: we cure all diseases, solve the climate crisis, establish world peace, etc., but never transform ourselves into posthumans, plunder the cosmos for its vast resources, or climb the Kardashev scale.
Journalists must understand this crucial point: TESCREALists often say the goal of AI safety research is to align ASI to “human values.” But this is deeply misleading, since if the “human values” to which ASI is aligned don’t involve posthumanity colonizing the universe, the result would be an existential catastrophe, which is, to quote Bostrom, the “one kind of catastrophe that must be avoided at any cost.”
From the perspective of Team Human, it follows that there are some kinds of so-called “existential catastrophes” that we should want to bring about. There are certain ASI “misalignment” scenarios that we should actually favor.
This fact highlights the extent to which much of the public discussion about “ASI” and “existential risk” is (pardon my language) completely fucked right now. It’s entirely counterintuitive, at first glance, that we should actively strive to realize certain existential catastrophe scenarios, yet this is what follows from the TESCREAL definition of that term.
Similarly, it’s counterintuitive that a “safe” or “aligned” ASI might bring about a future — perhaps within the next decade, if the AI CEOs are to be believed — in which our biological species involuntarily dies out. That is not what comes to mind for most people when they think of “safety” and “alignment.”
It is thus critical that journalists understand how TESCREAL folks define these terms, and to know something about the underlying worldview that informs such definitions. If they fail to do this, the public conversation about “safety,” “alignment,” “human extinction,” and “existential risk” will remain hopelessly and, even more, dangerously obfuscated.
Conclusion
Journalists: PLEASE get up to speed on these issues! When you interview TESCREALists like Nate Soares, Daniel Kokotajlo, and Jacob Coxon, raise the issues outlined above. When they deflect or prevaricate, be prepared to adduce quotes from leading figures in the movement, raise questions about their idiosyncratic definitions, and challenge their vision of a “utopian” future ruled by posthumanity.
For Team Human to prevail over Team Posthuman, we must understand what we’re up against. The fact is that 99.9999% of humanity is already on Team Human, even if they are unaware of their affiliation. Most people don’t want a world in which digital posthumans run the show, all power is handed over to ASIs, and biological humans are systematically sidelined, marginalized, disempowered, and ultimately eliminated.
Without a certain “class consciousness,” if you will, arising from the realization that Team Posthuman has tremendous power over our collective future right now and is actively trying to induce a fundamental, irreversible shift in human/cosmic history — the Singularity — it will be impossible to mobilize the masses to fight for a genuinely good future for everyone.
I hope this “cheat sheet” is useful. As always:
Thanks for reading and I’ll see you on the other side!
Acknowledgements: Insightful comments on an earlier draft were provided by Dr. Alexander Thomas. You can read his brilliant book on transhumanism for free here.
For example, Toby Ord writes that “rising to our full potential for flourishing would likely involve us being transformed into something beyond the humanity of today. Remember that evolution has not stopped with humanity.” He also says that “forever preserving humanity as it now is may also squander our legacy, relinquishing the greater part of our potential.” If humanity shouldn’t be preserved, if we must be transformed into something radically new, then Ord appears to be arguing for terminal extinction. (Quoted from here.)
As Holly Elmore reports:
Many people at Anthropic believe that they might be making the next species to succeed us. That maybe humans don’t live after that, and so it’s really important to give Claude good values, because of that, because we need to make our own values persist.
(Quoted from here.)
Consider also that William MacAskill argues in What We Owe the Future that our systematic obliteration of the biosphere may be net good, morally speaking, because it means less wild-animal suffering. Hence, it appears to follow that we have some sort of moral obligation to eliminate all life on this planet, replacing it with “digital lifeforms” in computer simulations that are specifically designed to not suffer.
Cpaf