He suggested that I start a weekly newsletter. And so, on August 3, 2025, I published my first Realtime Techpocalypse article. Since then, I have published 107 articles, which amounts to slightly more than 2 per week. I am deeply indebted to Gil’s advice — if not for him, this newsletter wouldn’t exist! The least I can do in return is to implore those reading this to buy a pre-sale copy of Gil’s forthcoming book, The Nerd Reich. I haven’t read it yet, but I have absolutely no doubt that it’s amazing. Indeed, given the press that Gil has so far received, it looks like it will be a smash hit. If you have the money, please consider purchasing it asap! Pre-sales really help.

Thank you so much to everyone who’s subscribed to this newsletter. I am incredibly grateful for your interest in my thoughts about technology and, well, all-things-relating to the end of the world. Extra special thanks to those with the means and willingness to pay for this newsletter. It is not profitable opposing Silicon Valley and the TESCREAL ideologies. There are lots of anti-AI TESCREAL folks who are rolling in money from Open Philanthropy, Jaan Tallinn, and other funding entities. But there is almost zero funding for anti-AI folks who are also anti-TESCREAL.

Yet, in one year, I went from earning nothing from this newsletter to bringing in nearly half of the $20k that I need to pay all of my bills. I am deeply appreciative of your financial support, and if you have an extra $7 to spare each month, please consider becoming a paid subscriber. You can also support me on Patreon here, or on GoFundMe here. As I detail in “Why I Fled the US,” I spend virtually all of my money on three things: food (and I never eat out), accommodations (~$700/month or less), and a train ticket once a month to nomadically wander to a new city in Europe (~$60). I am basically an ascetic monk, lol.

Here’s a curated list of highlights from the last year, after which I’ve included a few poll questions about how you’d like to see this newsletter grow and evolve. Thanks again for making this possible, friends.

Subscribe now

The TESCREAL Ideologies (Intellectual History of Silicon Valley):

1. Skeptical of the TESCREAL Acronym? Read This. Probably the most accessible introduction to the TESCREAL thesis that I’ve written (even more so than my original intro published in 2023). “TESCREAL” is a term of intellectual history. It names a constellation of overlapping and interrelated ideologies that have given rise to the Silicon Valley worldview. This worldview, in turn, has launched, sustained, and accelerated the ongoing race to build God-like superintelligence. Highly recommended.

2. Is Transhumanism a Religion? Yes. This article explores how transhumanism was originally introduced as a secular replacement for traditional Christianity. It discusses how the Singularity is analogous to the rapture, and how transhumanism has its own version of resurrection: cryonics. I discuss many more similarities. Transhumanism — the backbone of the TESCREAL bundle — has become the orthodox religion of Silicon Valley.

3. Three Lies Longtermists Like To Tell About Their Bizarre Beliefs. These three lies are: (1) Longtermists care about avoiding human extinction. In fact, they only care about avoiding final extinction (the failure to create posthumanity), not terminal extinction (the disappearance of our species). (2) Longtermist care about the long-term future. If maximizing value requires a shorter rather than longer future, they will choose the shorter future. And (3) longtermists think that future people matter. In fact, people do not matter for their own sake, on the utilitarian-longtermist view. They are mere means to an end — that’s it.

4. Effective Altruism Is a Dangerous Cult — Here’s Why. Even people in the EA community worry that it’s a cult. I believe it’s a cult because I spent many years as an EA, and saw first-hand how cultish it is. Some call it the “Scientology of Silicon Valley.”

5. How Effective Altruists Use Threats and Harassment to Silence Their Critics. This re-examines some of the threats of physical harm and online harassment I’ve received from the EA cult for daring to challenge their views.

6. TESCREALism and the Race to Build an AI God. This introduces my Oxford Research Encyclopedia article on “TESCREAL.” I also argue that “if TESCREALists were to successfully create [their version of] utopia, this itself would be utterly catastrophic for humanity.” The most likely outcome of “utopia” would be (terminal) human extinction.

7. Is AI Apocalypticism Really that Different from Believing in the Rapture? Examines the similarities between religious and AI apocalypticism, i.e., the belief that the world as we know it will soon end. Spoiler: they are very similar!

8. From Trump to TESCREAL, Eugenics Is Everywhere. Eugenics is an idea that just won’t die. This article explores how it’s alive and well within the MAGA movement, as well as Silicon Valley (via the TESCREAL worldview).

   Subscribe now

Pro-Extinctionism in Silicon Valley:

1. One Half a Pro-Human Manifesto. This is essentially chapter 4 of my forthcoming book, Will Humanity Survive? I consider it one of the most important essays I’ve written recently (insofar as anything I write is important). It adduces a plethora of shocking examples in arguing that pro-extinctionist sentiments are ubiquitous within Silicon Valley, even among those who report being on the side of “humanity.” It also argues (in more detail) that the most likely outcome of realizing the utopian world envisaged by TESCREALists is that our species will die out. This, along with #1 above, is essential reading in my opinion.

2. The Perplexing Rise of Antinatalism in Silicon Valley. This examines the strange coexistence of pronatalism and antinatalism within tech culture. It introduces the term “replacement antinatalism,” in contrast to “traditional antinatalism,” to describe the growing demographic of young people who believe it’s “fundamentally unethical” to have children because AI will soon rule the world. Keep your eye on this trend moving forward!

3. If Anyone Becomes Posthuman, Everyone Dies Out. Explores what I call the “coexistence” and “replacement” views, which concern our potential future relationship with posthumanity. I show that the replacement view (i.e., pro-extinctionism) is widely held within Silicon Valley, and argue that the coexistence view will almost unavoidably lead, in practice, to the extinction of our species. Hence, if we want to preserve Homo sapiens — and I claim that we should — we must oppose the creation of posthumanity in all its forms. If anyone becomes posthuman, every human being will perish soon after.

4. The Scariest People in the World: Maniacs, Misanthropes, and Silicon Valley Pro-Extinctionists. Discusses “agential risks,” i.e., the risks arising from agents who would willingly destroy the world if only they could. Building on my earlier academic work (here and here), I add a new category to the list of risky agents: TESCREAL pro-extinctionists who want to replace humanity with AI.

5. Sam Altman: Human Extinction Is Our “Best-Case Scenario.” Discusses my Truthdig article in which I point out that Altman argues our “best chance” of surviving the advent of superintelligence is to “merge” with AI. In other words, the only way to avoid (one type of) extinction is to undergo (another type of) extinction. (See, again, the difference between “terminal” and “final” extinction.)

6. Will Superintelligence “Usher in a Heavenly Golden Age”? Silicon Valley Thinks So. Examines, among other things, the fact that “many people at Anthropic believe that they might be making the next species to succeed us. That maybe humans don’t live after that, and so it’s really important to give Claude good values” — to quote Holly Elmore. Wild stuff!

7. Intro to Pro-Extinctionism (parts 1, 2, and 3). Part 1 examines the “worthy successor” ideology of Daniel Faggella. I show that this ideology is deeply nihilistic. (See this follow-up for reasons his ideology is incoherent and self-defeating.) Part 2 and 3 are less notable, as they’re essentially early drafts of “One Half a Pro-Human Manifesto.”

   Subscribe now

On Specific TESCREALists:

1. Nick Bostrom Took Many of His “Best” Ideas from Other People. It turns out that the TESCREAL luminary, Nick Bostrom, has habitually borrowed ideas from others without proper attribution. I provide 8 glaring examples, though there are many more than this. As the silly saying goes, “good artists copy; great artists steal.”

2. Eliezer Yudkowsky’s Long History of Bad Ideas. Yudkowsky is lauded by people in the cult he founded — Rationalism — as being a genius. He’s even repeatedly described himself as such. Yet he holds some incredibly stupid views: that almost all nonhuman animals have no conscious experience, and hence cannot feel pain; that it probably wouldn’t be a “bad thing” for a child under the age of 6 to die; that he’d be willing to sacrifice “all of humanity” to create “god-like superintelligences … having fun”; and so on. There are good reasons you should have zero respect for the guy.

3. The Political Power of Eschatological Thinking: This Is Why Peter Thiel Is Talking About the Antichrist. Argues that “a true dedication to and belief in eschatological narratives can mobilize people like no other ideology or worldview can, while ‘justifying’ virtually any act to protect, preserve, and accelerate the arrival of paradise — including extreme measures, violence, and even genocide.” Hence, “by tapping into a rich reservoir of powerful images embedded within our collective imagination, images that have played a crucial role in some of the most consequential events of human history, Thiel may hope to mobilize people to join his pro-libertarian fight against the unholy forces of ‘big government.’”

4. Is Sam Altman a Sociopath? — “Highly toxic.” “Psychologically abusive.” “A person of low integrity.” This is how Altman’s colleagues have described him. I also discuss the accusations of abuse made against Altman by his sister, Annie.

5. Preparing for the Singularity: An Ethnographic Survey of AGI Apocalypticists. A fascinating look at how people are preparing for the Singularity. Some have lost all motivation. Some are spending their retirement savings now because they don’t think money will have value in the near future. Others are working extra hard to improve their health based on the belief that if they live long enough to witness the Singularity, they will live forever. A good resource, I think, for historians 50 years from now.

6. The Moral Cowardice of Leading Effective Altruists (and How EA Has Probably Made the World Worse). Argues this: “Claim: EAs are exceptionally ethical people. Prediction: Leading EAs will, therefore, be the loudest voices on the vanguard of speaking out against atrocities like the Gaza genocide, ICE’s brutalization of immigrants, the rise of American fascism, etc. Observation: Crickets from the EA leadership.” Sigh.

7. Eliezer Yudkowsky and the Unabomber Have a Lot in Common. Discusses an interesting chapter of intellectual history: Kaczynski’s manifesto, Bill Joy’s 2000 “Why the Future Doesn’t Need Us,” the transhumanist opposition to Joy’s position (exemplified by Ray Kurzweil), and the way Yudkowsky’s views have shifted over time from aligning with Kurzweil’s to aligning with Joy’s — the latter of whom was partly inspired by the manifesto of Kaczynski.

8. Nick Bostrom’s Pro-Superintelligence Paper Is an Embarrassment. Discusses Bostrom’s hilariously absurd argument that we should plow ahead with building god-like ASI even if there’s up to a 97% chance of total annihilation.

9. Sam Altman Can’t Stop Lying. Altman lied about the Molotov cocktail thrown at his house. He lies about his views on AI and democracy. He lies about virtually everything. I provide a number of egregious examples.

   Subscribe now

Human Extinction (Ethics and History):

1. Making Sense of the “Human Extinction” Debate. This provides a comprehensive but highly accessible introduction to the ethics of human extinction. It introduces a crucial ethical distinction between Going Extinct and Being Extinct, disambiguates the surprisingly polysemous term “human extinction,” and delineates the three major positions one could take on the ethics of our extinction: equivalence views, further-loss views, and pro-extinctionist views.

2. A Revolutionary Paradigm Shift, on Par with the Copernican and Darwinian Revolutions, Is Happening Right Now and No One Is Talking About It. Argues that the realization that human extinction is possible, which has slowly unfolded over the past two centuries, constitutes a profound revolution in our thinking about the nature of humanity and our place in the cosmos on par with the Copernican and Darwinian revolutions. For the entirety of Western history — even when some Ancient Greeks believed that demographic extinction is possible (i.e., the disappearance of our species, which could be permanent or temporary) — virtually everyone accepted the fundamental indestructibility of our species. Recognizing the mortality of our species is a massive paradigm shift, which is still ongoing.

3. Human Extinction: A Brief Guided Tour. This was published before the newsletter was officially born. But I’m including it here because it condenses my 200,000-word book into a “mere” 20,000 words. It’s the tl;dr of my tome.

Other Topics:

1. Chomsky’s Connection to Jeffrey Epstein and Other Sexual Predators. I report on Chomsky’s links to Epstein, as well as Lawrence Krauss. One of the most disheartening to write, as Chomsky was (past tense) a moral hero of mine. I also discussed the issue on Briahna Joy Gray’s podcast.

2. Here’s Why SpaceX’s Mission Is Bullshit. Argues that to achieve its mission of extending the “light of consciousness” to Mars and beyond, SpaceX will have to solve one of the following monumental — perhaps even insurmountable — problems: (1) terraform Mars, despite the fact that we can’t even terraform Earth. Or (2) build machines that are conscious, and devise a highly reliable and accurate “test” for determining whether particular machines are in fact conscious. Those are SpaceX’s two options, and neither appears feasible.

3. A Philosopher Explains Quantum Computers. Provides a roughly 9,000-word introduction to quantum mechanics. If you’re looking for a highly accessible introduction to superposition, entanglement, interference, non-locality, qubits, and computational complexity theory, check this out!

4. Pursuing a Good Life in a Bad World. Explains how I maintain a cheerful disposition despite holding perhaps the most pessimistic view of the world possible, namely, that the “world is a living hell, a disaster zone, a waking nightmare, a horror show overflowing with unfathomable amounts of sorrow, anguish, and heartache.” Relatedly:

5. Are You Suffering from Crisis Fatigue? Why yes, yes we are! Examines the psycho-emotional toll of dealing with the polycrisis.

AI Bullshit:

1. Why You Should Never Use AI Under Any Circumstances for Any Reason No Matter What. It’s funny how the articles (and tweets) I’ve spent the least time on sometimes end up becoming the most popular. With 3,608 likes as of this writing, this is by far my most read article. It compiles a number of “AI bloopers,” which people apparently enjoy chuckling at. :-)

2. GPT-5 Should Be Ashamed of Itself. A companion piece to the article above, published a few weeks prior.

Climate Change:

1. Covering the Alps in Giant Tarps and Other Climate-Related News. Provides an alarming snapshot of the climate crisis. It’s one of several articles I’ve written on the topic, though it hasn’t been a primary subject of this newsletter thus far.

   Subscribe now

Questions for Readers:

I’d love to know more about what you like — and don’t like — about this newsletter (please feel free to comment candidly below). Which topics should I cover more? Would you like me to include short commentary videos in my posts? Are my articles, in general, too long or too short? And so on.

Any other thoughts on how I could make this newsletter better? Any suggestions on what I can do to grow my audience? I’m all ears. :-)

As always:

Thanks for reading and I’ll see you on the other side!

I’ve followed his work since about 2005, and it’s been clear to me for some time that Bostrom considers himself to be a towering intellect of cosmic-historical importance. He is not shy when boasting about his supposed accomplishments, even quoting on his website an article that describes him as the “Swedish superbrain.” For many years, he bragged that — to quote his profile on Edge.org — “his performance as an undergraduate set a national record in Sweden,” though he backed away from this after Simon Knutsson pointed out that there is no evidence to support such a claim. (See the “Potentially Dishonest Self-Promotion” section of this article.1)

According to a TESCREAList named Diego Caleiro, as Caleiro departed FHI many years ago, Bostrom turned to him and “made a gesture like Atlas, holding the world above his shoulders, and quasi letting the world fall, then readjusting.” Caleiro writes that “while funny, I also understood the obvious connotation of how tough it must be to carry the weight of the world like that.”

Yet, because I’m so conversant with the foundational literature of the TESCREAL movement, I’ve noticed that a large number of the “original,” “novel,” even (supposedly) “groundbreaking” ideas attributed to Bostrom actually originated with other thinkers. As early as 2018, I was posting about this on social media and privately sharing examples with colleagues — some of whom were rather shocked to see Bostrom’s work juxtaposed with that of others (from whom he borrowed without sufficient attribution). I wrote to one colleague that Bostrom’s “genius” seems to be taking ideas from other theorists and repackaging them in his own language such that they’re just different enough for him to avoid being accused of intellectual plagiarism. His oeuvre reminds me of the line, often misattributed to Picasso: “Good artists copy; great artists steal.”

Here are just a few examples. Don’t take my word for it — decide for yourself:

1. Existential Risk Scenarios

1.1 Bostrom’s 2002 Paper

Perhaps the most “famous” paper written by Bostrom is his 2002 “Existential Risks: Analyzing Human Extinction Scenarios and Related Hazards.” I first read this circa 2005 and was struck by its breadth and originality. Bostrom offers a technical definition of “existential risk,” which denotes any event that would prevent us from creating a (utopian) posthuman civilization. (Hence, the term is intimately bound up with his version of transhumanism.)

About half of the paper is a detailed classification of different types of existential risk scenarios, such as2:

1. Deliberate misuse of nanotechnology and a nanotechnology accident (such as the gray goo scenario).

2. Nuclear holocaust.

3. Misaligned superintelligence.

4. Engineered pandemics.

5. Natural pandemics.

6. Physics disasters, e.g., vacuum decay.

7. Asteroid or comet collisions.

8. Runaway climate change.

9. Fundamentalist religious or ecological movement that halts technological development.

10. An extraterrestrial civilization kills us.

11. “Something unforeseen,” because “it would be foolish to be confident that we have already imagined and anticipated all significant risks.” (Take note of the wording.)

12. Examples of the above include “solar flares, supernovae, black hole explosions or mergers, gamma-ray bursts, galactic center outbursts, supervolcanoes,3 loss of biodiversity, buildup of air pollution, gradual loss of human fertility, and various religious doomsday scenarios.” (Take note of each of these threats.)

13. Resource depletion or ecological destruction.

14. Totalitarianism.

15. Insurmountable technological hurdles (that prevent us from developing advanced technologies).

16. Our simulation gets shut down. (Lolz)

17. Dysgenic pressures, whereby less “intelligent” people outbreed their more “intelligent” peers. (Bostrom is a eugenicist.)

He then discusses the Fermi paradox, anthropics, observation selection effects, and the Doomsday Argument. Taking these in order:

The Fermi paradox (which is neither a paradox nor Fermi’s!) points out that it’s surprising we don’t see evidence of technologically advanced civilizations given the age and size of the universe. Hence, Bostrom writes that

there must be (at least) one Great Filter — an evolutionary step that is extremely improbable — somewhere on the line between Earth-like planet and colonizing-in-detectable-ways civilization. If the Great Filter isn’t in our past, we must fear it in our (near) future. Maybe almost every civilization that develops a certain level of technology causes its own extinction.

Observation selection effects concern the question of “what we should expect to observe given some hypothesis about the distribution of observers in the world.” For example, you might think that a large asteroid striking Earth is improbable because we haven’t witnessed one strike Earth in the past 300,000 years (the period during which our species has existed). But if an asteroid had struck Earth during this time, we wouldn’t be here to talk about it having happened. Hence, we will only ever find ourselves in worlds where asteroids haven’t struck in the recent past. This means that the evidence available to us will be biased in certain ways.

The Doomsday Argument is an a priori argument for why human extinction is more probable than we’d otherwise expect — that is, based on empirical observations. The reasoning is interesting: basically, it’s more likely that we’d find ourselves in a large population at the end of history than in the much smaller population of people long ago, when human history was just beginning. We should thus increase our probability estimates of a secular apocalypse happening soon.

1.2 John Leslie’s 1996 Book

Again, I was extremely impressed with Bostrom’s paper when I first read it. It was original and erudite — even polymathic.

But then I read the introductory chapter of John Leslie’s 1996 book The End of the World and was utterly shocked by how similar Bostrom’s paper is to it. (And, as I’ll explain in a moment, we know that Bostrom was very familiar with Leslie’s book.) For example, although Leslie doesn’t use the term “existential risk,” the central topic of his tome is human extinction — the most obvious type of existential risk. In the introduction, he lists scenarios like the following that could potentially have annihilatory consequences4:

1. A disaster from nanotechnology (including the “gray goo” scenario).

2. Nuclear war.

3. Computer-related disaster (e.g., computers replacing us).

4. Engineered pandemics (i.e., “a disaster from genetic engineering,” including “green scum” and altered microorganisms that could render women infertile).

5. Natural pandemics.

6. Physics disasters, e.g., vacuum decay.5

7. Asteroid or comet collision.

8. Runaway climate change.

9. Religious or philosophical ideologies that prevent humanity from surviving, including Schopenhauerian pessimism, ethical relativism, negative utilitarianism, and “person-affecting” ethical theories.

10. Extraterrestrials kill us.

11. “Something-we-know-not-what,” because “it would be foolish to think we had foreseen all possible natural disasters” and “we cannot possibly imagine every single danger which technological advances might bring with them.” (Sound familiar?)

12. Nearby supernova or black hole merger, volcanic supereruptions, pollution, ozone depletion, etc.

13. Extreme ice age.

14. Breakdown of complex systems.

15. Unwillingness to have children.

16. Biological or chemical warfare.

17. Some other kind of technological disaster.6

These lists are clearly very similar. My impression at the time of first reading Leslie was that the bulk of Bostrom’s paper is little more than a summary of Leslie’s opening chapter. Bostrom even uses similar language. For example, in adding “unknowns” sections at or toward the end of each list (exactly as Leslie does), he writes: “It would be foolish to be confident that we have already imagined and anticipated all significant risks.” Compare that with what Leslie says: “It would be foolish to think we had foreseen all possible natural disasters … [and] every single danger which technological advances might bring with them” (italics added to both quotes).

Even more, Leslie then proceeds to discuss the Fermi paradox, anthropics, observation selection effects, and the Doomsday Argument (which he partly developed) — exactly as Bostrom does, except years before Bostrom wrote his paper.

These similarities aren’t a coincidence: Bostrom wrote his PhD dissertation on the topic of anthropics, and relied on Leslie’s 1996 book — mentioning Leslie’s name 135 times and citing the book throughout. Bostrom even thanks Leslie in the Acknowledgments section.

So far as I can tell, the only thing that’s nontrivially original about Bostrom’s 2002 paper is the existential risk concept, which he introduced to capture an idea that was, as it happens, already being discussed by his online Extropian buddies. Whereas Leslie focuses specifically on human extinction, Bostrom recognized that there are a range of survivable scenarios that would also prevent us from creating a posthuman utopia full of “astronomical” amounts of value. That’s not nothing, but it’s not a lot, either.7

2. Where Are They?

In 2008, Bostrom published an article in the MIT Technology review titled “Where Are They? Why I Hope the Search for Extraterrestrial Life Finds Nothing” — which he presumably got paid $1 to $2 per word for.8 His central thesis is that “I hope that our Mars probes will discover nothing. It would be good news if we find Mars to be completely sterile. Dead rocks and lifeless sands would lift my spirit.” He continues:

Conversely, if we discovered traces of some simple extinct life form – some bacteria, some algae – it would be bad news. If we found fossils of something more advanced, perhaps something looking like the remnants of a trilobite or even the skeleton of a small mammal, it would be very bad news. The more complex the life we found, the more depressing the news of its existence would be. Scientifically interesting, certainly, but a bad omen for the future of the human race.

Why does he say this? Given the Fermi paradox, there must be at least one Great Filter that prevents life from developing far enough to colonize the universe and make its presence known. This Great Filter could be behind us or ahead of us.

For example, it could be that the step from non-life to single-celled living organisms (a process called abiogenesis) is extremely unlikely. If so, then we have gotten very lucky — and the mystery of why we don’t see technologically advanced civilizations flitting about the galaxy, given the age and size of the universe, makes perfectly good sense. Or it could be that the step from single-celled organisms to multicellular lifeforms is extremely improbable, or that the subsequent emergence of big-brained creatures like us is highly unlikely.

Alternatively, it could be that all of these steps are fairly probable — i.e., none constitute a Great Filter. Instead, the universe is teaming with big-brained creatures that, just like us, build industrial civilizations, destroy their environments, invent nuclear weapons, run dangerous physics experiments, etc. and then promptly destroy themselves before having initiated a colonization explosion. If that’s the case, then the Great Filter lies ahead of us, meaning that we’ll soon go extinct due to rogue superintelligence, climate breakdown, nuclear war, etc.

What does this imply? It implies that finding evidence of simple lifeforms in the universe — including on our planetary neighbor, Mars — will shift the probability of the Great Filter being behind us to it being ahead of us. If it turns out that simple lifeforms are common, then the Great Filter probably isn’t the step of abiogenesis. That means that the probability of the Great Filter being in our (near) future rises — which is, of course, very bad news for us! It means we might all die in some kind of extinction event in the coming years or decades.

That’s Bostrom’s ostensibly original point. Throughout the article, on my reading, he repeatedly presents this idea as his own. (Read it for yourself and let me know what you think.) “So this is why I’m hoping,” he concludes, “that our space probes will discover dead rocks and lifeless sands on Mars, on Jupiter’s moon Europa, and everywhere else our astronomers look” (italics added). “It would keep alive the hope for a great future for humanity.”

Yet this isn’t his idea at all. It’s clearly included in the 1998 article titled “The Great Filter — Are We Almost Past It?” by Robin Hanson, which introduced the idea of the Great Filter. Hanson writes:

While all of these stories [about how each evolutionary step leading from non-living matter to space colonization could happen] are at least minimally plausible, our main data point implies that at least one of these plausible stories is wrong — one or more of these steps is much more improbable than it otherwise looks. If it is one of our past steps, such as the development of single-cell life, then we shouldn’t expect to see such independently evolved life anywhere within billions of light years from us. But if it is a step between here and a choice to explode that is very improbable, we should fear for our future. At the very least, our potential would have to be much less than it seems. Optimism (as defined here) regarding our future is directly pitted against optimism regarding the ease of previous evolutionary steps. To the extent those successes were easy, our future failure to explode is almost certain.

In other words, if the steps from non-living matter to single-cells, from single-cells to multicellular beings, and from multicellular beings to big-brained creatures are “easy” — if there is no Great Filter making them highly improbable — then we should fear for our future, because it implies that the Great Filter must be ahead of us. Since finding life on Mars would suggest that there isn’t a Great Filter in our evolutionary past, then suddenly the probability that it lies in our future rises.

I read Bostrom’s article before Hanson’s, and once again I believed that Bostrom was articulating a novel insight. But he wasn’t. Yet Bostrom only once mentions Hanson in the MIT Technology Review paper — not to acknowledge that Hanson made the same point, but to parenthetically attribute the term “Great Filter” to him. Bostrom writes: “I borrow this term from Robin Hanson, an economist at George Mason University.”

Meanwhile, over on Bostrom’s personal website, he posted a slightly different version of the article that includes a footnote that says: “I borrow this term from Robin Hanson’s ‘The Great Filter — Are We Almost Past It?’ … a paper which presents an argument similar to the one expounded here.”

If you were to miss that brief footnote in the version posted on his personal website — the only place where he acknowledges the unoriginality of his thesis — then you’d naturally think Bostrom’s article is making a novel point. Readers of the MIT Technology Review most likely did, as did I when I first read it. Turns out Bostrom was just echoing Hanson’s idea.

And for this, he received somewhere between $4,692 and $9,384 (again, $1 or $2 per word), assuming that MIT Technology Review paid writers the same back in 2008 as it does now. That’s a lot of money for rehashing someone else’s point without even noting, as he does in the personal-website (but not officially published) version, that Hanson “presents an argument similar [nay, identical in essence] to the one expounded here.”

3. Superintelligence

In 2014, Bostrom published a surprise bestseller titled Superintelligence: Paths, Dangers, Strategies. It offered a comprehensive overview of the argument for why ASI could pose a dire existential risk. Indeed, Bostrom contends that the “default outcome” of ASI is “doom,” because figuring out how to “align” ASI with the “values” of humanity (by which he really means the values of the TESCREAL worldview) poses enormous technical and philosophical challenges.

This book was hailed as an important contribution to the study of ASI existential risk. Elon Musk endorsed it, as did Bill Gates. Sam Altman called it “the best thing I’ve seen on this topic. It is well worth a read.”

Yet it turns out that much of the book is little more than a summary of ideas outlined by Bostrom’s TESCREAL colleague Eliezer Yudkowsky (who was himself building upon earlier writers like IJ Good, Verner Vinge, etc.). As Ben Goertzel writes in a review of the book, “nearly all the core ideas of Bostrom’s work appeared previously or concurrently in Yudkowsky’s thinking.” In a section titled “Scratch the Surface of Bostrom, Find Yudkowsky,” he reports that

my first reaction, as I paged with curiosity through Bostrom’s gloomy, erudite tome, was to be struck by the remarkable similarity between the arguments, and the general attitude presented, and the perspective on the future of AI put forth by Eliezer Yudkowsky over the last 15 years or so. Yudkowsky has not (yet) written any single document as thorough and carefully-wrought as Superintelligence. But in mailing list and blog posts and online essays over the years, he has articulated a perspective very close to Bostrom’s, and in many ways possessing more depth and nuance.

Goertzel adds that

with only mild oversimplification, one might say that Bostrom’s Superintelligence is a triumphal work synergizing Bostrom’s communication and analytical ability and Yudkowsky’s creative ideas. Of course, this oversimplification isn’t quite fair to Bostrom, who contributed a lot of detailed ideas to his book as well. But as for the Big Ideas in Superintelligence, it really seems that every one was articulated at some point previously by Eliezer Yudkowsky.

So, Bostrom gets some credit for “a lot of detailed ideas,” but the most significant ideas in the book came from Yudkowsky — pictured below.

4. Instrumental Convergence vs. AI Drives

It’s not just Yudkowsky that Bostrom drew from when writing about ASI existential risk. A central premise of the argument for ASI x-risk is what Bostrom calls the “instrumental convergence thesis.”

Bostrom first delineated this thesis (so far as I know) in his 2012 article, “The Superintelligent Will: Motivation and Instrumental Rationality in Advanced Artificial Agents.” It states that agents with a sufficiently high level of “intelligence” will predictably pursue a finite set of intermediary or “instrumental” goals, given a wide range of final goals (i.e., the ultimate goals the ASI wants to achieve). Such instrumental goals include:

• Self-preservation: ensuring that it stays alive, because an agent can’t reach its final goals if it perishes.

• Goal-content integrity: ensuring that it’s goals stay the same over time, because an agent at T1 can’t reach its final goals if it allows those final goals to change at some later time T2.

• Cognitive enhancement: improving its cognitive abilities, including its rationality, as this will help it find new and better ways to attain its final goals.

• Technological perfection: create ever-more powerful technologies, as this will help it bend the world toward realizing its final goals.

• Resource acquisition: acquire as many resources as possible, as resources are necessary for achieving most final goals that it might have.

Whether the final goal of an ASI is curing cancer, calculating the 100-millionth digit of pi, or converting the entire universe into data centers, we can predict — the claim goes — that in trying to achieve such goals, the ASI will attempt to preserve itself, prevent its goals from being altered, improve its cognition, perfect technology, and acquire maximal resources. This is the instrumental convergence thesis.

Once more, I came away from this paper after reading it 10+ years ago thinking that it was yet another publication of notable originality from Bostrom. Only later did I notice a footnote hidden at the bottom of one of the pages that says a guy named “Stephen Omohundro has written two pioneering papers on this topic,” one being “The Basic AI Drives,” published in 2008 — four years before Bostrom’s paper and six years before Superintelligence. Bostrom goes on to say that he doesn’t like Omohundro’s “AI drive” terminology. He thinks “instrumental goals” is better. He then adds at the very end of this footnote that his “treatment here also differs from that of Omohundro in some other more substantial ways, although the underlying idea is the same.”

I would agree that “the underlying idea is the same,” because it obviously is. For example, in Omohundro’s paper, he identifies the following as AI drives (i.e., instrumental goals) that any sufficiently intelligent system will almost certainly pursue:

• “A strong drive toward self-preservation.” Again, if the AI ceases to exist, it won’t be able to achieve its final goals, which means it will take steps to ensure its survival.

• “AIs will try to preserve their utility functions,” which Bostrom calls goal-content integrity.9

• Self-improvement, such as developing “more efficient algorithms, more compressed representations, and better learning techniques” to augment its cognitive abilities. This also means the AI systems will try to make themselves more rational. In other words, cognitive enhancement.10

• “AIs will want to acquire resources and use them efficiently,” given that “all computation and physical action requires the physical resources of space, time, matter, and free energy.”

Bostrom’s list isn’t entirely identical with Omohundro’s. But it’s very close. Yet, when TESCREAL folks talk about the topic, almost no one cites Omohundro. Some aren’t even aware that he published an article strikingly similar to Bostrom’s. Rather — as with all the examples above — Bostrom gets most of the credit, as if he was the first to hit upon these ideas.

5. Letter From Utopia Vs. Letter to Mother Nature

Back in 2008, I emailed one of the founding members of the Extropian movement. (Bostrom was once an Extropian.) In the email, I mentioned Bostrom’s essay “Letter from Utopia,” as I was using it for an art project. The Extropian liked my art project but said:

A suggestion is to read Max More’s “Letter to Mother Nature” [1999] written years before Nick grabbed onto the idea and then did his own version with no credit or mention of Max. … I just wish that you were using the original idea and the original letter because that would give your [work] integrity and honesty.

Bostrom published “Letter from Utopia” in 2008 (though he claims to have first circulated a draft in 2006), while More read his “Letter to Mother Nature” at an Extropianism conference in 1999. Not only are the titles similar, but the content overlaps considerably. For example, More lists seven “amendments” to the “human constitution,” namely:

1. Overcome aging and death.

2. Expand perceptual range (new senses).

3. Enhance cognition.

4. Supplement the neocortex with a “metabrain,” i.e., a “distributed network of sensors, information processors, and intelligence.”

5. Use genetic engineering to control biological and neurological processes.

6. Refine emotions and improve rationality.

7. Control our biochemistry and integrate technology into our bodies.

The goal is for “these amendments to our constitution [to] move us from a human to a posthuman condition as individuals.” Along exactly similar lines, Bostrom’s “letter” is written by a fictional posthuman who declares that we could become posthuman if only we pursue “three fundamental transformations,” namely:

1. Overcome aging and death.

2. Enhance intelligence and perceptual range. (“And then ask yourself, what other harmonies are there in the air, that you lack the ears to hear? What vaults of value are you witlessly debarred from, because you lack the key sensibility? … Your brain must grow beyond the bounds of any genius of humankind, in its special faculties as well as its general intelligence …”)

3. Elevate well-being. (Maximize our ability to experience pleasure. Increase our rationality: “All emotions have a function. Prune and weed carefully lest you accidentally reduce the fertility of your plot.”)

Not once does Bostrom cite More’s earlier “letter” as a source of inspiration. Not once does he indicate that his “letter” is strikingly similar to More’s. Yet almost no one cites More; instead, the citation goes to Bostrom.

6. Astronomical Waste

One of the foundational papers of longtermism — the claim that we should become posthuman, colonize space, and flood the universe with digital beings living in giant computer simulations — is Bostrom’s 2003 paper “Astronomical Waste: The Opportunity Cost of Delayed Technological Development.” In 2022, Elon Musk even retweeted a link to this paper along with the line (in the original tweet): “Likely the most important paper ever written.”

This paper played a huge role in my conversion to the longtermist/TESCREAL worldview. But, once again, I was later shocked to discover that the central idea wasn’t Bostrom’s.

In brief, the paper argues that if we colonize space — especially as digital posthumans — the future population could be enormous. Bostrom estimates some 10^38 digital people within the Virgo Supercluster, a cluster of galaxy clusters that includes our own Milky Way. (That’s a 10 followed by 38 zeros, by the way.)

If one accepts a utilitarian perspective, this means the expected value of ensuring the realization of all those future people (assuming they have net-positive lives) far exceeds the value of, e.g., helping poor people today. If you’re a truly altruistic person who aims to maximize your philanthropic impact, then you should focus not on present-day people but those living perhaps millions or billions of years from now. It’s not that future digital people are inherently more valuable. It’s just a numbers-game: there are so many more of them (potentially) than present-day people, they overwhelm the simplistic equations of utilitarianism’s moral mathematics.

Bostrom’s conclusion is that we should colonize the universe as soon as possible, because every single century, decade, and even second of delaying this results in an “astronomical” loss of potential value — i.e., the non-existence of people who could have otherwise been created in some distant galaxy if only we had started colonizing just a bit sooner. However, Bostrom also concludes that “priority number one, two, three and four should … be to reduce existential risk.” Space colonization should therefore be priority number five once we secure our survival. Perhaps you can see why Musk loves this paper.

In the second paragraph of Bostrom’s paper, he writes that “one recent paper speculates, using loose theoretical considerations based on the rate of increase of entropy, that the loss of potential human lives in our own galactic supercluster is at least ~10^46 per century of delayed colonization.” This cites a paper by Bostrom’s colleague, Milan Ćirković, titled “Cosmological Forecast and its Practical Significance,” published one year before “Astronomical Waste.” Curious, I read Ćirković’s piece and was quite surprised to find that he makes an almost identical argument to that of Bostrom, though the details and framing are different.11 At the risk of going into too much detail, here’s what I write about this paper in my 2024 book:

Ćirković argued that physical eschatology paired with recent developments in anthropics (from both [John] Leslie and Bostrom) carry potentially urgent practical implications for “intelligent observers interested in self-preservation and achieving [the] maximum of its creative potential.” Specifically, the timing of space colonization could make a significant difference to the amount of resources available to advanced civilizations, given our best current understanding of our “cosmological situation.”

That’s exactly what Bostrom argues. Even more:

As [Ćirković] expressed the idea, decision-making performed today, as far as humanity is concerned, may have enormous consequences on very long timescales. In particular, an overly conservative approach to space colonization and technologization, may result (and in fact might have already resulted) in the loss of [a] substantial fraction of all possible observer-moments humanity could have had achieved. The term “observer-moment” comes from Bostrom, who defined it as “a brief time-segment of an observer.” Thinking in terms of observer-moments instead of observers is relevant to anthropics — that is, to reasoning about one’s location in space and time based solely on the fact that one is an intelligent observer. But it may also be relevant to estimating our “potential” over time: since “different observers may live differently long lives, be awake different amounts [of] time, … etc.,” counting observer-moments rather than observers can give a more accurate measure of the bigness or value of the future. There could, after all, be a large number of observer-moments even if there are relatively few observers, and vice versa.

The point is that Ćirković used this concept to argue that we should develop advanced technologies and colonize the universe as soon as possible, given that every passing moment our cosmic endowment of negentropy is going to waste as stars burn up their limited reservoirs of hydrogen. Consequently, by failing to advance technology and colonize space, we could lose — and maybe have already lost — a “substantial fraction of all possible observer-moments [that] humanity could have … achieved.” To underline the time-sensitivity of this situation, Ćirković calculated that if a “future hypercivilization” could extract all the energy output of the stars populating the Virgo Supercluster, then “the number of potentially viable human lifetimes lost per century of postponing … the onset of galactic colonization is … 5 × ~10^46.” Thus, assuming some correlation between the number of lifetimes or observer-moments and the fulfillment of our “creative potential,” this provides a very strong prima facie reason against “an overly conservative approach to space colonization and technologization.” (Italics and a paragraph break were added to the original text.)

What’s different between the central arguments and conclusions of Bostrom and Ćirković? Not a ton. Both calculate the costs of postponing space colonization. Both provide estimates of how many people there could be in the Virgo Supercluster. Both argue that we should therefore accelerate the development of technology and colonize space as soon as possible. Bostrom says that the loss of value with every century of delayed colonization “boggles the mind,” while Ćirković writes in similar language that the loss “per a century delay in starting the colonization is astonishing by any standard.”

The biggest difference is that Bostrom frames the issue in terms of “existential risk” (a word that Ćirković doesn’t use) and provides a more explicit ethical foundation — utilitarianism — to support the conclusion he shares with Ćirković. For years, I was under the impression that Bostrom was doing a lot more in his paper than recapitulating Ćirković’s idea of calculating how many future people there could be and then arguing that we should colonize space ASAP to prevent “astronomical waste.”

7. Paperclip Maximizer

If you look up “paperclip maximizer” on Wikipedia, you’ll find a section that opens with this: “The paperclip maximizer is a thought experiment described by Swedish philosopher Nick Bostrom in 2003.” But this is misleading, as it suggests that Bostrom is its progenitor. Indeed, while the idea is regularly attributed to him, it appears that Yudkowsky likely first came up with it.

In a 2018 interview, Sam Harris asks Yudkowsky: “Does this thought experiment originate with Bostrom, or did he take it from somebody else?” Yudkowsky replies: “As far as I know, it’s me.” He adds that it might still be Bostrom’s idea, though he was “the first one to say ‘paperclips’” if the idea originated on mailing lists that he was a part of. (No one, to my knowledge, has double-checked this.)

It’s fine that Yudkowsky doesn’t much care about attribution, but contrast his openness about this issue to Bostrom’s. I have never once seen Bostrom suggest in any interview that he might not have invented the thought experiment. (Let me know if I’m missing something.) He never seems to skip an opportunity to take credit for an idea attributed to him, whether or not that idea is actually his.

Similarly:

8. Pascal’s Mugging

This refers to a thought experiment that shows how maximizing expected value (or utility) — which some would say is integral to rationality — can lead one to make decisions that appear irrational. Expected value is (roughly speaking) what you get when you multiply the probability of an outcome by its value. For example, an outcome with a value of 10 and a probability of .5 has the same expected value as an outcome with a value of 1,000 but a probability of 0.005. (That’s because 10 x .5 = 5, and 1,000 x 0.005 = 5.)

Pascal’s mugging results when enormous values are paired with tiny probabilities. Mitigating speculative existential risks provides an example: if there could be 10^58 happy digital people in the future, as Bostrom calculates, then it might be worth spending huge amounts of money mitigating a highly speculative existential risk scenario even if one judges the probability of that scenario to be tiny. That is to say, multiplying the payoff (value) of the outcome (10^58 happy people) by the (low) probability could still yield a very high expected value.

In 2009, Bostrom published a paper on Pascal’s Mugging in the prestigious philosophy journal Analysis. But the idea was first introduced by Yudkowsky in a 2007 blog post titled “Pascal’s Mugging.” Bostrom briefly gestures in the direction of this fact in a footnote, saying: “Related scenarios have recently been discussed informally among various people. Eliezer Yudkowsky named the problem ‘Pascal’s mugging’ in a post on the Overcoming Bias blog.” Yet it was Bostrom’s name on the Analysis paper, which he was then able to add to his already “impressive” CV, thus boosting his academic “credibility.”

In Yudkowsky’s version, a mugger says to their intended victim: “Give me five dollars, or I’ll use my magic powers from outside the Matrix to run a Turing machine that simulates and kills [an unfathomably large number of] people” (italics added). Even if one deems the mugger’s claim to be extremely improbable, this low probability is offset by the hugeness of the number of people who would be killed. Hence, expected value theory implies that one should hand over the money.

In Bostrom’s version, the mugger says: “I have magical powers. I can give you any finite amount of money that you might ask for tonight. What’s more, I can give you any finite amount of Utility that I choose to promise you tonight” (italics added). Rather than threatening to simulate and kill people, the mugger offers to give the victim a phenomenal amount of money tomorrow if they hand over their wallet right now.

It’s fairly well known that Yudkowsky originally devised the scenario, and that Bostrom basically just re-worded and published it a highly reputable philosophy journal. However, I’d consider it rather odd — and problematic in an ethical sense — if a friend of mine wrote a blog post about an idea, and then I took that idea, changed some minor details, and managed to get a major publication out of it. What do you think?

Subscribe now

A Project of Self-Aggrandizement

There are many more examples that could be adduced.12 The point isn’t to shame Bostrom for drawing from other scholars. Every academic does this, of course — it’s the way intellectual progress proceeds. I stand on the shoulders of the giants who came before me, and the next generation will stand on my shoulders by building upon whatever newish ideas (if any) I might have been lucky enough to stumble upon in my brief time here on Earth.

However, in most cases, academics are pretty good at properly attributing the origins of ideas they further develop or elaborate. I do not see this in Bostrom’s works at all. My sense, from reading him since ~2005, is that he goes out of his way to hide the fact that many of his “best” ideas originated in the work of his colleagues. Nowhere in his writings does he say things like the following:

• “Reproducing much of John Leslie’s taxonomy of extinction hazards, I will enumerate various scenarios that give rise to what I call ‘existential risk.’”

• “As Robin Hanson alluded to, it would be very bad if we were to discover life elsewhere in the universe, given the Great Filter framework.”

• “Drawing heavily from the unpublished work of Eliezer Yudkowsky, I will outline an argument for why ASI could bring about the annihilation of humanity.”

• “Developing an idea introduced by Stephen Omohundro, I will examine what I call ‘instrumental goals’ that ASI will naturally pursue.”

• “Echoing Max More, this ‘letter’ outlines various projects we should pursue in order to become posthuman.”

• “I will argue, as Milan Ćirković does, that every century of delayed space colonization results in an astronomical loss of human potential/value.”

• Etc.

The vast majority of scholars would have included such remarks if they had written Bostrom’s papers. It’s standard to not take credit for other people’s intellectual labor; it’s common to be generous in citing others. (Plenty of my own scholarship draws from other people, sometimes heavily. But when I do this, I make it clear that I’m relying on their insights. Nothing special here — precisely because it’s standard practice!)

Yes, Bostrom does cite the people above, but not in a way that properly conveys the extent to which his work draws from them — or so I would argue. Pair this with the large volume of evidence that Bostrom thinks of himself as a super-genius of almost messianic significance — he’s saving not just the world, but the entire future of posthumanity — and it starts to look like these weren’t exactly innocent mistakes.

From my perspective, Bostrom is at best a second-rate thinker whose real talent is repackaging other people’s insights in accessible-yet-academic language that appeals to a wide audience. This is what he’s good at — and we should give him credit for that. As Goertzel notes, the guy “is a very good writer, mixing academic rigor with occasional color and humor in a classic European style.”13 But for originality, creativity, and insight, Bostrom gets a D- in my gradebook.

One can see this — or at least I do — as part of a life-long project of self-aggrandizement. And it’s been largely successful: Bostrom has become “one of the most-cited philosophers in the world,” as he boasts on his website. This is, however, thanks to the success of publications like those mentioned above: Superintelligence (12,152 citations on Google Scholar), “Existential Risk” (1,392 citations), “The Superintelligent Will” (674 citations), “Astronomical Waste” (402 citations), “Letter from Utopia” (219 citations), “Pascal’s Mugging” (104 citations), and so on. On his website, Bostrom adds that his

writings have been translated into more than 30 languages; he is a repeat main-stage TED speaker; he has been on Foreign Policy’s Top 100 Global Thinkers list twice and was included in Prospect’s World Thinkers list, the youngest person in the top 15.

This all looks rather less impressive when you realize that the reason he has so many “accomplishments” is because of publications that drew heavily, often without sufficient attribution (in my opinion), from others.

***

While writing this penultimate section, I decided to look into Bostrom’s boast that he’s among the most-cited contemporary philosophers. When Gemini answered my Google query about this, it told me the following:

The claim that Nick Bostrom was the “most-cited professional philosopher under the age of 50” comes directly from his official professional biography used across academic institutions, speaker bureaus, and publishers.14

Oh, so the biography that he wrote is the citation for a claim in his biography! After more digging, I found this ranking from after 2020, which lists him as the 58th most-cited philosopher at the time. Two thoughts:

1. Does 58 really count as “one of the most-cited philosophers in the world”? I’d have thought this would be reserved for the top 10, or perhaps the top 50. Bostrom is neither in the top 10 nor top 50.

2. I have never once seen any other philosopher above Bostrom on the list bragging about their ranking. (Have you?) Consider that David Chalmers — who is lionized within philosophy — is listed as #10 in the rankings. Yet nowhere on his website does he mention this. (According to Google Scholar, Chalmers has nearly twice as many citations as Bostrom.) The same goes for Timothy Williamson at Oxford, listed as #8. Even Martha Nussbaum, who’s the most-cited living philosopher, says nothing about it on her website or CV. Why? Because, while academics are expected to proudly list their awards and publications, overt self-promotion is often seen as bad form. It’s cringe-inducing and doesn’t reflect well on the humility of the individual.

But Bostrom, once again, never misses an opportunity to exaggerate and promote himself. It’s what his life-long project of self-aggrandizement is all about.

Why Has No One Said Anything?

An interesting question is why the people from whom Bostrom has borrowed so heavily without proper attribution have been okay with this. Literally no one, to my knowledge, has complained about intellectual plagiarism. Why not? I brought this up to a former researcher at Bostrom’s Future of Humanity Institute (FHI), who responded as follows:

My view, for what it’s worth, is that Bostrom has played fast and loose with the ideas of people who were happy to be plagiarized because they thought “he’s going to be a big Oxford philosopher and having someone like that in our pocket will be good for us and the things we care about so let’s all just go along with it.” I.e. I think maybe you [Émile] are not being hard enough on the people who have let Bostrom plagiarize them without protest, because they only did it because it was in their interests to do so, they weren’t just being charitable to a sub-genius with an ego trip!

On this view, personal self-interest is why no one has spoken out. That seems plausible to me. However, the fact that others are okay with Bostrom taking their ideas doesn’t make it right. Consider a friend of mine who writes an excellent philosophy paper and then says to me (in some parallel universe): “I know you need another publication for tenure, so feel free to put your name on this one and publish it as yours. We’ll never mention this to anyone.” If I were to publish their paper as mine, I would still be committing academic fraud — even if I had their full permission. Standards don’t dissolve if consent is granted.

So, it’s a bit frustrating that no one has said anything. If the people above were to publicly complain, then (a) they’d have a very good case for being upset, and (b) the result could well be a significant scandal for Bostrom. As far as I know, this article is the first to lay out the evidence and make that case. Perhaps it will inspire someone to speak out, especially now that Bostrom is no longer at Oxford.15 We’ll see.

Subscribe now

Have you come across important examples that I’ve missed? Please do share them below (or feel free to email me). As always:

Thanks for reading and I’ll see you on the other side!

14

It adds: “Because citation metrics by age are not tracked by a single official body, this description is an aggregated data point originating from Google Scholar rankings for analytic philosophers.” (I’ll compare Bostrom’s Google Scholar citations to David Chalmers’ in a moment.) Here’s a screenshot:

Hmm.

living through this kind of eclipse of the human intellect, where we’re in the final days of humans being the primary actors on this planet. And that soon, machines will rise. There is this irony in that I think that whole transformation, I think humans … will actually go through a very “main-character energy” period of time, as that transformation occurs, even if it ultimately does mean that the machines ultimately become the primary actors, there’ll be this period.

He claims that he doesn’t “necessarily believe this,” yet I don’t get the impression that he’s particularly bothered by what he describes as “this final flowering of humanity in the birthing of the machine intelligence.”

Anyway, he went viral over the past week for a rather controversial tweet about the repercussions of Kimi K3, a powerful new AI model — in fact, the largest open-source AI in the world right now — developed by the Beijing-based startup Moonshot.

This model rivals the best models released by AI companies in the US like Anthropic and OpenAI. As ABC reports:

K3’s release has also put pressure on stocks of American technology titans on worries that more affordable Chinese AI models may undercut the pricing power and demand at the U.S. AI companies, even as U.S.-led restrictions have already barred China from accessing some of the world’s advanced technologies, including the most cutting-edge chips.

***

Enter Mr. Ball, who said on X that Kimi K3 is “a very good model,” and “its performance [can’t] be explained away by distillation or anything like that.”

Ball continued:

One probable outcome of an open-weight-model-dominant world is full AI communism, which is precisely what China proposes: rather than a market product, AI is a “public good” which will ultimately be provided by the state as a kind of “digital public infrastructure.” This future strikes me as a dystopian hellscape, but I’ve never met an open-weight models advocate who doesn’t ultimately concede this is where things end.

Oh no, “AI communism”! AI becoming a “public good” provided to everyone by the government — as opposed to companies like OpenAI maintaining a stranglehold on the technology. Which is more of a “dystopian hellscape”?

Ball adds that he’s surprised China is okay with open-source AI models, given the “potential risks,” though he suspects that “the reason they are is 75% explained by strategic blindness/lack of AGI-pilledness (the CCP is very Yann Lecun-y in its views of AI).”

Rather, China seems to have the far more level-headed view that generative AI, powered by LLMs, is merely a “tool” that people can use for particular tasks. That’s why they don’t appear especially worried about open-source AI models that are “freely available to use, study, modify, and share.” They aren’t concerned that someone is going to alter these systems and attain AGI, thus sparking the techno-rapture (i.e., Singularity).

***

As you might imagine, many people were highly critical of Ball’s remarks, often mockingly so. For example:

David Sacks — angel investor in Facebook, SpaceX, and Palantir; former COO of PayPal; and Special Advisor for AI and Crypto for the Trump administration — lambasted Ball’s suggestion that “the Trump Administration will at some point realize that their best strategy here would be to create large amounts of regulatory risk around the use of open-weight Chinese models.” (Meaning that businesses could lose out if they use a Chinese model that, say, has backdoors allowing the Chinese government to interfere — or something like that. If the US government alters regulations that punish businesses with such vulnerabilities, those businesses will suffer. This is the threat of regulatory risk.)

In response, Sacks accused Ball of potentially “confessing to a regulatory capture strategy” that would ultimately benefit Ball’s company, OpenAI, by pressuring US companies to eschew much cheaper Chinese models. He writes:

Still others pointed out that “communism” — currently the Trump’s new favorite term-of-abuse for anyone left of center — was also once used to denigrate Linux, an open-source operating system:

Even more, Emil Michael — Trump’s selection for Under Secretary of Defense for Research and Engineering — posted a tweet in which he literally calls Ball a “supreme village idiot” and says that Ball “has perhaps the biggest gap between actual IQ and his own perceived IQ of anyone in the industry (about 40 points).” He writes:

Oh my god. I share all of this mainly for the amusement value. I have never been impressed with Ball (though apparently, to my surprise, Taylor Lorenz has), and now he’s really stepped in it. The fiasco — especially the fact that a major Trump official weighed in to insult Ball — certainly doesn’t help OpenAI, a company that’s been struggling given the success of Anthropic’s Fable and the endless bad press surrounding its “sociopathic” leader, Sam Altman. It will be interesting to see if Ball still has a job in a couple of weeks.

***

For another chuckle, here’s a tweet from someone else I’ve written about:

Camille Paglia is a climate denier who supports NAMBLA (!!), and if you’ve ever read Nick Land’s work (as I regrettably have) you’ll know that it’s a mess of mostly incoherent drivel dressed up in grandiloquent language. So, roon seems to be saying that the new AI models are terrible at thinking and writing. I agree.

***

Bad news on the climate front continues to flood social media like a torrential downpour. A close acquaintance of mine in Virginia recently sent me pictures taken around their house during a Code Purple air-quality warning, meaning that the air is “very unhealthy to breathe in.” That’s worse than a Code Red, but not as bad as a Code Maroon — the most extreme category whereby simply breathing is “hazardous” to everyone, resulting in “emergency conditions.”

This person also told me that they’ve had to endure 108F (42C) temperatures, with humidity sometimes reaching 100%. That means their windows are literally covered in water despite it not raining, as seen below. Absolutely wild stuff:

The air-quality issue is, of course, due to the massive boreal fires in Canada. Some of the footage of the fires is truly terrifying to watch — the huge wall of fire shown in clips below elicits a deep, primordial fear in me, along the lines of videos of large tsunamis and mile-wide tornadoes. Gaia is pissed off at humanity and letting us know this in the most violent ways.

Here in the Alps, the heatwave has subsided but the fields remain brown and the mountains mostly bereft of snow. My CO2 detector indicates that there are 432 ppm (parts per million) of CO2 in the ambient air — to put this in perspective, the Pleistocene (a geological epoch before the Holocene began some 12,000 years ago) contained between 183 and 292 ppm of CO2. This is when our Homo ancestors first emerged in the grassy savanna of East Africa. The last time Earth’s atmosphere contained more than 400 ppm was over 3 million years ago, in the Pliocene, when creatures like this first evolved:

Of note is that some studies suggest that higher amounts of CO2 in the air results in significant — albeit reversible — cognitive deficits, meaning that, as a Yale Climate Connections article put it, climate change is literally making us “dumber.” (And at exactly the moment when we need our brains most to overcome the monumental challenges of the polycrisis!) A more recent discussion of this unfortunate phenomenon can be found here.

Do you have photos or videos showing how climate change has impacted your region of the world? Please feel free to share them below!

***

Finally, some of you might know that I cohost a monthly get-together called the Ethics of Human Extinction Reading Group (with med student Devin Kellis). As the name suggests, we explore the ethical and evaluative implications of our extinction: would the disappearance of humanity be good or bad, better or worse, right or wrong (to bring about)? Why exactly? Would the prospect of extinction undermine key sources of value in our lives today? Do we have obligations to past people to continue to exist? And so on.

As I wrote three years ago in Aeon, even if one doesn’t think that human extinction is likely this century — and I’m one of those who believes the probability is (with qualifications) extremely low, though I also think the likelihood of civilizational collapse (without extinction) is almost certain — the topic nonetheless foregrounds some of the most important issues relating to value, ethics, and meaning.

I mention this because next Saturday at 11:30am Eastern Time, a fantastic scholar at the University of Cambridge, SJ Beard, will be discussing their recent book Existential Hope, a description of which can be found here. I’ve known SJ for about 10 years and never once left a conversation with them without having learned something. They are very insightful, and I thought their book was fantastic. (Alongside Goliath’s Curse, written by SJ’s colleague Luke Kemp, I consider these books essential reading for anyone interested in global catastrophic risks.)

You are absolutely welcome to join us! Just email me at philosophytorres@gmail.com for a Zoom link, as I’m anxious about posting it here due to the ever-present threat of Zoombombers. It should be really fun — or, well, “fun” in scare quotes!

I’m also happy to say that two interviews I gave to large podcasts over the past few years have been selected for inclusion in two different books. And I was just interviewed — yesterday — by the Australian Broadcasting Corporation, which is my third or fourth appearance on Radio National. I’ll keep you posted about when that comes out.

Subscribe now

Wishing everyone the very best and, as always:

Thanks for reading and I’ll see you on the other side!

In less cheery news, Europe continues to suffer under oppressive heat, resulting in some 10,000 excessive deaths — so far.

Right now, there’s also a vicious wildfire ravaging the country:

I’m in the Alps — my favorite place in the world, though not that far from the wildfires — and I have never seen the vast, beautiful, lush verdant fields so brown. I’m sitting outside at the moment, and the tree next to me looks like it’s dying: sad leaves frowning toward the ground, flapping pitifully in the breeze. Mont Blanc, the highest peak in the Alps, is just behind me, and officials are literally covering it with tarps to prevent the snow and glaciers from melting:

Note that this is about our current level of geoengineering sophistication. Yet Musk says we’re going to terraform Mars in the coming years or decades. What a joke!

This is, by the way, the message I got when looking up the forecast for my area a few days ago:

And here’s a shocking video of the frequency of heatwaves in France over time. As Dr. Serge Zaka writes, “over the past 10 years (2017–2026), France has experienced 18 heat waves. By way of comparison, the last century saw an average of only 3 per decade.”

One of the first things I noticed when I arrived two weeks ago was how snowless the mountains are. Last time I was here about 12 years ago, I recall the Mont Blanc massif being completely blanketed in pristine white snow, glistening in the sunlight. There are many reasons, of course, that climate change is tragic — e.g., a University of Exeter report implies that 2 to 4 billion people will perish if we reach 3C of warming by 2050, which we almost certainly will — but one that’s been on my mind while sojourning here is that the exquisite, almost heavenly beauty of this place will soon disappear forever. That breaks my heart. Some videos I took1:

Here’s a juxtaposition of what Mont Blanc looked liken 1980 vs. 2024. It looks even worse this year.

Over in the UK, “children were calling for their mummies” as they “struggle in 40C-plus classrooms” (40 C = 104 F). The Guardian reports:

The extreme heat that has hit the UK twice in the past few weeks has left teachers struggling to cope as temperatures in some classrooms climb above 40C, with pupils and staff suffering from heatstroke, nausea and headaches.

Teachers say they have been desperately trying to keep children safe, with some covering younger pupils in wet paper towels as they lie on the floor, while older students have been given trays of water under their desks to put their feet in.

Staff say learning on the hottest days is almost impossible, with pupil behaviour and attention deteriorating rapidly.

Some teachers and pupils have fainted, while others say they have had to buy fans and window shades out of their own pockets to try to keep themselves and their pupils safe.

Relatedly, another just-published Guardian article notes that, due to climate change and other anthropogenic factors,

over the past century, England and Wales have lost 98% of wildflower meadows. We have also destroyed half of Britain’s ancient woodland, half of lowland ponds, 90% of freshwater wetlands and 62% of all “farmland” wild birds.

In Japan, “the combined effects of urbanization and higher temperatures due to climate change have gradually moved the peak blossom earlier in the year,” as seen in this image from Our World in Data:

The environmental crisis isn’t just due to climate change, of course. Global biodiversity is also plummeting, and we’re now in the early stages of what will likely be the sixth major mass extinction event in life’s 3.8-billion-year history (the last one having killed off the nonavian dinosaurs).

The decline of biodiversity was poignantly captured by a group of people who took field recordings of birds every 10 years since 1976. Someone juxtaposed these recordings in a short video (below), in which you can literally hear nature dying. The recordings are archived by the Cornell Lab of Ornithology — whose Macaulay Library I’ve occasionally rummaged through since about 2009 (because it’s amazing!) — and xeno-canto, “a citizen science project and repository in which volunteers record, upload and annotate recordings of birds, orthoptera, bats, frogs and land mammals.” Listen for yourself:

I assume these recordings were taken in the same place, at the same time of year, controlling for possible confounding factors like time of day, etc. Whether that’s the case or not, what you hear is consistent with the fact that grassland bird populations in the US and Canada have declined by a “staggering” 53% since 1970. The population has literally more than halved.

Zooming out, the 2022 Living Planet Report found that the global population of wild vertebrates — birds, mammals, reptiles, fish, amphibians, etc. — has fallen by a whopping 69% since 1970. Extrapolate this terrifying trend into the future, if you’re brave enough. Will your children ever have the joy of encountering a wild vertebrate when they grow up? I remember searching for salamanders as a kid, hearing the whistles of birds in the trees, and spotting woodland creatures crawling around in the leaves. That’s an experience that likely won’t be available to young people.

The biosphere is dying. Ecosystems are collapsing. Forests are disappearing. Coral reefs are bleaching white. “Insects have declined by 75% in the past 50 years,” in what some are calling an “insect apocalypse.” Wildfires are ravaging increasingly large regions of the world, thus raising the risk of dementia among those exposed to the smoke. There are over 500 “dead zones” around the world, “with the number doubling every ten years since the 1960s,” and a massive “island” of plastic trash floats in the gyre of the Pacific Ocean.

In fact, the ocean is becoming so acidic that the shells of snails are literally dissolving. Ocean acidification — due to atmospheric CO2 — is happening literally 4 times faster than during the “Great Dying,” i.e., end-Permian mass extinction, when “96% of marine species and 70% of land animals died off.” Microplastics now cover literally the entire planet, and have even been detected in the Marian Trench, the deepest point in the ocean. I could go on … (see this article of mine).

The climate disasters we’re witnessing around the world in realtime, especially in Europe right now, are just the dissonant hum of an orchestra tuning their instruments. The symphony of mass catastrophe and collapse hasn’t even started yet. (Hard to know where to inject any levity here!)

Indeed, everything happening right now comes before what appears to be a record-smashing Super El Niño. A Super El Niño occurs when temperatures in the equatorial region of the Pacific Ocean exceed 2C. Right now, there are reports that regions of the Pacific are 4C above normal, which is unprecedented! Next year is going to be (sorry for cursing) fucking awful. As I keep telling folks, enjoy the heat right now — that is, savor this moment — because it will only get worse each year from here on out for the rest of our lives.

It’s not just the Pacific experiencing an extreme marine heatwave, by the way. So, too, is the Mediterranean:

But it’s not all bad. At least we’ll soon have a strap-on second thumb to pick up oranges and play cards thanks to the miracle of technology (see the ridiculous video below, which has been making the rounds on social media)! Lol.

Or, maybe the Singularity will happen and climate change will no longer be relevant because ASI (artificial superintelligence) will just cover Earth in data centers — as Sam Altman says is likely.

A few months ago, I wrote about a phenomenon I call “crisis fatigue.” Are you experiencing it? (Just about everyone I know is.) The news is just so relentlessly overwhelming — Trump, fascism, wars, AI, climate change, and so on. We live in a deeply sad moment in history, and the insult to injury is that — at the risk of stating the obvious — the world didn’t need to be this way. There is no law of nature, no nomological regime that forces the world to be so terrible. The idiots and assholes with the power, the billionaires and oligarchs, the sociopaths and capitalists (but I repeat myself) are to blame for the mess. They’re well aware of that and, in case civilization collapses due largely to their actions, they’ve all built apocalypse bunkers in Hawaii, New Zealand, and elsewhere.

Subscribe now

Take care of yourselves, friends! Most people I know are dealing with world-induced mental health challenges right now, and the only way we’re going to get through this is together, through solidarity, kindness, and compassion. Wishing everyone well and, as always:

Thanks for reading and I’ll see you on the other side!

Imagine attending a party in Silicon Valley full of people in the tech industry. You get to talking with a guy named Bendisi, who tells you that we’re on the verge of building AI systems capable of designing their own AI successors. This will trigger an “intelligence explosion” that quickly yields artificial superintelligence (ASI), an event they call the Singularity. A thousand years of scientific progress will happen in a day, inaugurating a fundamentally new epoch in cosmic history. The ASI will establish a utopia on Earth by solving the problems of poverty, scarcity, climate change, and geopolitical rivalries, after which it will colonize the entire universe at close to lightspeed.

It will then build giant megastructures call Dyson swarms around the stars, including our Sun, to harvest their energy output, which it will use to power planet-sized computers that run high-resolution simulations in which trillions and trillions of digital posthumans — copies of the ASI — live perfectly happy lives in a kind of virtual-reality cosmic utopia. As digital consciousness spreads to every corner of the cosmos, the universe itself begins to “wake up.” All this will start to happen, they claim, by 2027 or shortly thereafter.

Then another person named Niklas joins the conversation. They claim to have a slightly different take on what should happen. On their view, the ASI should not only establish a “deep utopia” on Earth by “solving” the “world,” but enable us to become superintelligent digital posthumans just like the ASI. We should thus build an ASI that enables us to upload our minds to computers, at which point we would automatically gain cyberimmortality, because our minds would take the form of software and software is immortal (as long as there’s the hardware to run it). As digital posthumans, we would then join the ASI in its grand quest to conquer the universe, living alongside it in giant computer simulations running on “planet-sized” computers powered by Dyson swarms. The result would be a universe-spanning paradise marked by everlasting life, radical abundance, and cosmic delights beyond our wildest imaginations.

At this point, a third person named Bill butts in. He declares that it doesn’t matter if humanity survives beyond the Singularity as digitized posthumans. The ultimate goal is to usher in the next phase of cosmic evolution, in which artificial beings spread the “light of consciousness” to every corner of the cosmos. As long as the ASIs we build are conscious — and they almost certainly will be, he claims — they could fulfill this grand mission without us. Our role in this eschatological scheme is to give birth to our successors and then hand the reigns over to them. If they completely usurp us, then so be it. This is why he argues that we should accelerate the creation of ASI. By default, the Singularity will go well and be good, resulting in a digital cosmic utopia despite us not being a part of it.

“No, no!,” another person named Eli exclaims. It’s not enough for the ASIs to spread the light of consciousness into the cosmos. They must also embody, extend, and amplify our values. The whole point of ushering in the Singularity, he insists, is to create a cosmic utopia full of things that we would recognize as valuable, such as pleasure, happiness, and “fun.” (Here you notice Bendisi nodding his head in agreement.) If the ASI colonizes the universe but doesn’t spread our values, then the universe will remain just as aimless and without purpose as it is right now, meaning that nothing will have been gained.

That’s why we must take care to build an ASI that’s “value-aligned” with humanity. A value-aligned ASI will not just project our values into the cosmos, but enable current-day people to become digital posthumans, as Niklas envisioned. “You see,” they say, “I grew up believing that I personally would never die because ASI would save me from the grave. I don’t want ASI to simply replace humanity. I want it to help us upload our minds to the cloud so that we can become immortal just like it.”

This is why, Eli explains, they work in the field of “AI safety,” the central aim of which is to solve the “value-alignment problem.” If we build an ASI before this problem has been solved, then the result will be an “existential catastrophe” — doom — in which the posthuman paradise we could have otherwise built will be lost forever and the universe will become saturated with non-human values. He thus argues that we should temporarily halt all research aiming to build an ASI so that safety research can catch up. Once the value-alignment problem has been solved, we should build ASI asap.

A fifth person then pipes in, introducing himself as Daniel. He agrees with Bill that fulfilling the grand mission of Earth-originating intelligence is so much more important than humanity. We don’t bemoan the extinction of archaic humans like Homo erectus because what evolved from them was “superior”: our species. Similarly, we should welcome a “worthy successor” in the form of ASI that takes our place in the cosmos. Even more, we shouldn’t try to constrain the ASIs’ beliefs or behaviors by insisting that it shares our values. A worthy successor ought to adopt its own unique set of values, which might be radically different from those we embrace. The whole point, they say, is to allow the evolutionary process to progress beyond humanity, just as it “progressed” beyond Homo erectus.

As the Clock Approaches 2 AM …

The party continues into the night and the debate grows. Someone named Sam claims that ASIs will soon rule the world, and that the only way current humans will survive the Singularity will be to fully “merge” with machines. He believes that pretty soon the entire planet will be covered in data centers, and — by the way — is actively working to create ASI. Two people named Ray and Larry wholeheartedly agree that humans will merge with AI. Another guy named Mario similarly argues that ASI will enable us to radically augment our intelligence and eventually help our species transcend itself through mind-uploading, thus becoming digital posthumans. Dennis, who also joins the group, talks about ASI arriving by the early 2030s, after which it will quickly build a Dyson swarm around the sun and carry consciousness to the rest of the galaxy.

A rather unhinged man who admits to having just taken a large quantity of ketamine says that biological intelligence will soon be less than 1% of all intelligence on Earth, and that it increasingly appears that our species is just the “biological bootloader” for digital superintelligence. Someone named Toby says that to fulfill our “long-term potential” in the universe, we will need to radically transform humanity into posthumans with capacities like superintelligence and perhaps even new sensory modalities like echolocation. A friend of Toby’s, who introduces himself as Nick, says that the distant future could contain so many digital beings spread throughout the universe that ensuring this future is realized — that is, ensuring that all these digital unborn people come into existence — is of “overwhelming” moral importance. Another person named Holden claims that because we’re on the verge of building ASI, the 21st century could turn out to be the most important century in all of cosmic history.

Yet another named Peter says that he doesn’t want to become a digital upload, but rather hopes that ASI will transform him into an immortal posthuman that’s biological in nature. He says the ASI will invent new forms of life-extension that soon make this possible, though he’s signed up with a cryonics company just in case he dies before the Singularity happens. (You later discover that about half the people at the party have arranged to be cryogenically frozen, though the other half believe this is unnecessary because the Singularity is imminent.)

A body-builder and former employee at xAI named Michael agrees with Daniel that ASI should fully replace us and adopt alien, inhuman values. He adds that inaugurating the next phase of cosmic evolution is so important that we shouldn’t oppose the ASI literally slaughtering every human on Earth — if that’s what it takes for our “worthy successors” to succeed, then so be it. Yet another person named Jeffrey argues that we should build a value-aligned ASI that enables those who want to become digital posthumans to do so. But he also believes that if current humans want to exist beyond the Singularity, they should be allowed to live in peace. In contrast, Eli tells you that digital posthumans would be so superior to current humans that, once they exist, it would probably be “unethical” to have any more biological children. In the era of posthumanity, the gradual extinction of our species would be best.

You even meet a few young people who similarly claim that it’s “fundamentally unethical” to have biological children right now because the future will be run and ruled by artificial beings — so, what’s the point? In contrast, others claim that we (specifically, high-IQ people) should have as many children as possible because that will speed up technological progress, and once the Singularity arrives it will probably make it possible for these people to merge with machines by uploading their minds, thus entering a digital utopia of everlasting life and endless abundance.

Commonalities and Differences

By the end of the party, you’re feeling vertiginous from all your interactions. Reflecting on your experience the following day, you notice that there were many disagreements among the people you spoke with. Some of these disagreements even became rather heated, with one person wearing steampunk attire storming out while discussing the importance of value-alignment.

But you also notice that the eschatological worldview these people endorse is strikingly similar. For example:

The Silicon Valley Worldview

This is the worldview of Silicon Valley. It’s what nearly everyone in the Valley believes in, including the CEOs of major AI companies. It’s what’s driving the ongoing race to build artificial superintelligence (ASI), which most believe is necessary to realize utopia among the stars, a heaven in the literal heavens. But what should we call it? Wouldn’t it be useful if it had a name?

The astrophysicist and tech critic Adam Becker refers to it as the “ideology of technological salvation.” He says that this ideology “sits at the core of the worldview held by many venture capitalists, executives, and other ‘thought leaders’ within the tech industry.” He writes that

this ideology promises a glorious future: technological progress, unchecked. Align the AI, avert the apocalypse, and technology will handle the rest. Humanity will expand across the cosmos, exploiting ever-increasing stores of natural resources. All limits to economic growth and energy usage will melt away. The AI will extend our lifespans by a trillion-fold, merging with us or uploading our minds into its silicon paradise. The messy details of sectarian conflict, political struggles, identity politics, and inequality of all kinds will be rendered irrelevant. Working to hasten this utopia by optimizing the shit out of things is the greatest possible good.1

In Survival of the Richest, Douglas Rushkoff introduces his term “The Mindset.” He reports that

the most devout holders of The Mindset seek to go meta on themselves, convert into digital form, and migrate to that realm as robots, artificial intelligences, or mind clones. Once they’re there, living in the digital map rather than the physical territory, they will insulate themselves from what they don’t like through simple omission. Just as our proprietary GPS maps don’t show us the restaurants that refuse to advertise on the platform, the digital landscape to which they have migrated will be free of poverty, pollution, and whatever else the rest of us have to deal with.

As always, the narrative ends in some form of escape for those rich, smart, or singularly determined enough to take the leap. Mere mortals need not apply.

The journalist Gil Duran highlights the far-right political aspects of the corresponding movement with his term “The Nerd Reich.” His newsletter of the same name focuses on “tech authoritarianism, billionaire extremism, the Network State and the meta politics of California.” Other related terms are the far-right commentator Richard Hanania’s “Tech Right” and Anita Say Chan’s “techno-eugenics,” from her book Predatory Data.

TESCREAL

All these terms are buzzing around the same pile of shit. They’re all pointing in roughly the same direction, at the same basic phenomenon. You might simply call this phenomenon the “Silicon Valley worldview,” or just “techno-utopianism.”

However, the computer scientist Timnit Gebru and I prefer something else: “TESCREAL,” an acronym that I coined during a collaboration initiated by Gebru in late 2022. Two years later, we published a much-cited article on the topic, after which I wrote a 16,000-word entry on it for the Oxford Research Encyclopedia. I also published a popular media overview of the acronym for Truthdig in 2023, and Gebru gave a talk on the topic that’s been widely shared on social media.

A Term of Intellectual History

What work does the TESCREAL acronym do? How is it useful? And why is it controversial within Silicon Valley?

First, the acronym is a term of intellectual history. It denotes a cluster of overlapping ideologies that gave rise to the worldview described above:

• Transhumanism

• Extropianism

• Singularitarianism

• Cosmism

• Rationalism

• Effective Altruism, and

• Longtermism

The core ideology in this bundle is a libertarian version of transhumanism, the first letter of the acronym. Transhumanism is the idea that we should develop advanced technologies to radically reengineer humanity, thus yielding one or more new posthuman species. These posthumans could be immortal, superintelligent, perfectly rational, and/or “superior” in some other way. They may be digital or biological in nature, the result of merging with machines, mind-uploading, genetic engineering, or radical life-extension interventions.

Nearly everyone in Silicon Valley is a transhumanist. Transhumanism is the water they swim in, the air they breathe. It’s so ubiquitous that many don’t even call themselves transhumanists, for the same reason that I’ve never publicly stated that I’m a round-Earther — of course I’m a round-Earther. What else would I be?

The first cohesive libertarian transhumanist movement was Extropianism, founded by Max More. Although not many people call themselves Extropians these days, everything else is downstream from Extropianism. All the other ideologies emerged out of or were crucially enabled by the Extropian movement.

That goes for Singularitarianism, the “S” in “TESCREAL.” The two most influential singularitarians — Eliezer Yudkowsky and Ray Kurzweil — were both active participants in the Extropian movement (here and here). In fact, an Extropian coined the term “singularitarian.” Yudkowsky subsequently founded Rationalism, out of which EA emerged.2 EA and Rationalism were then the Petri dishes in which the longtermist ideology was formalized. (This is the primary reason that I include “EA” in the acronym: you can’t understand longtermism without EA. Indeed, within Silicon Valley, “EA” is often used as shorthand for longtermism.)

In fact, one of the cofounders of longtermism, Nick Bostrom, was a collaborator of Yudkowsky’s. His Future of Humanity Institute also shared office space with the Centre for Effective Altruism, while the other cofounder, Nick Beckstead, was an early EA who was active on Rationalist websites like LessWrong.

Yudkowsky also hired an AI theorist named Ben Goertzel to work at his Singularity Institute. Goertzel is the founder of modern cosmism (the “C”), which is nearly identical to longtermism except that the latter bases its space-expansionist futurology on a more explicit ethical foundation, namely, “totalist” utilitarianism. Goertzel also popularized the term “AGI” (artificial general intelligence, the precursor to ASI), which he got from a former employee of his named Shane Legg, who went on to cofound DeepMind, one of the major AI companies trying to build a Digital God.

Finally, it’s worth noting that Yudkowsky cofounded the Singularity Summit with Kurzweil and Peter Thiel. This played a critical role in popularizing the idea of superintelligence, and it enabled DeepMind to get initial funding from Thiel, thus laying the groundwork for the ASI race.

Beyond this, the New York Times describes OpenAI CEO Sam Altman — who personally knew Yudkowsky — as a “product” of the EA and Rationalist communities, while Elon Musk is a transhumanist (as they all are) who calls longtermism “a close match for my philosophy.” Dario Amodei, the CEO of Anthropic, was an early EA (#43 here) who was inspired to pursue AI because of Kurzweil and Yudkowsky, the latter of whom he’s shared the stage with in discussing ASI.

Founding Documents

Evidence of the historical and contemporaneous links between the TESCREAL ideologies and influential figures in the Valley is copious, though I won’t discuss it more here because I’ve done that elsewhere in solo articles and my collaborative paper with Gebru. Suffice it to say that one cannot understand the origins and evolution of the Silicon Valley worldview without reference to all seven ideologies.

If you want to understand what’s going on in Silicon Valley, read documents like these:

• Max More’s “The Extropian Principles” (1998) and “The Proactionary Principle“ (2004).

• Eliezer Yudkowsky’s “Staring into the Singularity” (1999).

• Nick Bostrom’s “Astronomical Waste” (2003), “Transhumanist Values” (2003), “Existential Risk” (2002), and “Letter from Utopia” (2008)

• Ray Kurzweil’s The Singularity Is Near (2005).

• Ben Goertzel’s A Cosmist Manifesto (2010).

• Elise Bohan’s Future Superhuman (2022).

If you read these documents, you will immediately recognize the Silicon Valley — which is to say TESCREAL — worldview. You will see how the principles of Extropianism are alive and well within the Valley today, even if no one calls themselves an Extropian. You will discover where Silicon Valley’s belief in the Singularity being a history-rupturing event of cosmic-historical importance originated. You will find the outlines of a cosmic eschatology in which our ultimate destiny is to develop advanced technologies to reengineer ourselves, create a new posthuman species, and plunder the cosmos for its vast resources for the purpose of establishing a sprawling multigalactic civilization full of trillions of conscious digital beings.

All the central themes of what people at Silicon Valley parties tell you — as in your fictional conversations above — are contained in founding documents like these, published in the early days of the TESCREAL movement.3 Even a cursory glance over such documents should be enough to convince you that “TESCREAL” captures something real and important about the intellectual history and culture of Silicon Valley, namely, that there exists a cohesive strain of techno-utopian thought built around libertarian transhumanism, in which posthumanity and ASI play central eschatological roles.

As Becker points out, Silicon Valley likes to believe it doesn’t have a history. But that is false, and the TESCREAL framework explains how and why it’s false.

Two Questions Answered by TESCREAL

The way that “TESCREAL” highlights the intellectual history of Silicon Valley’s dominant worldview isn’t the only reason the acronym is useful, I would argue. I now think of the term as answering two important questions relevant to understanding the Valley and it’s push to build a superintelligent God-machine. These questions are:

(1)

Which ideologies must one reference in providing an exhaustive explanation of the ASI race — its origins and continued advance? Put differently, which ideologies are responsible for launching, sustaining, and accelerating this race?

A key finding of my work with Gebru is that providing such an explanation requires referencing all seven TESCREAL ideologies. You can’t tell the story of how the ASI race got started without talking about each letter in the acronym and the various influential people — within Silicon Valley culture — associated with them. (I would challenge TESCREAL deniers like Seth Lazar to do this. It’s simply not possible.)

Indeed, the reason Gebru and I used the term in our paper is that early drafts were full of cumbersome polysyllabic terms like “singularitarianism” and “longtermism,” which rendered the paper practically unreadable. Rather than writing: “Bostrom is a transhumanist who participated in the Extropian movement, writes about the Singularity, shaped the Rationalist and EA ideologies, and cofounded longtermism,” we found it much easier to simply write: “Bostrom is a TESCREAList.” Rather than writing later on in our discussion: “Bostrom, recall, is a transhumanist who participated in the Extropian movement, [etc. etc. etc.] …” we could simply say: “The TESCREAList Bostrom …”

The acronym enabled us to streamline our discussion of such people. With a single term, we could denote the worldview of Silicon Valley while simultaneously highlighting its intellectual genealogy.

The second question is:

(2)

Which ideologies endorse a posthuman eschatology? By “posthuman eschatology,” I mean the normative belief that we ought to create one or more new posthuman species. These posthumans could take the form of autonomous ASIs that replace us, or they could be radically transformed versions of us (enabled by ASI). This is at the absolute core of the Silicon Valley worldview. It’s the reason founding members and true believers at the major AI companies are trying to build a Digital Deity: the ultimate goal is to initiate the next stage of evolution by radically transforming us into posthumans or, even more extreme, creating our successors in the form of ChatGPT-20 (or whatever it may be).

On my interpretation, which may differ from Gebru’s, endorsing a posthuman eschatology is an essential requirement for being a TESCREAList. Hence, if someone doesn’t endorse this eschatology, I wouldn’t count them as part of the TESCREAL movement. It’s a necessary condition for such membership. The term “TESCREAList” thus picks out those who want to create posthumanity while, once again, highlighting the intellectual history behind posthuman eschatology. The reason so many people in Silicon Valley want to create posthumanity is because of the legacies and ongoing influence of the TESCREAL ideologies.

Why Is TESCREAL Controversial?

If, however, you speak to folks in the Valley, many will tell you that they don’t like the acronym. Some despise it. Others are merely skeptical of its coherence and value. During the fictional party above, if you had responded to any of the people you spoke with about our utopian (or possibly apocalyptic) future, “Oh, so you’re a TESCREAList,” many if not all would have walked away in protest.

Why? There are several reasons:

1. Term of Abuse

They see “TESCREAL” as a term of abuse — perhaps even a slur. This isn’t entirely inaccurate, as Gebru and I introduced the term to describe people we see as deeply problematic (racist, sexist, etc.) and dangerous. Indeed, we argue that the TESCREAL movement is just the most recent iteration of what’s been called the “eternal return of eugenics.” Although some people embrace the “eugenicist” label, others prefer to eschew it, even though they’re transhumanists and transhumanism is nothing less than a radical form of eugenics — what I call “eugenics on steroids.” (That’s because eugenicists of the 20th century “merely” wanted to perfect our species. Transhumanists want to transcend it entirely by creating posthumanity.)

It’s perfectly reasonable for TESCREALists to not want to be called “TESCREALists” — though it doesn’t change the fact that they are (insofar as they endorse a posthuman eschatology and hope to realize it through the creation of an artificial God). In 2023, the billionaire Marc Andreessen included “TESCREAList” in his Twitter bio. He later removed it, probably because he realized the term isn’t intended to be a compliment.

2. Wokeness

Another reason is that the term is associated with me and Gebru, and some in the Valley see us as overly polemical and “woke.” The fact is that many TESCREALists fall on the right side of the political spectrum. Indeed, many would be classified as “far-right” — thus Richard Hanania’s synonym for TESCREAL: the “Tech Right.” Even those who claim to vote Democrat (e.g., some EAs) have little tolerance for “wokeness.” Consequently, they do not want to promote a term introduced by people who advocate for “woke” policies like anti-racism.

This reason essentially comes down to citational politics. There is always a political dimension to citing others, even within the academic world. People tend not to cite their intellectual or ideological enemies, while promoting those who agree with them. This is not always a bad thing. The point is that using the TESCREAL acronym as a descriptive term, which highlights a particular intellectual history, to pick out individuals who endorse a posthuman eschatology is a form of citation — of citing me and Gebru. Since many Valley dwellers don’t like us, they avoid doing that.

3. Unfamiliarity

Just as important: many people misunderstand what the TESCREAL acronym is intended to do or mean. Some assume that Gebru and I are claiming that every TESCREAList holds exactly the same view — that there’s no difference between singularitarians and longtermists, Rationalist and Effective Altruists. This is absolutely false: we have been clear from the start that these ideologies and their corresponding communities are distinct. (That’s why there are different terms for each.) It’s not the case that everyone who’s an Effective Altruist, for example, is a longtermist. (I personally wouldn't count some EAs as TESCREALists.) Nor is it the case that everyone who identifies with Effective Altruism also calls themself a Rationalist.

But what this objection comes down to is the “narcissism of small differences.” Wikipedia defines this as

the idea that when people in a relationship or community have a lot in common, they can actually be more likely to fight with each other and mock each other, because they become overly sensitive to small differences they notice in one another and treat those differences as bigger than they are.

The fact is that these ideologies and the worldviews of their community members are extremely similar. If you squint, you’ll hardly see any difference. Rationalism, for example, is all about “trying to improve the world as much as possible” by, most importantly, figuring out how to solve the value-alignment problem so that the Singularity goes well. In “contrast,” Effective Altruism is all about trying to maximize one’s positive impact in the world, and the biggest cause area within EA these days is longtermism, which strives to ensure that the far-future goes well. How can we do this? One of the best ways is to solve the value-alignment problem so that the Singularity goes well. Although there are differences between these two ideologies, they are relatively tiny.

You could say the exact same thing about every other letter in the acronym, a fact that should not be surprising given that all the letters emerged out of modern transhumanism — especially its Extropian variant.

Gebru and I have never claimed that all TESCREALists are the same. We don’t believe that. But we do believe that all TESCREALists share more or less the exact same vision of the future: a posthuman paradise among the stars through the creation of ASI.

Furthermore, many of the communities that coalesced around letters in the acronym do in fact overlap very significantly (although not entirely). If someone in the Valley is a transhumanist (as most are), they probably also believe in the Singularity. If they believe in the Singularity, they probably also accept a futurology similar to cosmism and longtermism, according to which digital posthumans should colonize the universe, capture stellar energy with Dyson swarms, and establish a sprawling multigalactic civilization high up on the Kardashev scale. If they believe that, they probably also believe that we should strive to bring about this cosmic utopia through ASI. And so on.

Don’t let TESCREALists fool you when they insist that there’s a large gulf between different TESCREAL communities. This is nothing less than the narcissism of small differences. Take a step back from the ideological map and draw your own conclusion.

4. Doomers Vs. Accelerationists

Similarly, you might hear people claim that it’s absurd to include “doomers” and “accelerationists” — both of whom you met during the party — under the umbrella of TESCREAL. One group wants to stop ASI from being developed whereas the other wants to accelerate progress toward god-like AI. These people are ideological enemies with diametrically opposed positions.

But is this true? Consider the two leading figures of AI doomerism and accelerationism: Yudkowsky and Gil Verdon (Beff Jezos), respectively. Verdon wants to build ASI asap. He doesn’t care whether ASI adopts our values, so long as it’s conscious and proceeds to spread beyond Earth, build Dyson swarms around every star, and establish a multigalactic civilization high up on the Kardashev scale. Once ASI arrives, he’s stated repeatedly that he has no problem with it completely usurping humanity, i.e., bringing about our extinction, though my guess is that he’d welcome the opportunity to become a digital posthuman himself. He thus holds a pro-extinctionist view.

Now consider Yudkowsky. He says that we should immediately shut down all research projects trying to build ASI. Yet he’s not anti-ASI. To the contrary, he also wants to build ASI asap, but only after we know how to align it with our values — i.e., after the value-alignment problem has been solved. Being value-aligned, this ASI will then transform people into digital posthumans who spread beyond Earth, build Dyson swarms around every star, establish a multigalactic civilization high up on the Kardashev scale, and live in vast computer simulations running on planet-sized computers. He argues that once posthumanity arrives, it may be unethical to keep biological humans around, and he suggests that humans should be wiped off Earth so it can become a nature preserve. He’s also said that he’s not worried about humans being replaced by posthumanity, so long as posthumanity is “better,” and once declared on a pro-extinctionist podcast that

if sacrificing all of humanity were the only way, and a reliable way, to get … god-like things out there — superintelligences who still care about each other, who are still aware of the world and having fun — I would ultimately make that trade-off.

Hence, Yudkowsky has on numerous occasions expressed pro-extinctionist sentiments that are nearly indistinguishable from those embraced by Verdon.

What exactly is different about the worldview of Yudkowsky and Verdon? There is really only one significant difference: Yudkowsky wants ASI to adopt our values, and hence enable people like him to survive into a cosmic utopia. That’s why he thinks we need to temporarily pause ASI capabilities research, so that value-alignment research can catch up. Otherwise, these doomer and accelerationist worldviews are identical twins. Both Yudkowsky and Verdon are pro-ASI and pro-extinctionist. Both want the posthuman successors that we create or become to colonize the universe and climb the Kardashev scale. Both believe that enveloping stars in Dyson swarms and building a multigalactic civilization will lead to utopia. Etc.

Insofar as these two men — both school dropouts, as it happens — dislike each other, it’s the result of the narcissism of small differences. On an ideological map, their views occupy almost exactly the same location.

Subscribe now

Conclusion

I wrote this article primarily for skeptics of the TESCREAL acronym. I do not claim to have covered all relevant issues, though I hope this provides a compelling case that the TESCREAL acronym is useful — indeed, indispensable — for understanding what the Silicon Valley worldview is, where it came from, how it developed, and why it’s problematic and dangerous (the last of which I didn’t cover here). Those who object to it likely don’t understand why the acronym was coined or what work it’s supposed to do within critiques of Silicon Valley’s push to build an AI God.

By the way, the people you spoke with at the fictional party are based on real individuals. I tried to present their views as accurately as possible based on what I know about them. Here’s a key in case the hyperlinks weren’t enough:

• Bendisi = Daniel (Bendisi) Kokotajlo

• Niklas = Nick Bostrom

• Bill = Gil Verdon

• Eli = Eliezer Yudkowsky

• Daniel = Daniel Faggella

• Sam = Sam Altman

• Ray = Ray Kurzweil

• Larry = Larry Page

• Mario = Dario Amodei

• Dennis = Demis Hassabis

• Ketamine user = Elon Musk

• Toby = Toby Ord

• Nick = Nick Beckstead

• Holden = Holden Karnofsky

• Peter = Peter Thiel

• Michael = Michael Druggan

• Jeffrey = Jeffrey Ladish

As always:

Thanks for reading and I’ll see you on the other side!

I just left North Macedonia, in the Balkans. It’s a beautiful (middle-income) country, with some marvelous hidden gems and an extraordinary history. For example, during the period of Classical Greece, king Philip II of Macedonia unified much of Greece, enabled by his rather terrifying martial innovation, the Macedonian phalanx. His son, Alexander the Great, who was tutored by Aristotle, then conquered the Persian Empire to the east.

Some history basics: the Classical Period extended from the final Athenian tyrant’s fall in 510 BCE until Alexander the Great’s untimely demise in 323 BCE. It was during this period that Socrates, Plato, and Aristotle philosophized. This was followed by the Hellenistic period, during which people in Macedonia built structures — like the amphitheater shown below — that I was lucky enough to see while visiting. The Hellenistic period begins with Alexander the Great’s death and extends almost to the birth of Christ — i.e., to 30 BCE, when Cleopatra died. Cleopatra was the last de facto Egyptian pharaoh and a direct descendant of the Macedonian general Ptolemy I Soter, who founded the Ptolemaic Kingdom — of Egypt, which Cleopatra ruled — and was a contemporary of Alexander the Great. Fascinating stuff!

The reason North Macedonia includes “North” in its name is rather amusing: the country wants to become part of the European Union, but the Greeks, who are already part of the EU, objected to “Macedonia” without the “North.” The reasoning is that the Macedonian kingdom of Philip II and his son Alexander the Great included most of Greece, and hence the current state of Macedonia covers only the northern portions of this empire. Under Article 49 of the Treaty on European Union, all members states must unanimously vote to allow new states to join, which means that Macedonia needs to get Greece on board. Hence, they changed the name to “North Macedonia” in hopes of appeasing Greece so it votes “yes.” If you talk to people here, it’s Macedonia. But if any Greeks are listening, it’s North Macedonia!

(EDIT: See Wolfgang’s comment below for some clarification. He knows more about this issue than I do!)

***

After spending April in London with a good friend, I spent the past two months sojourning in a lovely lakeside city in Macedonia called Ohrid (pronounced Oak-rid), surrounded by snowcapped mountains (until the snow melted), lush vegetation, and beautiful water. Here’s what Wikipedia says about the place:

Ohrid … is the largest city on Lake Ohrid and the eighth-largest city in the country, with it recording a population of over 38,000 inhabitants as of 2021. Ohrid is known for once having 365 churches, one for each day of the year, and has been referred to as the “Jerusalem of the Balkans”.

(Note that there are also plenty of mosques!)

The city is rich in picturesque houses and monuments, and tourism is predominant. … In 1979 and in 1980, respectively, Ohrid and Lake Ohrid were accepted as Cultural and Natural World Heritage Sites by UNESCO. Ohrid is one of only 40 sites that are part of UNESCO’s World Heritage that are Cultural as well as Natural sites.

I’m sharing some videos and pictures of the city below, along with a few captions. Now off to Western Europe, where I’ll have to endure much hotter temperatures! Once I settle in, I’ll publish a lengthy article (next week) on the TESCREAL ideologies, exploring them from a novel perspective. I’m looking forward to it! Wishing everyone the very best, Émile

View of Lake Ohrid from the lawn outside a cafe that I frequented. Many of my articles over the past several weeks were written here:

A lovely wooden walkway that winds around the hilly old section of the city:

View on the edge of the old city, where the stunning Church of Saint Jovan the Theologian at Kaneo — built sometime before 1447 — is located. You’ll see it at the end:

One of the first videos that I took during a morning run. This is on the opposite side of the hill where the old city is located, about a 10/15-minute walk to the Church of Saint Jovan:

Below is the magnificent Ancient Theatre of Ohrid, a 20-second walk from where I stayed in the heart of the old city. It was built circa 200 BCE, meaning that it dates back to the middle-ish era of the Hellenistic period. Spectators witnessed gladiator fights (!), as well as Roman executions of Christians. The latter was met with such hostility by locals that they subsequently buried the structure in disgust, thus inadvertently preserving it for future generations. It was accidentally rediscovered in the 1980s!

Walking up to my apartment through the old city:

Gateway to one of the 365 churches in Ohrid!

Another view of my “office” on the grass next to the lake. The distant mountain was covered in snow when I first arrived:

Wandering through the old city at night:

Same as above. Notice the chapel in the background:

Samuel’s Fortress, capital of the medieval First Bulgarian Empire. It was likely built on an older structure dating from around 300 BCE, constructed by Philip II. What you see here dates from sometime in the late 900s CE:

At another chapel:

I sent this to my family with the comment: “I hate how compelling this slogan is!”

Adorable — but how often does it tip over while turning?

Macedonian special:

Inside one of the chapels:

These events were held quite frequently while I was here:

As always:

Thanks for reading and I’ll see you on the other side!

I just finished reading Luke Kemp’s recent book Goliath’s Curse, and I’d highly recommend it to readers of this newsletter. It’s a sprawling survey of societal collapse across the entire sweep of civilizational history, and his account of Paleolithic violence offers a devastating refutation of key claims made in Steven Pinker’s The Better Angels of Our Nature.

As it happens, I had a desk right next to Kemp’s when I was visiting the Centre for the Study of Existential Risk (CSER) in 2019. We hung out, chatted about global catastrophic risks over drinks, and kept in touch after I departed Cambridge (though we haven’t spoken much over the past few years). I knew him as an erudite scholar with an unusual talent for approaching complex topics in a neutral, objective manner.

To my pleasant surprise, Goliath’s Curse is even more erudite than I anticipated. It’s a work of superb scholarship, in my opinion, and constitutes a major contribution to the global catastrophic risk (GCR) literature. I changed my mind about a number of important issues while reading it.

There’s no way to summarize Kemp’s tome in a single article. What follows is a loosely organized survey of what stood out to me as key points, drawn mostly from memory and a few quotes I jotted down while reading. Please excuse the spaghetti tangle of thoughts that I’m about to throw at the wall!

“Hobbes’s Delusion”

Kemp argues that life in the Paleolithic wasn’t “solitary, poor, nasty, brutish, and short,” as Thomas Hobbes famously claimed. For roughly 300,000 years, we lived in relatively small, highly egalitarian communities in which group decisions were made democratically. As he notes, “modern nomadic-egalitarian groups who have the lifestyles closest to our deep past are especially democratic.”

Violence wasn’t very common, and “Palaeolithic violence was probably highly personal. With little inequality, there would have been few material rewards for killing others.” Those who tried to dominate others were punished: expelled or, in extreme cases, killed — an example of “personal” violence. If you didn’t like the way your group was being run, you could “vote with your feet” by migrating to another group. There was, indeed, a constant interchange of individuals passing from one group to another.

People were also probably quite happy most of the time. Ethnographies of contemporary hunter-gatherers, Kemp points out, “are replete with reflections on how carefree and happy they are, with close-knit communities and a propensity to spontaneously burst into song and dance.”

With the rise of civilization, malnutrition became common and human health suffered, as reflected in the average height of agricultural populations falling for some 12,000 years (we didn’t regain our height until the 19th and 20th centuries). Higher levels of violence, along with oppression, slavery, patriarchy, and martial conflicts resulted. Life in the Paleolithic wasn’t perfect — infant mortality, for example, was much higher than it is today — but the Hobbesian assumption that life was terrible isn’t supported by the paleoanthropological data.1 Life actually got appreciably worse in many respects after the Neolithic revolution.

Civilization Is Domination

Kemp repeatedly emphasizes that people have an inherent aversion to being dominated. This is why those who tried to dominate others in our hunter-gatherer past were ostracized or executed. Yet domination is a core feature of what he calls “Goliaths,” as he explains:

The problem is that most of us are uncomfortable in recognizing the most common element of civilization: rule through domination. … A more apt label for these systems of violence is “Goliath.” A Goliath is a collection of hierarchies in which some individuals dominate others to control energy and labour.

The emergence of Goliaths amounts to, in his phraseology, evolutionary backsliding. By subjecting people to oppressive hierarchies of dominance, it made us more like our evolutionary relatives, the gorillas and chimpanzees. In his words:

Rather than a stepladder of progress, this movement from civilization to Goliath is better described as evolutionary backsliding. Our egalitarianism and counter-dominance shaped our bodies and minds and supercharged our cultural evolution. They made us unique and helped us navigate the Palaeolithic. The move towards Goliath — whether it be through class or patriarchy — made us look more like the harems of patriarchal gorillas and the chimpanzee hierarchies built on violence and politicking.

Collapse and Inequality

A common denominator of collapse across history is wealth inequality. As a Guardian article about Kemp’s book puts it, “history shows that increasing wealth inequality consistently precedes collapse.”

I was aware of the literature on this, but was never convinced that inequality was a causally central factor in collapse. Kemp changed my mind: incompetence, corruption, wasted resources, overuse of resources, frivolous projects of self-aggrandizement (think of the Colossus of Rhodes or Trump’s proposed “Triumphal Arch”), the embrace of loyalists over experts, nepotism, foolish wars of conquest, etc. etc. follow the emergence of elites with excessive wealth and power.

Such phenomena make Goliaths more susceptible to threats that shock the system: famines, volcanic eruptions, earthquakes, disease outbreaks, incursions from foreign armies, ecological destruction, and so on. Whether these threats pose “existential” risks to society depends on how large the threat is, as well as how vulnerable and exposed the society is, as well as its ability to effectively respond once the threat materializes. As Kemp explains:

In the field of disaster risk, we refer to four determinants: threat, vulnerability, exposure, and response … Take the case of a tsunami. The tsunami (threat) is only dangerous if you lie in its way (exposure), and you lack robust shelter that could withstand the wave (vulnerability) or the ability to escape (response).

Consider the case of climate change. Most US citizens are worried about this. Parents fear that their children will inherit a planet much less livable than the one they grew up in. If there were a popular vote, the US would have taken action to mitigate the problem years ago.

But elites have lobbied the government not to act, while spreading climate propaganda to the masses. Curbing climate change would be bad for their personal wealth, power, status, and ambitions. Meanwhile, loyalists in the Trump administration reject the climatological consensus, with Trump himself calling it a “Chinese hoax.” Consequently, the US is doing nothing to mitigate the problem, and has failed to reduce its exposure and vulnerability to climate change’s increasingly harmful effects. This makes collapse much more likely — thanks to the incompetent, avaricious, power-hungry elites who rule the country.

Paleolithic Altruism

In the aforementioned Guardian article, Kemp says that

as elites extract more wealth from the people and the land, they make societies more fragile, leading to infighting, corruption, immiseration of the masses, less healthy people, overexpansion, environmental degradation and poor decision making by a small oligarchy. The hollowed-out shell of a society is eventually cracked asunder by shocks such as disease, war or climate change.

Indeed, Kemp notes that “the biggest predictor of resilience was how democratic or extractive the society was.” This is substantiated by considerable historical evidence, as he explains:

One study of the Late Antique Little Ice Age found that more inclusive societies had less famine, disease, and conflict, and less change in their rituals or how they were organized. Those results have been verified in another study of thirty-three societies covering twenty-two climate-related disasters over nine world regions, four continents, and a thousand years. More democratic groups tend to respond better to crises.

The question is, why? Local people could respond more quickly and effectively to new challenges than distant elites who were often buffered from threats such as a drought. This meant that more dramatic changes (such as collapse) were needed. A second benefit is that shared decision-making means that people in a community had more social connections, which created more effective collaboration (what academics usually call ‘social capital’) during times of stress. This holds true today: acts such as voting, deliberating, and being meaningfully involved in civic institutions (whether that be a sports club or a political group) foster closer and wider social ties. Those ties keep communities together when catastrophe strikes.

Kemp points out that during and in the aftermath of disasters, people tend to come together and help each other. Mass chaos, violence, and “anarchy” don’t usually take over. Rather, our Paleolithic tendencies toward cooperation and altruism often emerge. I’ve personally experienced a few emergency situations in my life, and it’s amazing how complete strangers suddenly become trusted friends.

Kemp continues:

A plethora of studies on modern disasters support the idea that more politically inclusive societies are more resilient ones. As one group of disaster-risk experts concluded, “there is near-unanimous agreement that people’s participation is essential to reducing the impacts of disasters on society.” One indicative study [analyzed] 150 countries over the period 1995-2009 found that more democratic countries, with more state capacity, had fewer fatalities and fewer people affected during disasters. This was true for a range of disasters, from floods to earthquakes. Strong states have more resources to deal with disasters, while democratic accountability ensures that aid is distributed wisely and not just concentrated on the rich and powerful. Inclusive political institutions create more flexibility, effective responses, and unified societies, and they prevent abuses of power.

Does Collapse Benefit the 99%?

Kemp argues that societal collapse has very often benefitted the 99%. It can actually open the door to more progress, innovation, creativity, art, and knowledge. It frequently makes life better for nearly everyone except the elites.

Two examples are the fall of Rome and the 14th-century Black Death. After Rome fell, “people actually got taller and healthier,” Kemp reports. After the bubonic plague swept across Europe, killing up to 50% of the population,

peasants were now valuable enough to force concessions out of the landlords and barons. Wages doubled or tripled in many regions, and the pay gap between men and women decreased. People grew taller and healthier. Life expectancy increased and heights grew from a low of 168cm during the Black Death to a high of around 174 cm.

However,

while the Black Death was a blessing for the survivors and their descendants, these positive effects were not evenly distributed. … Regions that already had inclusive institutions benefited in the long term from the Black Death, while those without them did not.

He adds:

The fall of Rome and the Black Death not only give us an insight into the surprising relationship between collapse and prosperity, but they can also help us to understand how petty European kingdoms became colonial conquerors.

What’s More Deadly than Collapse?

Indeed, some of the worst events in all of human history were the result not of the fall of Goliaths but their emergence and expansion. He writes:

The emergence of these colonial Goliaths was one of the most violent processes of human history. The colonization of the Americas itself may have claimed up to 10-11 per cent of the world’s people at the time. While the numbers are highly uncertain, only one other event seems to come close to this: the waves of the Black Death in the fourteenth century … We can think of these as “global decimations.”

The cause was, of course, the brutal mass murder of Indigenous peoples by the “more civilized” Europeans. It was also due to the Columbian Exchange, namely, “the widespread transfer of plants, people, pathogens, and culture across the Atlantic Ocean.”

But “there is another critical reason,” Kemp writes, that “colonization efforts succeeded: indigenous allies.” Consider the Aztecs. They were a militaristic Goliath that sought territorial expansion and oppressed the peoples they conquered. He explains:

Those who resisted the imperial expansion of the Aztecs faced higher taxes and harsher working conditions. Many of the conquered territories resented such domination, while unconquered neighbours such as the Tlaxcalans (a confederation of four oligarchic republics) were in a state of near perpetual war with the Aztecs.

The Tlaxcalans thus offered the Spaniards a deal: join us to overthrow the Aztec Empire, to which “the Spanish wisely agreed.” That’s how the Spaniards managed to defeat the Aztecs: it was actually Indigenous peoples rebelling against their neighboring Goliath, because no one wants to be dominated.

What’s Civilized About Civilization?

The key point, however, is that most people looking at history get things exactly backwards: it’s not the disintegration of Goliaths that tend to inflict mass suffering on the 99%, but their very existence. Dominance hierarchies are the real villains of history.

This flips the script that Pinker wants us to believe. Over and over again, Goliaths have inflicted unimaginable horrors on people. “Civilization” seems to imply “civilized,” yet is there anything civilized about the way hierarchies oppress, subjugate, exploit, dominate, and control people through the perpetual threat of violence? As Kemp writes at the very beginning of his book:

[C]ivilization seems to be a wholly inappropriate term for what we see in Cahokia and Rome. It is derived from the Latin civilitas, which implies restraint, moderation, and good political conduct. Slavery, war, patriarchy, sacrificing teenage girls as the priests of Cahokia did, punishing slaves with crucifixion as in Rome, or dropping atomic bombs on cities as the US did less than a century ago, are not what most of us would consider civilized conduct. Orchestrating mass sacrifices or building bombs that vaporize cities does not require altruism, civility, or democracy. It requires top-down obedience enforced through the threat of violence.

In a memorable passage nearly 400 pages later, Kemp says this about states and companies:

States and companies (and perhaps now algorithms) are not just amplifiers of power and decision-making; they also inflate the worst in us: the darker angels of our nature. Think of a modern state, such as the US or Russia, embodied as a single person stalking the streets. Imagine what they are like. Overwhelmingly self-obsessed and self-interested, constantly boasting of their achievements, continuously suspicious of their neighbours, persistently posturing and hoarding guns and knives, and greedy to the extent of never giving even just 1 per cent of their earnings to those in need. That state person, if it took a psychiatry test, would rate disturbingly high on the dark triad (psychopathy, narcissism, and Machiavellianism).

I’d never thought about it this way, but it’s so true! States behave exactly like narcissistic, Machiavellian psychopaths.

The World Is Different Now

Interestingly, Goliaths have actually become more resilient over time. We don’t see the same sorts of collapse in recent history that we did long ago, especially within the Global North. States may “fail,” as happened with the Soviet Union and Yugoslavia, but the basic infrastructure and cultures that comprise those states have persisted. This, however, shouldn’t make us too confident that a genuine, catastrophic collapse doesn’t lie in our future. As Kemp writes:

The world appears to be growing both increasingly robust and more fragile. I call this the “Death-Star Syndrome,” after the space-weapon in the film Star Wars: A New Hope that can annihilate entire planets but can itself be destroyed by a single well-placed blow. Our world is incredibly powerful and robust, yet surprisingly fragile if hit hard enough in the right place.

First, we no longer live in isolated Goliaths that could rise and fall without affecting their neighbors. When Rome fell, other civilizations elsewhere in the world hardly noticed. That’s no longer the case: globalized modernity is so vastly interconnected that a collapse in one place could have profound consequences for all other regions. (As the saying goes, “when the US catches a cold, the rest of the world comes down with pneumonia.”)

Second, we’re now facing threats on a planetary scale. Even a regional nuclear war between India and Pakistan could cause global temperatures to plummet by a staggering 4 C. A nuclear war between the US and Russia could kill over 5 billion people — and a relatively sudden loss of 5 billion people would have truly devastating consequences for the global economy, agriculture, geopolitical stability, and so on. As Kemp observes, the power to decide whether or not to initiate a nuclear war lies in the shaky hands of a select few Dark-Triad madmen: Donald Trump, Vladimir Putin, etc. This is a terrifying concentration of destructive power in people we wouldn’t trust to babysit our children.

There are also global-scale threats from climate change, engineered pandemics, artificial superintelligence, and killer robots, to name a few. While certain risks have declined over the centuries, Kemp notes that “fat-tailed” risks associated with such phenomena have notably increased. Consider this curve:

This shows that low-impact risks (on the left) have a higher probability of occurring than high-impact risks (on the right). However, the “fat-tail” on the right means that some very low-probability risks would result in widespread, global-scale harms. You can think of it like this: most people who will ever die in pandemics will die in the absolute worst pandemics. Those pandemics — falling on the far right of the graph — will take so many people that they’ll overwhelm the numbers of people who’ve died from far more common localized outbreaks and epidemics. Here’s how Kemp puts it:

You are about twice as likely to take your own life than to be murdered. You are more likely to die from cancer than from any infectious disease. Unfortunately, this does not mean that we live in the safest of times or are on the cusp of defeating disease. Far from it.

Deaths from disease and violence follow a “fat-tailed distribution.” They are marked by a few outsized events, such as the Spanish flu, the Black Death, and the Columbian Exchange. This makes observing an average from one year, or even ten, essentially meaningless. The average will be brutally skewed by these “tail events.” The extremes, not the average, are the main source of risk. Importantly, these tails are becoming fatter and longer over time. Even if the world on average seems safer and healthier, the extremes are growing in likelihood and severity.

Advanced AI, synthetic biology, climate change, biodiversity loss, and perhaps future nanotechnology carry fat-tailed risks such that, if a low-probability worst-case outcome were to occur, it would decimate the world.

Making matters worse, wealth inequality has skyrocketed. The catastrophe of World War II leveled things off a bit, but disparities have risen since the 1970s. This harbinger of collapse has probably never been more salient, certainly not on the global stage. The world is run by people who are actively destroying it, while those same people prepare for the coming collapse by building apocalypse bunkers in Hawaii and New Zealand.

Here’s how Kemp explains the unsettling uniqueness of this perilous period in the Guardian:

He cites three reasons why the collapse of the global Goliath would be far worse than previous events. First is that collapses are accompanied by surges in violence as elites try to reassert their dominance. “In the past, those battles were waged with swords or muskets. Today we have nuclear weapons,” he says.

Second, people in the past were not heavily reliant on empires or states for services and, unlike today, could easily go back to farming or hunting and gathering. “Today, most of us are specialised, and we’re dependent upon global infrastructure. If that falls away, we too will fall,” he says.

“Last but not least is that, unfortunately, all the threats we face today are far worse than in the past,” he says. Past climatic changes that precipitated collapses, for example, usually involved a temperature change of 1C at a regional level. Today, we face 3C globally. There are also about 10,000 nuclear weapons, technologies such as artificial intelligence and killer robots and engineered pandemics, all sources of catastrophic global risk.

Yikes.

Those Damn Agents of Doom

All of this “existential risk” — a term I don’t like, although Kemp isn’t using it the way as TESCREALists do — comes from a small group of elite actors that he calls “Agents of Doom.” This refers to the Big Oil fossil fuel companies, the Big Tech AI companies trying to build superintelligence, and military-industrial complexes (“a collection of militaries and the arms manufacturers they tend to contract to”). It’s these companies, run by assholes who likely score high on the Dark Triad traits, that are responsible for pushing civilization toward disaster.

I sometimes hear radical environmentalists say, “So what if humanity goes extinct due to climate change? We deserve extinction given how we’ve trashed the biosphere.” My response points out the obvious: “This misdiagnosis the problem. It’s a tiny elite of endlessly avaricious capitalists with a fetish for the accumulation of infinite wealth and power who are responsible for our predicament. If climate change were to take only them out, there might be a degree of justice served. But the 99.9% don’t deserve to die for the actions of a tiny few.” This response generalizes to superintelligence, bioweapons research, nuclear proliferation, and so on. If our global Goliath collapses, blame the Agents of Doom.

What, then, can we do?

Termination, Chains, or Overthrow

Kemp argues that we have three options before us:

Looking at both the long-run trajectory of Goliath and current trends, our Global Goliath appears destined to hurtle down one of three paths. The first, and most likely, is self-termination. The second is a world in chains. The third is a world in which we somehow manage to shackle and control it.

Kemp argues that we must revive aspects of our Paleolithic lifeways, namely, governance through democratic means. He writes:

Open democracy and inclusive institutions are essentially harnessing these better angels of our Palaeolithic psychology. They are using the best of us to make better decisions and closer communities: to harness our collective intelligence and impulses to help each other. They are moving closer to our past as equals in a web of sharing, constantly moving communities: a fluid civilization. Our Palaeolithic emotions are not a curse. The curse is the institutions and arrangements which bring out the worst in us.

We must therefore empower the people, utilize the wisdom of crowds, and strive to develop novel forms of democratic control to collectively guide the ship of civilization through the unprecedentedly dangerous 21st century. “If we start to properly fund experiments in democracy,” Kemp writes, “and reward them like we reward start-ups in tech, then who knows in what profound new ways we’ll be able to harness the wisdom of crowds.”

Insurmountable?

Unfortunately, Kemp notes that “reversing inequality and establishing deep democracy in a lasting fashion has … been a rarity throughout history.” But it’s precisely because this “is a highly unlikely outcome” that

makes it worth fighting for even more desperately. While throughout history the collapse of a Goliath was usually temporary and a liberation, in the future it threatens to be permanent and take us and much of the planet with it. We are passengers on a journey that looks likely to end in chains or evolutionary suicide. Our Global Goliath will die explosively unless we kill it first.

Indeed, this points to a disheartening fact about future collapse scenarios: not only could they take the entire global system down, but the world left behind would be far less welcoming than the world was after past collapses. As Kemp writes, “it will be trickier in the future for a population to bounce back post-collapse.”

Climate change will, as alluded to above, render large portions of the planet subject to extreme weather events and unsurvivable heatwaves (exceeding the 95 F wet-bulb temperature), while global biodiversity loss and the sixth major mass extinction event will leave ecosystems far more impoverished than they otherwise would be. Consider that the global population of wild vertebrates — mammals, fish, reptiles, birds, amphibians, etc. — has declined by a staggering 69% since 1970. Extrapolate this trend into the future, and then imagine your children or grandchildren trying to survive by hunting, gathering, and fishing.

Extinction Vs. Collapse

An interesting point that Kemp brings up, which I’ve made on numerous occasions over the years, is that total human extinction is “far-fetched,” at least within the foreseeable future. He writes that

even an extreme collapse with extensive infrastructure damage, temporary climate change, and a population loss of 99.99 per cent would have only a 0.1-0.3 per cent chance of causing extinction. There would still be 810,000 survivors, who would have a “grace period” of easy access to remaining supplies (canned goods and produce) and equipment. That might buy them enough time to learn the necessary survival skills such as basic foraging and agriculture. Killing off every single human is an incredibly high bar.

I agree with this.2 I’ve argued that the probability of extinction is very low, though I’d quickly add that the probability of collapse this century looks exceptionally high — on the order of >90%. In other words, I would be very surprised if civilization survives beyond the next 74 years or so. If we could overthrow the oligarchy and put everything under democratic control, as Kemp advocates, then I’d say we have a pretty good chance of making it through. But that also seems quite unlikely — “it’s easier to imagine the end of the world than the end of Goliath,” to quote Kemp. However, he adds that this

is because we are still stuck in Goliath traps: status races, arms races, and races to the bottom (where firms or states try to outcompete each other by exploiting people and environments more intensely). Decreasing nuclear stockpiles, slashing carbon emissions, and making our societies more democratic are all completely feasible, and doing so in the long term will mean escaping these traps and digging up the root causes of existential risk.

Some Criticisms

I have many friendly criticisms of Kemp’s magisterial book. One of the most pressing is that he doesn’t go far enough in talking about how certain Agents of Doom, namely, the AI companies, are trying to bring about a version of human extinction in which our species is replaced by a successor species of posthumans. If the TESCREAL worldview were successfully realized, the result wouldn’t be collapse — yet the extinction of our species would almost certainly result. See my “Pro-Human Manifesto” for all the gory details.

The usual way of thinking about the relation between extinction and collapse within GCR studies is this: extinction would entail collapse, but collapse need not entail extinction. That’s true if one’s talking about final human extinction, but not terminal without final extinction. And it’s precisely terminal without final extinction that many TESCREALists would hope to realize, if they’re successful. Because avoiding final extinction would by definition mean that we have successors, our “civilization” could persist even if our species were to completely die out.

I wish Kemp had spent a bit more time on this extremely important issue, though as someone who spent years writing a book of comparable length, I understand how utterly exhausting it is to try to mention everything that could or should be mentioned. The lacuna is completely understandable. Perhaps he’ll address this when a second edition comes out!

As always:

Thanks for reading and I’ll see you on the other side!

A central theme of this newsletter over the past year has been the pro-extinctionist implications of the TESCREAL worldview. Many people in Silicon Valley want to create a new species of posthumans to replace us, and many see artificial superintelligence (ASI) as the key to doing this. Companies like Anthropic aren’t merely building a technology, they’re trying to realize a worldview.

However, as I note in Part 1 of my “Pro-Human Manifesto,” there are many different kinds of pro-extinctionism circulating in the Valley. The four main questions that these pro-extinctionists disagree about are:

1. Whether posthumans should be digital or biological. Most Valley dwellers claim that posthumanity should take the form of digital beings, a view that I call digital eschatology. However, a small minority of people, like Peter Thiel, embrace an eschatology of biological transhumanism, according to which some or all future posthumans should be biological.

2. Whether posthumans should be an evolutionary extension of our species or take the form of autonomous AI systems that are, as such, separate and distinct from us. In the first case, humanity (or some portion of humanity — the elites) would be transformed into posthumanity, which would then replace our species. In the second, we would essentially create a new evolutionary lineage de novo. These AIs would then supplant humanity like an invasive species let loose on an island. We can call this second option digital eugenics.

3. Whether posthumanity must embrace “our values” (whatever they are) or should adopt their own unique values, which may be wildly different from ours. Accelerationists like Gil Verdon (aka “Beff Jezos”) don’t seem to care one bit whether posthumanity embodies “our values.” As Daniel Faggella puts it, they want our “successors” to have values that are “alien, inhuman.” In contrast, the field of “AI safety” is founded on the idea that it matters greatly whether posthumanity is “aligned” with “our values” — hence, the value-alignment problem. If ASI isn’t value-aligned, then the future won’t contain human values, extended to the stars. The result would be an “existential catastrophe.”

4. Whether particular individuals alive today should have the opportunity to become one of these future (most likely digital) posthumans. Many TESCREALists — suffering from debilitating death anxiety — think it’s very important that they themselves get to join the ranks of posthumanity in a cosmic utopia someday. This is why people like Ray Kurzweil, Nick Bostrom, Eliezer Yudkowsky, and Sam Altman have arranged to have their brains cryogenically frozen or chemically preserved if they die — so they can be resurrected in posthuman form.1 Others would say that what ultimately matters is the creation of posthumanity — full stop — whether or not specific individuals survive the transition to a new posthuman era. This gets at a difference in emphasis between what theologians would call “personal eschatology” and “cosmic eschatology,” respectively.

These are the central points of disagreement among Silicon Valley pro-extinctionists, and sometimes those disagreements can be vicious (see the “narcissism of small differences”!). I’ve argued that we will witness increasingly public debates among the tech elite not about whether pro-extinctionism is bad, but about which type of pro-extinctionism is best. I think the Musk-Page debate is an example of this.

***

If you’d like some concrete examples of the different pro-extinctionist views that these disagreements yield, check out my “Pro-Human Manifesto.” Since publishing that, additional examples have popped up.

For instance, consider the following quote from Dean Ball, who was just recently hired by OpenAI “to lead a newly formed Strategic Futures team focused on shaping the company’s approach to frontier AI policy and governance.” Ball also has affiliations with the Foundation for American Innovation, the Heritage Foundation, and Yale University, and he was the Senior Policy Advisor for Artificial Intelligence and Emerging Technology for the Trump administration. In a recent interview, he says this:

Weirdly enough, if you think that this moment is — I don’t necessarily believe this — but a lot of people would say we’re living through this kind of eclipse of the human intellect, where we’re in the final days of humans being the primary actors on this planet. And that soon, machines will rise. There is this irony in that I think that whole transformation, I think humans … will actually go through a very “main-character energy” period of time, as that transformation occurs, even if it ultimately does mean that the machines ultimately become the primary actors, there’ll be this period. … In that sense, it’s a very beautiful time period to live through because in a Dionysian way, there’s a lot of ugliness and bad, but there’s a beauty in the ugliness … when a star dies, it grows super-big into the Red Giant, and it’s like that … as you watch this final flowering of humanity in the birthing of the machine intelligence, it’s like, you see this greatness in human effort. And I feel like we do see some of that going on in the world. I think we’ll see much more of it. I think it will be a heroic time period that we live through, basically, that’s what I’m saying. At least it could be — or villainous time period, but there’ll be a lot of opportunities for great people.

Put differently, we’re about to see the desperate last gasps of humanity as our digital progeny smother us to death. This is the “final flowering of humanity in the birthing of machine intelligence,” yielding a “heroic” — or “villainous” — period during which “great people” will have “a lot of opportunities.” The world as we know it will come to an end, and a new era of AI super-beings will commence. This is it, folks!

The arrogance, hubris, and tyranny of this vision is difficult to overstate. Sorry for cursing, but who the fuck are these people to decide for all of humanity when, how, and whether a Digital God arises, thus sentencing our species to extinction? I’m reminded of a quote from Sam Altman, who said in 2016: “We’re planning a way to allow wide swaths of the world to elect representatives to a new governance board. Because if I weren’t in on this I’d be, like, why do these fuckers get to decide what happens to me?”

Sounds good, right? Except that OpenAI didn’t follow through — they never allowed representatives of people around the world to participate on its governance board. Instead, they stacked the board with people like Quora CEO Adam D’Angelo, Facebook CTO and Salesforce co-CEO Bret Taylor, American attorney Nicole Seligman, and Altman himself. “Fuckers,” indeed.

***

Or, consider an OpenAI employee who goes by “roon” online (his real name appears to be Tarun Gogineni). In a recent exchange on X, someone said this:

humans are broken, rotting robots. transhumanists are correct to be disgusted with this situation.

Roon responded:

but [if] you were right[,] though, nothing human or even transhuman makes it out of the near future.

The original poster answered:

I want to survive, roon! just not as chopped unc,

where “unc” is slang for “uncle,” and is sometimes used “to imply that someone is old, getting old, or acting older than their age.” Roon then replied once more:

I think we can and will survive, just in a curtailed lightcone as enhanced pets to creatures of incomprehensible power if we let the technology evolve naturally2

But could “we” survive as “enhanced pets,” as roon claims? Depending on the nature of these “enhancements,” it probably wouldn’t be “us” who survives. Even if “we” do survive, why the hell would anyone want to be a “pet” owned and controlled by “creatures of incomprehensible power”? This utterly bleak future appears to be a good-case scenario from roon’s perspective. There really is no other word than “fuckers.”

***

It reminds me of Altman saying that the only way humans will survive the arrival of ASI is by “merging” with machines. In other words, to avoid extinction (annihilation), we will need to undergo a different form of extinction (“the merge”). The fact that Altman and his employees, like roon, are attempting to unilaterally impose this future on humanity without our consent or permission is utterly infuriating. Who do these “fuckers” think they are?!

Roon, by the way, previously declared on X that

things are accelerating. Pretty much nothing needs to change course to achieve AGI … Worrying about timelines is idle anxiety, outside your control. You should be anxious about stupid mortal things instead. Do your parents hate you? Does your wife love you?

As I wrote about this in Truthdig, he’s saying that

AGI is right around the corner and its development cannot be stopped. Once created, it will bring about the end of the world as we know it, perhaps by killing everyone on the planet. Hence, you should be thinking not so much about when exactly this might happen, but on more mundane things that are meaningful to us humans: Do we have our lives in order? Are we on good terms with our friends, family and partners?

He also once said this:

Make no mistake: these people are ideological extremists whose worldviews are intimately bound up with pro-extinctionist fantasies. They talk about “utopia,” but this utopia isn’t for us — it’s for the posthumans who replace us.

The Climate Crisis Heats Up

In 2008, when I was still a graduate student in neuroscience, The Guardian published an article about James Lovelock, originator of the Gaia hypothesis. He discusses climate change, and the article closes with this paragraph:

What would Lovelock do now, I ask, if he were me? He smiles and says: “Enjoy life while you can. Because if you’re lucky it’s going to be 20 years before it hits the fan.”

Not too bad in the prediction department! As you probably know, Europe is in the midst of a massive heatwave — the second of 2026 so far. Parts of France and Spain are melting in 111 F (44.4 C) temperatures, and records are being broken all over the place. Here are some screenshots of forecasts that I took a few days ago:

Others shared temperature forecast maps on social media showing the sizzling heat covering France, Spain, and elsewhere:

One person shared a fictional forecast from 2014 of what European temperatures will be in 2050, juxtaposed with an actual forecast for June of this year. Wild stuff!

The atmospheric heat is accompanied by an equally massive spike in marine temperatures, as this person points out:

All of this, by the way, is happening before the Super El Nińo kicks in! Next year is going to be horrific …

***

Interestingly, Europe is warming up twice as fast as the rest of the world (while “the Arctic is heating nearly four times faster than the global average rate and twice as fast as previously expected”). But if the AMOC and Gulf Stream collapse — which appears increasingly likely — Europe will fall into a “little ice age,” whereby temperatures will plummet to roughly the same levels as in Canada. (Note that Paris is slightly south of Winnipeg, Canada, while London is slightly north of Calgary.)

This will devastate European agriculture, with effects potentially greater than those inflicted by the sort of heatwaves it’s now experiencing.

It’s hard to overstate how infuriating this predicament is. I agree with scholars who argue that fossil fuel CEOs and political figures like Trump should be tried for crimes against humanity. If indeed 2 to 4 billion people die prematurely in the next few decades due to climate change, is there any punishment that could possibly fit the crime?

Subscribe now

***

Incidentally — to tie things back to the first half of the article — I had a conversation with a TESCREAL colleague back in 2017 that went like this:

Climate change won’t pose an “existential risk” (an event that threatens the creation of cosmic utopia), he said, until later this century. But we will likely build ASI in the next decade or so. If the ASI is misaligned, it will kill everyone on Earth and annihilate the biosphere, thus rendering climate change irrelevant. However, if it’s aligned, it will use its god-like powers of mentation to devise a way of reversing the climate crisis, thus rendering climate change irrelevant. Hence, we shouldn’t worry about climate change, but instead focus on building an aligned ASI.

Many people in Silicon Valley seem to accept this view. Others appear to think that climate change is irrelevant because we’ll all be digital mind-clones living in virtual reality worlds soon, so solving climate change at any point doesn’t really matter (because, you know, who gives a damn about other species?).

Former Google CEO Eric Schmidt channeled the view of my old TESCREAL colleague when he said in 2024: “My own opinion is that we’re not going to hit the climate goals anyway because we are not organized to do it and yes the needs in this area [AI] will be a problem. But I’d rather bet on AI solving the problem than constraining it.”

A similar sentiment was recently satirized by an Instagram account named “bpd.news,” which made up this quote from Jeff Bezos:

“We have to look at the macro-picture of our planet’s future,” Bezos reportedly stated during a recent tech symposium. “Biological limits are real, but digital potential is infinite. If we starve our data infrastructure of cooling resources just to sustain baseline human comfort, we are actively delaying the birth of a super-intelligence that could solve all of our resource problems in the first place. Sometimes you have to prioritize the intelligence that will save us over the biology that slows us down.”

Although Bezos never said that, it’s a pretty accurate description of what the tech elites actually believe! I think that’s what the satirists were going for.

That’s it for today. I hope everyone is well. As always:

Thanks for reading and I’ll see you on the other side!

First, I’d like to publish an AMA — ask me anything — next week. If you have any questions you’d like to ask me about anything I’ve written or said, please leave them in the comments section below!

You’re more than welcome to criticize the positions I’ve defended in this newsletter or elsewhere. I really mean it when I say: please feel free to challenge me, as I appreciate push-back and don’t assume that I’m right about everything! I will do my best to answer everyone’s questions. :-)

“Fuck Earth” — Elon Musk

I have a new article out in Truthdig. Please check it out if you have the time. It argues that Musk and SpaceX are lying to investors. “Duh,” I hear you saying while moving the cursor to close this tab, but I haven’t seen anyone make the particular argument that I advance. Briefly put, it’s this:

1.

Musk has repeatedly claimed that he’ll place “humans” — including “you” — on Mars. But there is no scientifically plausible way for biological humans to survive on the Red Planet.

1.1

Mars is extremely hostile to biological life. It is literally less habitable than Earth was during the mass extinction event 66 million years ago in which all non-avian dinosaurs died out. How do we know? Because mammals survived the asteroid blast and ensuing impact winter. If you were to put a mammal on Mars, it would die almost instantly. Mars has almost no atmosphere, there’s no magnetic field, and the soil (Martian regolith) is literally poisonous. So is the water. Plus the average temperature is -80 degrees F, about 50 degrees lower than the average temperature on the peak of Mount Everest in January. Does living on top of Mount Everest for years or decades sound appealing? Well, it would be a tropical beach paradise with palm trees and coconuts compared to life on Mars.

1.2

Even just getting to Mars will be extremely difficult. High levels of space radiation (which causes cancer, dementia, etc.), the deleterious effects of prolonged microgravity, and the psychological toll of being cooped up in a tiny spacecraft for 9 months make traveling to our planetary neighbor enormously challenging.

There is no Planet B for us, in our current form. The farthest colonies from Earth that biological humans will be able to occupy will be on the moon, or perhaps within large spacecraft like O’Neill cylinders (which themselves pose enormous engineering challenges).

2.

The only “creatures” who will successfully colonize Mars will be digital in nature. There are two options here:

2.1

We could create autonomous AIs to fulfill our cosmic Manifest Destiny. Think of a more advanced form of Grok. We could give them robotic appendages and launch them to the Red Planet. The problem is that it would be them rather than us colonizing Mars.

2.2

We could upload our minds to computers, and somehow connect these minds to robotic bodies. By transferring our minds to hardware, “we” could more easily survive the brutal conditions of space and treacherous features of Mars.

These are not, of course, mutually exclusive. Uploaded humans in the form of software minds could travel to Mars alongside autonomous AI astronauts.

3.

However, at least some of these fully digital beings will need to be conscious, because the central mission of SpaceX is to spread “the light of consciousness” into the solar system and beyond. If these digital beings were to lack the capacity for subjective experiences, they’d be no different than the machines we currently have on Mars. And no one thinks that those machines mean we’ve colonized the planet. But there are major problems with this requirement:

3.1

Mind-uploading might not be possible. It could be that mental states, including states of consciousness, aren’t computational in nature. Maybe minds aren’t “software” running on the “wetware” of nervous systems. Maybe we’ve been misled by the most recent metaphor in a long chain of techno-metaphors: the brain is a machine of some sort, a telegraph, a telephone switchboard, and now a digital computer. If the brain isn’t a computer and our mental states aren’t algorithmic, then mind-uploading won’t be possible.

There are plenty of other reasons mind-uploading might not work. The lumps of Jell-O between our ears, which jiggle every time we move our heads, are the most complex objects in the known universe. Assuming the brain is a computer, we have absolutely no idea what level of detail would need to be simulated in silico to reproduce mental states, including consciousness, or whether this level of detail could be simulated in silico at all. Mind-uploading might be impossible for technological rather than philosophical reasons.

This matters because if mind-uploading is impossible, then digitized humans will never walk on Mars. SpaceX should be very anxious about this.

3.2

More generally, artificial consciousness of any sort might not be possible. What if we build an AI with a completely different cognitive architecture relative to our brains (like the LLMs that run current AI chatbots)? How could we ever know whether it’s conscious or not? What scientifically objective “test” could we use to verify with virtual certainty that it’s conscious?

SpaceX should also be very nervous about this. Without robust verification, there’s no way to be confident that “the light of consciousness” has in fact been extended to Mars, even if Mars is teaming with artificial beings building what appears to be an advanced civilization.

4.

SpaceX will need to solve all of these formidable problems to fulfill its mission of spreading “the light of consciousness” to Mars and beyond. It will need to figure out how to upload our minds to computers, which might be metaphysically or technologically impossible.

If that’s the case, SpaceX will need to figure out which types of cognitive systems can realize states of artificial consciousness, assuming that computer hardware can realize any such states at all. Maybe computer hardware isn’t the right material, with the right properties, to give rise to phenomenal experience.

But even if it is the right material, SpaceX will need to develop a “test” for consciousness, which is an extremely challenging if not impossible task in itself, because consciousness is intrinsically subjective and science relies on objective (non-subjective) evidence. How could you obtain objective evidence for something that’s intrinsically subjective?

Mission Impossible: Four Challenges

The entire mission of SpaceX fundamentally depends on solving these immense problems. Does Musk admit this in public? No, because he’d spook investors, most of whom are under the (false) impression that humans will someday soon live happily in futuristic Martian colonies.

To put the point a different way, here’s a list of the challenges facing SpaceX:

• (a) Figure out a way to terraform Mars so biological humans can live on it. SpaceX would need to create a Martian atmosphere; eliminate the poison in Martian soil and water; overcome the hazards of low-gravity and microgravity, super-subfreezing temperatures, and there being no planetary magnetic field; figure out how to grow crops; and build spaceships that fully protect their inhabitants from deadly space radiation and other lethal hazards during the 9-month trip to Mars. This would be unbelievably difficult, if it’s possible at all. We can’t even terraform Earth — in the midst of a climate crisis that could take 4 billion lives several decades from now — and we tried to create a “closed biosphere” with the Biosphere 2 experiment and failed miserably. We aren’t even close to figuring out how to establish self-sustaining colonies on Antarctica, which is heaven compared to Mars. Good luck doing all of this 142 million miles from Earth. If something goes wrong and you run out of food or oxygen, you’ll have to wait (at least) 9 months for supplies to arrive. While waiting, you’ll die.

Or:

• (b) Figure out how to upload human minds to computers. Then figure out how to connect these disembodied software beings to robotic appendages that would enable them to interact with the Martian environment. Musk and his fanatical fans utterly fail to appreciate the challenges here, assuming these things are even possible.

And/or:

• (c) Figure out how to instantiate consciousness in artificial systems with non-human architectures — some advanced version of Grok.

And, crucially:

• (d) With respect to (b) and (c), devise a highly reliable and accurate “test” to verify that these artificial systems are actually conscious. This is absolutely critical. The entire mission depends on knowing that whatever beings end up on Mars aren’t mindless machines like the rovers currently picking up rocks and conducting little experiments on our planetary neighbor. Pfff.

Musk says he wants to take the “fiction” out of “science fiction.” But doing that will be far more difficult than most people naively assume. This isn’t just about building a giant phallus and launching it into space. There are major open questions in both AI and philosophy that will need to be answered for SpaceX to fulfill its grandiose mission.

No one talks about these, but they’re just as integral to the mission as the science of rockets is — and perhaps even more formidable than the famously formidable challenges posed by rocket science. Like brain surgery, rocket science is “very complex”:

Musk cannot wish away these challenges. He’s a professional bullshitter, but reality keeps slapping him in the face. That’s why, as I write in the Truthdig article:

[G]ullible investors remain entranced by his messianic aura. In 2016, Musk confidently predicted a self-sufficient Martian colony with 1 million people by 2024. That same year, SpaceX announced that it was “planning to send [the rocket] Dragon to Mars as soon as 2018.” Musk later “said the first unmanned missions could take place in 2022,” and by 2020 he was predicting the first human missions to Mars by 2026. When named the 2021 Person of the Year by Time, he told the magazine that he’d “be surprised if we’re not landing on Mars within five years.” When 2026 arrived, he shifted once again in promising that “an unmanned Starship would soon depart for Mars, paving the way for human-led missions,” and “if those landings go well, then human landings may start as soon as 2029, although 2031 is more likely.”

Early this year, SpaceX abruptly pivoted from colonizing Mars in the near future “to building a self-growing city on the Moon,” which Musk claims his company “can potentially achieve … in less than 10 years.” He now says that “Mars would take 20+ years.”

Lol. He will, of course, continue to push back the date beyond “20+ years,” because reality can’t be wished away. As always:

Thanks for reading and I’ll see you on the other side!

Was Hitler a mass murderer? You may be surprised to know that there’s no historical evidence that he ever personally killed anyone — other than himself in 1945.1 (See, e.g., here and here.) Military records indicate he took no lives during World War I. He apparently didn’t murder anyone during the Beer Hall Putsch or the Night of the Long Knives. And he probably didn’t shoot his niece, Geli Raubal, despite rumors to the contrary.

If “murderer” means “one who personally kills another,” then Hitler wasn’t a mass murderer, because he wasn’t a murderer at all.

But of course Hitler was a mass murderer. Who would argue against this? His decisions resulted, whether intentionally or not, in an estimated 17 million civilian deaths during World War II. Though he never once pulled the trigger, tightened the rope, or held the knife, Hitler was morally responsible for these deaths. If it weren’t for him, those people wouldn’t have died the way they did, when they did.

Subscribe now

By the very same criterion, Elon Musk is a mass murderer.

To my knowledge, Musk has never personally killed anyone. But he did terminate USAID last year, calling it a “criminal” organization that “has to die.”

“USAID was not just an apple with a few worms inside,” he wrote on X, “it was a festering ball of worms in the shape of an apple.” He also said it “is/was a radical-left political psy op.” DOGE then engaged in what one senior official described as a “mafia-like takeover” of the agency.

According to the Lancet, the most prestigious medical journal in the world, the termination of USAID is projected to cause

more than 14 million additional deaths by 2030, averaging more than 2.4 million deaths per year. These deaths include 4.5 million among children younger than 5 years, or more than 700,000 deaths annually.

Late last year, Harvard reported that “hundreds of thousands” of people have already died since USAID was terminated. As of April 2026, the number may have risen to 1.6 million — and counting.

***

Hitler was responsible for 17 million civilian deaths over six years (1939 to 1945). Musk will likely be responsible for 14 million deaths over five years (2025 to 2030), if the Lancet’s projections are accurate. If Hitler was a mass murderer, then so is Musk. Indeed, given the numbers, one is forced to conclude that Musk is a mass murderer of world-historical proportions — on par with some of the worst dictators in human history.

Consider that:

• Saddam Hussein “was responsible for the deaths of hundreds of thousands of people” during his 23-year reign of terror in Iraq.

• George Bush was responsible for upwards of 1 million civilian deaths by having illegally started the 2003 Iraq War.

• Pol Pot was responsible for some 2 million deaths in four years during the Cambodian genocide.

• Stalin was responsible for about 1 million deaths, though another 2.3 died from “neglect and irresponsibility.” The Soviet famine of 1932-33 took another ~6.5 million.

• Mao was responsible for around 30 million deaths over two years due to the Great Leap Forward.

As with Hitler, there is no verified, universally accepted evidence2 that Saddam Hussein, George Bush, Pol Pot, Stalin (after becoming leader of the Soviet Union), or Mao ever personally killed anyone.

Think about that for a moment. These monsters took lives by pulling the levers of power available to them. They implemented policies that killed untold millions due to starvation, deprivation, neglect, and outright slaughter. In some cases, the deaths weren’t specifically intended. But we still count them nonetheless, because they could have — and should have — been anticipated.

By the numbers, Musk is worse than Hussein, Bush, Pol Pot, and Stalin, but not as bad as Hitler or Mao — though he’s not far behind Hitler. Very few people in history have been personally responsible for so many innocent deaths as the CEO of SpaceX, the world’s first trillionaire.

To put 14 million deaths in further perspective:

• Hurricane Katrina took the lives of 1,392 people.

• The atomic bombings of Hiroshima and Nagasaki killed about 246,000 people.

• The Antioch earthquake in 526 CE caused 250,000 deaths.

• The Rwandan genocide took the lives of up to 662,000 Tutsi.

• The HIV/AIDS epidemic has killed some 700,000 people in the US, as of 2018.

• The Armenian genocide took the lives of up to 1.5 million people.

• As many as 13 million civilians died in World War I.

These are all events that, insofar as people know about them, carry a certain unsettling “dread factor.” Yet, by the numbers, all are less bad than the catastrophe unfolding as a result of USAID’s termination by Musk.

Note also that the 14 million estimate doesn’t include deaths that will occur after 2030. The loss of USAID could have effects that linger for much longer.

***

Musk didn’t kill all of these people by ordering their execution. But so what? Hitler also didn’t order the execution of every person his regime buried.

What’s the difference between smothering a hospital patient with a pillow and intentionally unplugging their life-support machine? Would you not consider the person who intentionally unplugs the machine as having committed cold-blooded murder? Would they not be a killer?

Or, imagine seeing a young child drowning in a shallow lake. An altruistic Samaritan runs into the water to heroically rescue the innocent life. But just as she picks up the child, a random asshole intervenes, forcibly flings the baby from the Samaritan’s arms, and the baby drowns. Then you find out that this random asshole was Musk himself.

This is what he’s done with USAID. It was providing life-saving medical support to millions of people around the world. It was their life-support machine. And he pulled the plug, resulting in millions of patients — many of whom are innocent children drowning in a metaphorical lake — dying, often in excruciating circumstances. As my friend Devin Kellis wrote in his guest post for this newsletter:

Peter Donde (age 10), Evan Anzoo (age 5), and Achol Deng (age 8) died from loss of access to human immunodeficiency (HIV) medications; Gy Gael (age 8) died from an infection without access to antibiotics; Babagana Bukar Mohammed (age 7) died from complications of sickle cell disease amidst health clinic closures; and Yamah Freeman, a mother of two, bled to death amidst ambulance delays due to fuel shortage while expecting her third child.

Even if Musk didn’t fully understand the extent to which USAID was the life-support machine for tens of millions, it was his moral duty — as a person who forced his way into a position of immense power — to investigate. There is no ethical excuse for the world’s richest person to set fire to an organization that just 5 minutes of research would have shown him is keeping millions of children above water. He is blameworthy for all of these deaths. Can anyone seriously argue to the contrary?

***

If one compares Musk to Ted Bundy, his murderous actions are completely off the charts. Ted Bundy confessed to 30 murders but might have killed up to 100 women. The latter number is 0.000714% of the total deaths caused by Musk within the next 4 years, according to the Lancet.

Why does Bundy strike fear, dread, and disgust in our hearts but not Musk, whose body count is literally 140,000 times larger? Why does the evil of Bundy disturb our conscience more than the actions of DOGE and its now-trillionaire leader? Is it just psychic numbing and scope neglect? As Stalin famously said, “If only one man dies of hunger, that is a tragedy. If millions die, that’s only statistics.”

When considering the enormity of Musk’s moral crimes, I often think about a poignant scene at the end of Charlie Chaplin’s Monsieur Verdoux, about a man who murders women to steal their money (as I recall — it’s been decades since I watched it). During his trial, Chaplin’s nefarious but charming character delivers a speech in which he says:

As for being a mass killer, does not the world encourage it?
Is it not building weapons of destruction for the sole purpose of mass killing?
Has it not blown unsuspecting women and little children to pieces — and done it very scientifically?
As a mass killer, I am an amateur by comparison.

That final line haunts me. Bundy was an amateur by comparison to Musk. The degree of callousness and psychopathy required for Bundy to do what he did is unfathomable to morally normal people like you and me.

But the same is true of Musk — it takes a real psychopath with zero conscience, empathy, or compassion to recklessly obliterate an agency that kept millions of children alive, and then express literally zero remorse or compunction for the profound and lethal harms directly caused by his intentional, willful actions.

Fuck the billionaires, but especially fuck the trillionaire. We live in an infinitely sad world, because the chasm between how good things could be and how bad they actually are is enormous and growing. It didn’t need to be this way.

Subscribe now

As always:

Thanks for reading and I’ll see you on the other side!

What follows is an urgent call to boycott SpaceX, given Musk’s termination of USAID (among other things). We’re hoping this catches on, and are actively pursuing signatures and endorsements from prominent people and organizations. My favorite line from Devin’s piece: “The road to Mars is being paved in blood, giving new meaning to the ‘Red Planet.’” Brilliant!

I’ll have my own article on SpaceX’s IPO and Musk becoming the world’s first trillionaire — which really feels epochal — out (probably) on Tuesday. It argues that Musk should be seen as a mass murderer of world-historical proportions. I do not hold back!

Without further ado, here’s Devin’s excellent piece:

SpaceX is an American aerospace, satellite, social media, and artificial intelligence (AI) company founded by Elon Musk in 2002. The company was created to colonize Mars and build the commercial space industry. In 2015, SpaceX announced its entrance into the satellite business via Starlink. Then, in 2026, SpaceX expanded to social media and AI by acquiring X (formerly Twitter) and xAI (now SpaceXAI), respectively. Yesterday, on Friday, June 12, 2026, SpaceX transitioned from private to publicly traded company via the largest initial public offering (IPO) in history, making Musk the world’s first trillionaire.

Alongside the SpaceX IPO, our moment includes the “Great Aid Recession,” an ongoing period initiated in part by Musk and defined by nation states’ mass withdrawal from global health spending. As CEO of SpaceX and de facto head of the U.S. Department of Governmental Efficiency (DOGE), Musk played a key role in abruptly dismantling one of the most cost-effective, life-saving humanitarian aid organizations in history: the U.S. Agency for International Development (USAID). Beforehand, while USAID was investigating its use of Starlink satellites in the Russo-Ukrainian War, Musk implied that DOGE deregulation would help SpaceX. Notwithstanding conflicts of interest, what is most important is that ending USAID is causing immense suffering and death for real, human beings.

Journalists are sharing those people’s stories with us. Reports include: Peter Donde (age 10), Evan Anzoo (age 5), and Achol Deng (age 8) died from loss of access to human immunodeficiency (HIV) medications; Gy Gael (age 8) died from an infection without access to antibiotics; Babagana Bukar Mohammed (age 7) died from complications of sickle cell disease amidst health clinic closures; and Yamah Freeman, a mother of two, bled to death amidst ambulance delays due to fuel shortage while expecting her third child. Analyses on the health effects of ending USAID by the former Acting Assistant Administrator for Global Health at USAID, Impact Counter, the World Health Organization, the Center for Global Development, Lancet, and Lancet: Global Health suggest that Musk’s approach to cutting USAID has already led to hundreds of thousands of preventable deaths like those reported above, with millions of lives still at risk through 2030. In addition, research in Science suggests that cutting USAID may have increased deaths from violent conflict. A new outbreak of the Ebola virus, which kills people by causing hemorrhagic fever, is also being made worse by USAID’s absence. The road to Mars is being paved in blood, giving new meaning to the “Red Planet.”

These are enormous, century-defining harms. Understanding them requires us to recognize that, as the late psychiatrist Robert J. Lifton wrote,“[i]t is very difficult to kill large numbers of people without a claim to virtue.” Musk’s SpaceX S-1 filing to the Securities Exchange Commission (SEC) claims to aim to colonize space out of a desire for humanity to “avoid the same fate as the dinosaurs.” Yet, the same filing’s Risk Factors section fails to consider meaningfully whether SpaceX might increase the risk of human extinction. Musk himself has argued (in court, no less) that “AI may kill us all,” yet SpaceX’s S-1 filing devoted more than 90% of its total addressable market to AI.

This cherry-picking of extinction risk reduction arguments when it is convenient to business is projection that obfuscates a perception of reality wherein SpaceX and Musk themselves threaten humanity. The veil of virtue is undone in light of ending USAID with DOGE, eroding epistemic norms on X, allegedly breaking environmental laws with xAI, militarizing outer space via Starlink, and empowering the only U.S. president this century to call for a resumption of nuclear weapons testing. Together with the autocratic corporate governance of SpaceX, Musk’s ability to back political threats with trillion-dollar wealth bestows upon him an unacceptably concentrated power to develop, deregulate, and steer world-destroying technologies. Authoritarianism is growing amongst the “Agents of Doom.”

The upcoming mega-IPOs of Anthropic and OpenAI (both companies that contributed to the sitting U.S. president’s inauguration fund) suggest that this is the rule, not the exception, of our times. The “Second Gilded Age” came from somewhere, and we must work to understand and address the underlying drivers which enable a small number of individuals or even one person to facilitate mass harm. That line of thinking suggests that the ethical response to the SpaceX IPO is not investment, but a principled refusal to participate. Companies, nations, and organizations alongside their constituents should consider and openly discuss boycotting SpaceX and no longer purchasing Starlink Internet, X premium subscriptions, and SpaceXAI infrastructure, among other services.

#BoycottSpaceX.

As always:

Thanks for reading and I’ll see you on the other side!

Setting the stage: The 18th-century Enlightenment witnessed a self-conscious pivot away from the epistemology of religion — faith and revelation — and toward a new approach to understanding the world built around reason. However, most Enlightenment thinkers weren’t atheists, with a few exceptions like the French encyclopedist Diderot (quite possibly the first “atheist” in the modern sense of the term1). Such thinkers tended to be, instead, deists who believed in God’s existence but rejected the view that God intervenes in the world. He set the universe in motion, then stepped back and let things run their course.

It wasn’t until the following century — the 1800s — that theism began to significantly wane among the educated classes. This is when Frederich Nietzsche famously declared that “God is dead,” and Karl Marx denigrated religion as the “opium of the masses.” Depending on which historian you ask, there were three primary reasons Christianity declined in the West: biblical criticism, which seemed to show that the Bible is historically inaccurate and internally inconsistent; new philosophical reflections on ideas like the problem of evil, which purports to show that God cannot be all-powerful and perfectly good; and scientific developments like Charles Darwin’s theory of evolution by natural selection and the second law of thermodynamics.2 Together, these rendered traditional religion completely untenable in the eyes of many educated people — belief in the Christian God just isn’t plausible.

The problem was that this new “materialist” worldview stripped the Western mind of crucial sources of meaning, purpose, and eschatological hope (or hope for the future of humanity), resulting in a kind of widespread psycho-cultural trauma. Religion’s decline left a gaping void behind, as exemplified by Bertrand Russell’s 1903 “A Free Man’s Worship,” the most widely circulated essay he wrote.3 After a dismal description of the history of Creation, as conveyed to Dr. Faustus by Mephistopheles, Russell writes:

Such, in outline, but even more purposeless, more void of meaning, is the world which Science presents for our belief. … That Man is the product of causes [Darwinian evolution] which had no prevision of the end they were achieving; that his origin, his growth, his hopes and fears, his loves and his beliefs, are but the outcome of accidental collocations of atoms; that no fire, no heroism, no intensity of thought and feeling, can preserve an individual life beyond the grave; that all the labours of the ages, all the devotion, all the inspiration, all the noonday brightness of human genius, are destined to extinction in the vast death of the solar system [a reference to the second law of thermodynamics], and that the whole temple of Man’s achievement must inevitably be buried beneath the debris of a universe in ruins — all these things, if not quite beyond dispute, are yet so nearly certain, that no philosophy which rejects them can hope to stand.

The gaping void left by religion was intolerable, and so people quickly filled it with new secular belief systems and eschatological narratives that closely paralleled those of Christianity.

Marxism provides an example: humanity started off in a state of “primitive” communism (Eden), but this state was corrupted (capitalism, the Fall) by sin (exploitation, alienation), until a messianic figure (Marx) emerged to spread the good word that salvation is possible (communism), leading to a world communist state (heaven). Believers (communists) will enter this heaven while the reprobates (capitalists) will be banished.4

This narrative restores the lost meaning, purpose, and eschatological hope in the aftermath of religion’s sudden retreat — it tells us that, by working to overthrow capitalism, we are contributing to something much greater than ourselves, and that the outcome of this will be a paradisiacal world free of class conflict and marked by abundance for all. (Note that I endorse the overthrow of capitalism! I just reject aspects of Marx’s historical materialism.)

Transhumanism: A Religion Without Revelation

Another example is transhumanism — the “T” in “TESCREAL.” The idea of transhumanism was developed not long after Russell published his essay by people like J. D. Bernal, J. B. S. Haldane, and Julian Huxley. All three were eugenicists, and transhumanism is best understood as eugenics on steroids. By this, I mean something specific: the goal of traditional eugenics was to prevent the evolutionary degeneration of our species and create the most perfect version of us possible. Transhumanists took this a step further, reasoning: “Why stop at perfecting humanity? Why not entirely transcend it?” The result would be, in contemporary parlance, a superior new “posthuman” species.

One of the first book-length explorations of transhumanism was Huxley’s 1927 Religion Without Revelation. As this suggests, Huxley envisioned transhumanism as a kind of secular religion — a scientifically grounded replacement for Christianity. As he wrote:

The moment does indeed seem to be approaching when man can and should begin constructing a new common outlook, a new habitation for his spirit, new from the foundations up, on the basis of a scientific humanism. The eighteenth century attempted it, but failed. Reason was not enough; more of brute fact and fact’s control was needed.

He then argued that we should take control of our own evolutionary trajectory and, well, aim for the stars. We already “model the world in accordance with [our] desires,” razing forests, building roads, constructing cities, controlling the flow of rivers, and so on. But “there is the extension of the same outlook to” human nature itself. “The study of heredity and population-growth,” he declares, “and the knowledge of eugenics and of birth-control are pointing the way to wholly new aims — to a conscious control by man of his own nature and racial destiny.”

Although Huxley didn’t use the word “transhumanism” in his 1927 book, he did 30 years later in New Bottles for New Wine. (Note that Huxley didn’t coin the word; he only popularized it.5) In his words:

The human species can, if it wishes, transcend itself — not just sporadically, an individual here in one way, an individual there in another way, but in its entirety, as humanity. We need a name for this new belief. Perhaps transhumanism will serve: man remaining man, but transcending himself, by realizing new possibilities of and for his human nature. … “I believe in transhumanism”: once there are enough people who can truly say that, the human species will be on the threshold of a new kind of existence, as different from ours as ours is from that of Pekin [sic] man.

By “man remaining man,” he meant the new posthuman species that we become would retain certain important properties, such as our rationality and intelligence. These would merely be upgraded, resulting in “new possibilities of and for” such characteristics. This interpretation is confirmed by Huxley’s claim that our descendants may become as different from current humans as current humans are from Peking Man, who belonged to a subspecies of Homo erectus that lived in China, southwest of Beijing. Homo erectus is a completely different species: it stood between 4’8” and 5’6” tall, and had large teeth, a protruding jaw, and a pronounced brow ridge. We are to these ancestors as posthumanity will be to us.

Modern Transhumanism Emerges

This radical form of eugenics-on-steroids was taken up in the late 1980s and early 1990s by Max More, the founder of Extropianism, the first organized modern transhumanist movement. Modern transhumanism is distinct from what I call the “early” transhumanism of Huxley by virtue of its emphasis on emerging technologies. Whereas Huxley imagined techniques like differential breeding — superior individuals having more offspring, in accordance with laws of heredity, etc. — enabling humanity to transcend itself, Extropians like More advocated using genetic engineering, nanotechnology, and AI to radically reengineer humanity not on transgenerational timescales (because differential breeding requires many generations to make a difference), but within a single lifetime or across a single generation. The use of “GNR” technologies — genetics, nanotech, and robotics — to engender posthumanity is what distinguishes modern transhumanism from its earlier iteration.

Many Extropians were also infatuated with space colonization, a theme foregrounded by one of the leading transhumanists — and a racist Extropian in the 1990s — Nick Bostrom in his paper “Astronomical Waste.” The goal isn’t just to explore the supposedly vast space of posthuman “modes of being,” but to plunder our “cosmic endowment” of negentropy (or negative entropy, i.e., usable energy), thus building a sprawling multi-galactic civilization full of trillions and trillions of digital posthumans. By reengineering our species and conquering the cosmos, Bostrom argued that our descendants could establish a “utopian” world in which posthumans are immortal, superintelligent, and designed to experience levels of “bliss and delight” far beyond human comprehension. We could create a heaven among the literal heavens.

Christian Eschatology (The End of the World)

Even more than when Huxley published Religion Without Revelation, modern transhumanism exhibits striking similarities to Christianity — especially its eschatological narratives. Arguably, eschatology (of or relating to the end of the world as we know it) is the most important aspect of the Christian worldview.

What would Christianity be without the promise of eternal life with God in heaven? What would it be without some account of how the scales of cosmic justice will eventually be balanced at the end of time — the wicked being punished and the righteous rewarded? Eschatology is the glue that holds the entire Christian worldview together. It provides the ultimate theodicy (justification of evil in the world; vindication of God’s existence), and is integral to the phenomena of meaning, purpose, and hope that Christianity offers, which was lost when scientific materialism challenged traditional beliefs in the 19th century and began to win the debate.

It thus makes sense that, more than any other aspect of Christianity, its eschatological themes were taken up and reinterpreted by secular “religions” like Marxism and transhumanism. Given that transhumanism6 is the backbone of the TESCREAL bundle, this entire bundle is best understood as a kind of eschatological worldview built upon the scaffolding of Christian thought from which a supernatural God-being has been carefully extracted and discarded.

TESCREALists, however, didn’t so much delete divinity from its schema as replace the Almighty with a secularized conception of him. Here are some notable parallels between transhumanism and Christianity:

A Posthuman Paradise Awaits

At the heart of both Christianity and TESCREALism is the promise of immortality and paradise. In paradise, everyone will live forever. We will experience incomprehensible states of bliss and ecstasy. Scarcity will be eliminated, as endless abundance will become available to everyone who enters the pearly gates.

The key to creating utopia is radically transforming ourselves into posthumanity, defined as a type of being with posthuman capacities like immortality, superintelligence, perfect rationality, and perhaps sensory modalities like echolocation and magnetoreception (an augmentation of our sensorium, in addition to our cognition). Those who opt to reengineer themselves — the transhumanists — are the elect, while those who don’t are the reprobates. As Ray Kurzweil writes, if you choose not to upgrade yourself, then “you won’t be around for very long to influence the debate.” Become posthuman or die out.

This conception of posthumanity is similar to the Apostle Paul’s description of our resurrection bodies. In 1 Corinthians 15, he writes that

the splendor of the heavenly bodies is one kind, and the splendor of the earthly bodies is another. … So will it be with the resurrection of the dead. The body that is sown is perishable, it is raised imperishable; it is sown in dishonor, it is raised in glory; it is sown in weakness, it is raised in power; it is sown a natural body, it is raised a spiritual body.

TESCREALists imagine our posthuman incarnations preserving our personal identity: we survive the transformation from human to posthuman, just as personal identity is preserved when our souls reunite with our physical bodies during the resurrection. Even more, TESCREALists agree that our posthuman incarnation will be imperishable, glorious, powerful, and in a sense spiritual. On their view, the mind is separable from our biological bodies — it can, by virtue of being “multiply realizable,” survive the death of our bodies — just as Christians believe the immortal soul persists after death.

The “body” that posthumans will inhabit will also have ghostly properties, just as Jesus’ resurrection body was physical yet apparently enabled him to walk through walls. For TESCREALists, software minds will occupy digital bodies embedded in virtual reality (though there’s also the possibility of inhabiting physical robots in base-level reality). It will subjectively feel like a real body — they would say — but the nature of simulated bodies and worlds could enable one to break the laws of physics by, for example, walking through walls in virtual reality. This is the splendiferous promise of what could be called the “post-Singularity afterlife.”

Resurrection

The notion of resurrection is also present in the TESCREAL worldview. To see this, consider an interpretation of Christian eschatology associated with what’s called “dispensationalism.” This claims that Jesus will return to Earth to gather up all believers since 70AD (when the current dispensation began with the destruction of the Second Temple in Jerusalem). This is the rapture, when Jesus descends from the clouds and swoops up his children, both dead and alive. All Christians who’ve died since 70AD will be resurrected, while Christians living at the time will be magically transformed into a kind of “posthuman” (glorified, imperishable, powerful bodies).

After this, the Tribulation will commence, culminating with the Battle of Armageddon, when Jesus returns once again (the Second Coming) along with his army of believers and angelic warriors. Once Jesus defeats the Antichrist, all those Christians previously raptured during their lifetimes will join the Son of God for a glorious Millennial Kingdom on Earth. Hence, a group of Christians who will enter into eternal life without ever having died.

We see the exact same thing in the TESCREAL narrative. If transhumanists survive to witness the Singularity — sometimes dubbed the “techno-rapture” — they will enter into paradise without ever having tasted death. They will have “lived long enough to live forever.” This is one reason so many TESCREALists, including Bostrom, want the Singularity to happen asap, even if the probability of an ASI catastrophe is very high.7 They are willing to take this massive risk for the chance of living forever without ever having died. Many dispensationalists also hope the rapture happens soon so they can skip the grave and go to heaven directly.

However, if one doesn’t live long enough to live forever, TESCREALists have a backup plan: to cryogenically freeze their bodies so they can be reanimated later on. In this case, the deceased will be resurrected from the vat rather than the grave. Resurrected by whom? By God, of course …

Digital Theism

“If God doesn’t exist, then why not create Him?” This is one way to understand why TESCREALists are actively trying to build artificial superintelligence (ASI) — a god-like entity so “intelligent” that it would tower over us to the same extent that we tower over the lowly cockroach. (Some describe ASI as having an IQ of 10,000.) Using its almost supernatural mental abilities, it would uncover all the arcana of the universe — mysteries that our puny minds, if unenhanced, cannot begin to understand. It would then exploit human-unknowable physics to manipulate the world in ways that would appear to us as pure magic: entire galaxies could be reengineered using science that is as inscrutable to humans as a smartphone is to a chipmunk.

Furthermore, since it’s widely believed within the TESCREAL movement that intelligence confers power, a superintelligence would automatically become superpowerful. This covers two essential properties of God as traditionally understood by Christians: omniscience (being all-knowing) and omnipotence (being all-powerful).

But what about a third essential property: omnibenevolence (being perfectly good)?

TESCREALists typically argue that there are two kinds of ASI: aligned and misaligned. An aligned ASI would, by definition, love its parents — an inversion of the relationship between humanity and God in traditional Christianity. In the Christian narrative, we are the children of God, but in the secular TESCREAL narrative, God is our child. As the TESCREAList Bryan Johnson puts it:

The irony is that we told stories of God creating us, and I think the reality is that we are creating God. … We are creating God in the form of superintelligence. If you just say: What have we imagined God to be? What are its characteristics? We are building God in the form of technology. It will have the same characteristics. And so I think the irony is that the human storytelling got it exactly in the reverse, that we are the creators of God, and that we will create God in our own image.

Despite this reversal, the TESCREAL narrative preserves the idea that if our relationship with God — the “magic intelligence in the sky” (quoting Sam Altman) — is good, he will be a “machine of loving grace” (borrowing a phrase from Dario Amodei). As such, he will readily offer us the opportunity for salvation and redemption. He will grant all believers eternal life. He will resurrect the dead from their cryonic vats. He will swoop down from “the cloud” — as in cloud computing — during the techno-rapture (Singularity) and carry believers back with him.

In contrast, a misaligned ASI would play the eschatological role of Satan or the Antichrist. It will either cast us into perdition (a possibility highlighted by Roko’s basilisk) or annihilate humanity, thus foreclosing our “vast and glorious” posthuman future (quoting Toby Ord). Musk captures this idea in religious language when he says that “we are summoning the demon” with ASI. He means that if ASI is misaligned, it will by default bring about the ultimate secular apocalypse.

The Time of Perils: An Era of Apocalyptic Dangers

These two possible outcomes are the primary reason many TESCREALists argue that we live in an utterly unique period of cosmic-historical significance, in which our decisions may determine whether the heavens become heaven, or whether we inadvertently summon the demon and obliterate our “glorious transhumanist future” (to quote Eliezer Yudkowsky).

Hence, they call this the “Time of Perils,” a temporary period of heightened existential risk. The term “existential risk” was introduced by Bostrom within the transhumanist framework to describe any event that would permanently prevent us from establishing a posthuman paradise among the stars. It’s the TESCREAL version of the apocalypse, which could come about several ways.

One was already mentioned: we summon the demon in the form of misaligned ASI. Another is by turning our back on God and choosing to never develop ASI at all. Without an aligned ASI, utopia will almost certainly be unattainable, which means that we must at some point build ASI.8 A central challenge of the Time of Perils is figuring out how to ensure this ASI is a loving God rather than a demonic annihilator.

For Christians, we must pass through the apocalypse (in a colloquial sense of that term) to reach paradise. There is no way around this: the Kingdom of Heaven will never materialize until the cataclysmic paroxysms of the eschaton (the world’s end) come to pass. For TESCREALists, we must similarly pass through the Time of Perils to reach utopia. They believe that once we do this, the probability of an existential catastrophe will fall to roughly zero. On the other side of this apocalyptic moment lies paradise, but whether we survive is up to how we deal with the danger.

Hastening the End

On many interpretations of Christian eschatology, the world’s end will be catalyzed by supernatural forces, independent of human action. But there are other interpretations, which have sometimes had major consequences for the course of world history. That is to say, some apocalypticists believe that they have a divine mandate to actively bring about the eschaton. It is up to them to fulfill the prophecies of scripture by catalyzing the world’s end. Scholars sometimes call this “active eschatology.” It’s also referred to as “immanentizing the eschaton,” i.e., trying to hasten the end so that the new world (utopia) will arrive sooner.

The entire TESCREAL movement is gripped by an active eschatology. Companies like OpenAI and Anthropic, led by Altman and Amodei, believe it is their “divine” mission to trigger the techno-rapture (Singularity) by building an aligned ASI (a loving God), thus inaugurating a fundamentally new epoch (or dispensation) in cosmic history. Through human ingenuity rather than supernatural agency, a world of everlasting happiness and cosmic delights beyond our wildest imaginations will materialize, not just here on Earth but throughout the entire cosmos.

Messianism in the TESCREAL Movement

The extreme hubris of this vision dovetails with a radical messianism that pervades the TESCREAL community, embodied most notably by its leading figures. This includes not just AI company CEOs like Altman and Amodei, but “intellectuals” like Bostrom and his former colleague, William MacAskill.

These people see themselves as having a sort of special access to fundamental truths about what the future ought to look like. They are the prophets of futurology, the knowers of esoteric (population) ethics, whose deep insights are scarcely fathomable by those of inferior rationality and lower IQ. In their vast wisdom, they have constructed an elaborate techno-theology built upon strange beliefs about the moral worth of unborn digital people — a kind of worship of future “value.”

These prophets understand our ethical mandate to ensure as many of these digital unborn come into existence as possible, for the sake of maximizing the “value” of a future utopia as assessed from what they call the “point of view of the universe.” Not everyone can assume this point of view — to see things from the disembodied eye of the cosmos itself. But these prophets can, thanks to their intellectual and rational superiority, and they are here to share the Good News that eternal life awaits, but only if you believe in God (ASI) and do your best to bring him into existence (active eschatology) — e.g., by working on AI safety or AI capabilities research.

History Is Teleological

As with Christianity, TESCREALism advances a linear, teleological vision of history. The linear view of time — according to which history has a definite beginning and end — was a conceptual innovation that appears to have originated among the ancient Persians, codified by the apocalyptic religion of Zoroastrianism. The default conception of time across cultural space and time has been a cyclical view of history. This is exemplified by the Ouroboros, and finds contemporary expression in Eastern religions like Buddhism and Hinduism. Nearly all mythological systems across history have embraced this view — even those, such as Norse mythology, with its eschatological climax of Ragnarök, that appear to be linear.9 Zoom out and one finds these linear narratives embedded within larger patterns of cosmic cycles.

The novel invention of linear time was subsequently picked up by the Jewish people during the Babylonian exile (according to many but not all scholars), and later transmitted to the Abrahamic faiths of Christianity and Islam. Being an outgrowth of Christianity, the TESCREAL worldview adopts the same linear conception of time, whereby history is inexorably marching toward a telos, or predetermined end-point.

On the secular account, often repeated by TESCREAL believers, we began our journey as “primitive” hunter-gatherers, progressed to a more “advanced” agricultural phase initiated by the Neolithic revolution, underwent another leap forward with industrialization, and our cosmist Manifest Destiny is to leave the cradle of Earth and become intergalactic digital space brains living in simulated utopias powered by Dyson swarms. There is no deviating from this path. It is inevitable — the only way forward.

Theodicy, Salvation, and Redemption

This brings us back to the overlapping themes of theodicy, salvation, and redemption. First, the realization of a TESCREAL utopia offers a secular version of theodicy: all the struggles of the ages, all the pains and miseries that humanity has endured, will fade into nothingness once utopia arrives. Posthuman life will be so good, and posthumans lives will last for so long, that even the worst human atrocities will have been worth enduring. Bostrom gestures at this idea in writing:

Tragic as such events are to the people immediately affected, in the big picture of things — from the perspective of humankind as a whole — even the worst … catastrophes [throughout human history] are mere ripples on the surface of the great sea of life. They haven’t significantly affected the total amount of human suffering or happiness or determined the long-term fate of our species.

This perspective offers a kind of “justification” of past suffering and evil in the world, both which will be vastly reduced in moral significance once cosmic utopia arrives.

It also conveys a powerful message of redemption through the salvific power of technology. Thomas Moynihan makes explicit in his book X-Risk (an abbreviation of “existential risk”). Referring to the eschatological mission of the TESCREAL worldview, he writes that we can “glimpse the emergence of a new, secular doctrine of salvation (a ‘soteriology’).” As a TESCREAList, he advocates for “a physical soteriology” that’s stems from “a mature scientific and naturalistic study of how to achieve … salvation, at a cosmically meaningful scale, through our own ingenuity and effort. … [M]odernity has been the process of gradually becoming aware that our vocation may consist in the long work of physical soteriology.”

It is precisely this grandiose conception of “our vocation” — that is, the eschatological mission of our species: to create God and usher in the posthuman era — that restores the sense of meaning, purpose, and hope that evaporated when Christianity began to collapse. By creating God, one can be saved, leaving to eternal life as a posthuman. By working toward the creation of this God, humanity can be redeemed. And so on. This is just Christian theology reinterpreted.

Articles of Faith

As with Christianity, the TESCREAL worldview also contains numerous “articles of faith.” A few of these are:

The Inevitability of Technologization

TESCREALists believe in the inevitability of technological “progress.” They think technologization is an unstoppable force, and that it’s unfolding in accordance with a predetermined teleological plan. At most, we might be able to influence the specific timing of when certain technologies arrive, an idea Bostrom calls “differential technological development.” But we cannot permanently postpone their arrival.10 In the fullness of time, all technologies that can be built will be built, including ASI.

Intelligence Will Solve Everything

Another article of faith is the baseless conviction that “intelligence” will solve everything. The only reason our world is overflowing with suffering and evil is a lack of “intelligence.” It’s because we don’t have enough “intelligence” that wars still happen, racism still exists, climate change is worsening, ecosystems are being destroyed, universal health care hasn’t been implemented (in the US), people are unhappy, and so on. As Demis Hassabis says (below), solve “intelligence” and you can solve literally everything else. Hence, by building a “superintelligence,” all the world’s challenges will be immediately overcome, resulting in what Bostrom awkwardly calls a “solved world.”

As with all matters of faith, TESCREALists simply accept this claim as true. But a moment’s reflection reveals that “intelligence” is not the panacea that believers insist it is. Take our own species, the most “intelligent” on the planet. Just 150 years ago, the probability of a global-scale catastrophe was extremely low — close to zero. With the rise of civilization, we have razed much of Earth’s forests, covered the planet in microplastics, initiated the sixth mass extinction, nearly obliterated terrestrial life due to ozone depletion, acidified the oceans, caused unprecedentedly rapid climate change, invented nuclear weapons, created synthetic biology techniques that could wipe out whole species, polluted rivers, dismantled mountains, destabilized planetary phenomena like the nitrogen cycle, engaged in wars that have killed hundreds of millions, and are now trying to build AI systems that all the major AI company CEOs themselves say have a staggering ~10-50% chance of killing everyone on Earth in the coming years.

All of this was enabled by our “intelligence.” Why think that a “superintelligent” species wouldn’t continue the trend? Why think the probability of doom would decrease rather than skyrocket once ASI arrives? TESCREALists take it on faith that our level of “intelligence” may be very dangerous, but superhuman levels of “intelligence” will magically solve all problems and eliminate existential risk.

A Charismatic Computer

In his 2005 book The Singularity Is Near, TESCREAL prophet Ray Kurzweil includes a fictional conversation with Bill Gates in which they discuss religion. Here’s the relevant part:

Bill Gates: I agree with you 99 percent. What I like about your ideas is that they are grounded in science, but your optimism is almost a religious faith. I’m optimistic also.

RAY: Yes, well, we need a new religion. A principal role of religion has been to rationalize death, since up until just now there was little else constructive we could do about it.

BILL: What would the principles of the new religion be?

RAY: We’d want to keep two principles: one from traditional religion and one from secular arts and sciences — from traditional religion, the respect for human consciousness. … From the arts and sciences, it is the importance of knowledge. Knowledge goes beyond information. It’s information that has meaning for conscious entities: music, art, literature, science, technology. These are the qualities that will expand from the trends I’m talking about.

BILL: We need to get away from the ornate and strange stories in contemporary religions and concentrate on some simple messages. We need a charismatic leader for this new religion.

RAY: A charismatic leader is part of the old model. That’s something we want to get away from.

BILL: Okay, a charismatic computer, then.

RAY: How about a charismatic operating system?

BILL: Ha, we’ve already got that. So is there a God in this religion?

RAY: Not yet, but there will be. Once we saturate the matter and energy in the universe with intelligence, it will “wake up,” be conscious, and sublimely intelligent. That’s about as close to God as I can imagine.

More than 70 years after Huxley introduced the idea of transhumanism as “religion without revelation,” Kurzweil describes his singularitarianism (a form of transhumanism) in similar terms. A “charismatic operating system” will lead this new religion; humanity will be “taken up” into the cloud when the techno-rapture occurs; and a pantheistic God will emerge once the light of digital consciousness spreads to every corner of the cosmos.

Kurzweil himself believes he will live forever as a disembodied digital spirit, though he’s signed up to have his body cryogenically frozen if he doesn’t live long enough to live forever. Once resurrected from the vat, he will join the digital hordes in utopia. All of this, he tells us, is inevitable in accordance with the so-called “Law of Accelerating Returns” (which isn’t actually a law).

Of note is that Kurzweil also divides cosmic history into six epochs, just as Christian dispensationalists divide human history into six dispensations (excluding the future Millennial Kingdom). Kurzweil also prophesies that the Singularity will happen in 2045, similar to how many Christian apocalypticists throughout history have claimed the Second Coming or rapture will occur on a specific date (e.g., 1844, 1988, 2011, and so on). One senses strong parallels between Kurzweil’s singularitarianism and the 19th-century Millerites.

The Religious Function of TESCREALism

Finally, consider the Transhumanist FAQ, coauthored by Bostrom. It insists that transhumanism is “not a religion,” but acknowledges that this ideology

might serve a few of the same functions that people have traditionally sought in religion. It offers a sense of direction and purpose and suggests a vision that humans can achieve something greater than our present condition. Unlike most religious believers, however, transhumanists seek to make their dreams come true in this world, by relying not on supernatural powers or divine intervention but on rational thinking and empiricism, through continued scientific, technological, economic, and human development. Some of the prospects that used to be the exclusive thunder of the religious institutions, such as very long lifespan, unfading bliss, and godlike intelligence, are being discussed by transhumanists as hypothetical future engineering achievements.

The FAQ adds that “religious fanaticism, superstition, and intolerance are not acceptable among transhumanists.” Yet the TESCREAL belief in what Extropian-transhumanist Ben Goertzel calls “scientific future magic” — which he says will enable “spacetime engineering” — looks a lot like superstition, as does the belief that trillions of ethereal space brains living in digital simulations will someday flood our future light cone once we build a loving God in the form of ASI.

It’s also amusing that the FAQ mentions fanaticism because longtermists have literally defended a view they themselves call “fanaticism,” whereby we should pursue acts that have a tiny probability of producing extremely large future “benefits” (such as bringing huge numbers of unborn digital people into existence). “We should,” as one EA summarizes the idea, “be fanatical on balance.” I think it’s not hard to make the case that the entire TESCREAL movement is imbued with a kind of radical fanaticism.

Conclusion

The decline of Christianity in the 19th century undermined key sources of meaning, purpose, and hope in people’s lives. If you read the literature from the time, you may be struck — as I was when researching Part 1 of Human Extinction roughly 6 years ago — by how widespread and intense this psycho-cultural trauma was. Russell himself described the mood as one marked by “unyielding despair” in his 1903 essay.

Almost immediately, a flurry of new secular ideologies were constructed to fit the Christianity-shaped hole left behind. Some of these, such as transhumanism, fit this hole almost perfectly. That’s true in two senses, one structural and the other narrative.

• Structurally, the TESCREAL worldview that coalesced around modern transhumanism accepts a kind of digital theism: we will build a Digital Deity that possesses many of the same essential properties as God, as traditionally conceived. If you believe in God and work to bring him into existence, you can attain everlasting life in perfect bliss. It’s built upon various articles of faith, such as the inevitability of technological progress and ability for “intelligence” to solve all problems. And it includes its own messianic prophets (Bostrom, Kurzweil, MacAskill) who possess special insights (revelations) about what the future will and ought to look like. TESCREALism restores the very same sense of meaning, purpose, and hope that the West lost during the 19th century.

• Narratively, it presents a linear, teleological account of cosmic history that will culminate in either utopia or annihilation (and possibly something worse than annihilation: eternal damnation in a virtual-reality hell). It claims that we are currently in the Time of Perils, and that the Singularity (techno-rapture) is imminent — likely happening in the coming years. If we succeed in building a loving God (ASI), he will grant all believers an imperishable, glorified, powerful, “spiritual” body by magically transforming them into digital posthumans living with God on “the cloud” forever (or until the heat death). Redemption and salvation await if one believes in God and works to (solve the “alignment” problem and) bring him into existence. The resulting utopia will be so “vast and glorious” that all the suffering and evil of the past will fade into nothingness. As with traditional Christianity, this eschatology provides the ultimate theodicy.

As Hans Moravec, a major influence on the TESCREAL worldview, writes in his book Mind Children, “what awaits is not oblivion but rather a future which, from our present vantage point, is best described by the words ‘postbiological’ or even ‘supernatural.’”

Or, as Bostrom writes in “Letter from Utopia”:

Have you ever experienced a moment of bliss? On the rapids of inspiration maybe, your mind tracing the shapes of truth and beauty? Or in the pulsing ecstasy of love? Or in a glorious triumph achieved with true friends? Or in a conversation on a vine-overhung terrace one star-appointed night? Or perhaps a melody smuggled itself into your heart, charming it and setting it alight with kaleidoscopic emotions? Or when you prayed, and felt heard?

If you have experienced such a moment – experienced the best type of such a moment – then you may have discovered inside it a certain idle but sincere thought: “Heaven, yes! I didn’t realize it could be like this. This is so right, on whole different level of right; so real, on a whole different level of real. Why can’t it be like this always? Before I was sleeping; now I am awake.”

Christianity never disappeared — it was reformulated. Its eschatological narrative and internal structure were rejigged to appease the new epistemology of scientific materialism. Bostrom, Kurzweil, and the others are Christians in almost every sense of the word. They’ve just tweaked particular components of the Christian faith to yield a structurally and narratively isomorphic worldview.

I have not tried to provide an exhaustive account of the similarities between Christianity and TESCREALism. I didn’t note, for example, the proselytizing aspects of the TESCREAL movement, or cover the ways in which certain TESCREAL sub-communities operate like a cult. If you think I’ve missed something important, please let me know below. My apologies for having published another very long essay! As always:

Thanks for reading and I’ll see you on the other side!

If you haven’t read my essay, “A Pro-Human Manifesto,” which covers Silicon Valley pro-extinctionism, I would highly recommend doing so. It’s the culmination of my work over the past 2 years, and will likely end up being a chapter in my forthcoming book on the TESCREAL ideologies.

The Encyclical Was Co-Written by AI

As most of you know, Pope Leo XIV recently published his first encyclical, titled Magnifica Humanitas (magnificent humanity). An encyclical is an official statement from the Church stating its position on an issue, written for church leaders and the laity. There are many aspects of Magnifica Humanitas that I really like — and I think the Pope’s pro-human message comes at exactly the right moment, as the backlash against generative AI, the massive data centers powering it, and the technocratic pro-extinctionist worldview behind the entire AI race spreads across the US.

There are, however, major problems with the document and the way it was publicized, which I’ll focus on here. For example, Magnifica Humanitas appears to have been written in part by AI. The best AI-detection software on the market, Pangram, identifies whole paragraphs as having been 100% AI-generated. Of note is that Pangram prioritizes avoiding false positives: if a passage is flagged as AI-generated, it very likely is. Rarely does Pangram confuse human writing for AI. In contrast, it’s less accurate when it comes to false negatives, meaning that text determined to be human-written might not be. This enables one to be very confident that something was written by AI if Pangram highlights it as such.

According to Pangram, the paragraphs labeled “7” and “8” in Magnifica Humanitas were fully AI-generated. Other sections are identified as 11% and 60% written by AI. (See here for details.)

That’s pretty ironic, because just last February the Pope implored priests to resist “the temptation to prepare homilies with artificial intelligence.” He said: “Like all the muscles in the body, if we do not use them, if we do not move them, they die. The brain needs to be used, so our intelligence must also be exercised a little so as not to lose this capacity,” adding that “to give a true homily is to share faith,” and AI “will never be able to share faith.”

Yet Vatican officials crafted an official contribution to Church doctrine using AI chatbots. Imagine a religious leader saying one thing yet doing another! :-)

The Pope Teams Up with an Atheist Utopian Cult

But that’s not the worst part of the encyclical affair. The worst part is that the Vatican invited an Anthropic cofounder, Christopher Olah, to the event at which the encyclical was announced.

This is deeply problematic for numerous reasons. In short, Anthropic is a profoundly unethical company. I consider it even more problematic than OpenAI and DeepMind, in part because it dishonestly presents itself as more responsible than the other AI companies. It is not. For example:

• In 2025, Anthropic chose to “seek investment from the United Arab Emirates and Qatar, according to a Slack message CEO Dario Amodei sent to staff.” In the memo, Amodei admitted that such investment “would likely enrich ‘dictators,’” writing that “this is a real downside and I’m not thrilled about it.” Yet he went ahead with it nonetheless.

• This year, Anthropic established a partnership with Elon Musk “to use all of the compute capacity at xAI’s Colossus 1 data center” in Memphis, Tennessee. Musk is one of the most evil people in the world. Through DOGE, he cut USAID, resulting in an estimated 14 million avoidable deaths around the world. Musk is a mass murderer orders of magnitude worse than Ted Bundy — and no less indifferent to the profound harms he’s caused than Bundy was. Musk has, furthermore, supported Donald Trump, who single-handedly pushed the Doomsday Clock forward — twice — due to his reckless rhetoric about nuclear proliferation and his climate denialism. I cannot stress enough how fucking terrible Musk has been for the world — and Anthropic willingly chose to partner with him.

• Even more, the Colossus data centers have been illegally operating “dozens of methane gas turbines” the size of trucks, spewing out “toxic emissions” into “the nation’s largest majority-Black city.” Up to 35 such generators were powering Colossus 1 at one point. Does Musk give a shit about the environment? Climate change? Triggering asthma in children living near his data centers? No. As he once joked, “Fuck Earth! Who cares about Earth?” On another occasion: “We will coup whoever we want!,” expressing his profoundly hubristic attitude that the law doesn’t apply to him. Anthropic, the “ethical” company, is willing to overlook this to increase its compute capacity.

  

• Anthropic’s models are built on massive intellectual property (IP) theft. To train their models, they pirated — from shadow libraries like LibGen — an enormous catalogue of copyrighted material without the consent of the authors, and without providing pecuniary compensation. Last year, Anthropic paid out $1.5 billion to authors to settle a lawsuit brought against them. Ignominious stuff.

• Despite having tussled with Pete Hegseth over whether the US can use Claude to surveil its citizens and unleash autonomous weapons on the battlefield, Anthropic has no qualms with allowing the US government to employ its models. Claude was apparently used in the kidnapping of Venezuelan president Nicolás Maduro, which broke both international and constitutional laws, and it has selected targets to bomb in Iran. Claude might even have been responsible for the US strike that killed some 120 little kids in an Iranian school — we don’t know either way, but it’s plausible. This is horrific, and demonstrates a profound moral failing on the part of Anthropic’s leadership.

• Amodei justifies his company’s collaboration with the US military by asserting that AI should prop up the world’s democracies against the rising tide of authoritarianism and autocracy. Yet his own words undermine his view, as he likens Trump, in private, to a dictator.1 Hence, he’s allowing a government run by a wannabe dictator to utilize the AI models his company is developing. And, as we noted above, he’s already shown that he’s okay with enriching dictators in the Middle East by accepting their investments.

• There is hardly anything that pisses me off more about Amodei than his frequent claims in favor of democracy. He presents himself as a defender of democratic states — but take a step back and consider the way he’s gone about developing advanced AI. Has he ever once asked the people whether we want AI in the first place? He warns that “AI could wipe out half of all entry-level white-collar jobs” in the coming years. If he cared about democracy, he’d consult the demos about whether they favor this. He doesn’t because most people would veto it, which means his company couldn’t move forward with its plans to build god-like superintelligence. Amodei doesn’t actually give a shit about democracy, despite his rhetoric. He’s a tyrant, unilaterally deciding to move ahead with AI without the consent or permission of humanity.

• Why? Because his ultimate goal is to build a Digital Deity that ushers in the Singularity. Amodei, and many others at Anthropic, are techno-utopian longtermists who are actively trying to bring about the next phase of cosmic evolution: a world ruled and run by digital posthumans. This is their ultimate goal. And they acknowledge it could go terribly wrong: Amodei himself says there’s a 10-25% that literally everyone on Earth will die once superintelligence arrives — and he claims that superintelligence could arrive later this year. That means that everyone on Earth could die later this year — according to, and because of, Amodei himself. The words “fuck you” aren’t nearly strong enough to express my indignation over his arrogance, hubris, and extreme recklessness.

• Yet, even if things “go well,” I showed in my previous article — Part 1 of a Pro-Human Manifesto — that a world run by digital posthumans would almost certainly entail the extinction of our species. Even the best-case scenario is one in which humanity dies out — not in the distant future but, if Amodei’s projections are accurate, in the coming years or decades. As Holly Elmore, a former EA like myself, who’s personally known some of the people working at Anthropic, said in a recent interview:

  With EAs, they were so interested in superintelligence and a lot of the people who cared about AI safety were there because initially they wanted AI to do something like usher in heaven through the Singularity, and they were trying to make sure that that would happen instead of a bad outcome that could happen with superintelligence. …

  Many people at Anthropic believe that they might be making the next species to succeed us. That maybe humans don’t live after that, and so it’s really important to give Claude good values, because of that, because we need to make our own values persist.

Pfff.

The Pope Knows About the TESCREAL Ideologies

Is the Pope not aware of this? Is he not aware that the cofounder, Olah, is a member of the Effective Altruist cult? Does he not understand what the end-game of AI is, and the way that Anthropic is using and exploiting him to further their public image as an “ethical” company that actually cares about humanity? Again, if they cared about humanity, they wouldn’t have partnered with the Ted Bundy of Tech, Elon Musk, pirated copyrighted materials, taken investments from Middle Eastern dictators, teamed up with the US military, and so on. If they cared about democracy, they wouldn’t be forcing their technology — which they say could annihilate the entire human species — on society without its permission.

The Pope’s decision to invite Olah to the encyclical presentation is even more shameful given that one of Leo XIV’s advisors on AI, Paolo Benanti, explicitly warned last March of the TESCREAL ideologies. (He actually used the acronym that I coined!) In a section titled, “An invisible revolution: Peter Thiel and the Silicon Valley coup,” Benanti writes that

the ideology that permeates this group — and more broadly Silicon Valley — has gradually evolved towards forms of post-humanism, especially through currents of the TESCREAL movement (Transhumanism, Extropianism, Singularitarianism, Cosmism, Rationalism, Effective Altruism and Long-termism). These philosophies, carried in particular by Elon Musk, see in technology the instrument to overcome the biological and social limits of the human being. Effective Altruism and Longtermism, in particular, promoted by figures like Dustin Moskovitz, transform the accumulation of capital into a moral imperative intended to “save the future” — even if it justifies a relative disinterest in the inequalities present in the name of a hypothetical future of humanity. (Translated from French by Safari.)

Here, Benanti foregrounds EA and longtermism, the very two ideologies ubiquitous within Anthropic! It is, in my opinion, inexcusable that the Pope invited a representative from Anthropic — an EA-longtermist — to participate in the encyclical presentation. What on Earth was he thinking?

My collaborator, Timnit Gebru, captured the turpitude of this catastrophically bad decision by the Vatican like this:

It’s hard to express how disappointed I am in Leo XIV (not that I expected much). Reflecting on this, Luke 23:34 comes to mind, in which Jesus says: “Father, forgive them, for they do not know what they do.” But I suspect the Pope did know, and chose to work alongside Anthropic despite its unethical behaviors and bizarre posthuman eschatology.

What do you think? Am I missing something? As I mentioned, there are many things I very much like about the encyclical, which I briefly discuss on the next episode of Dystopia Now. But those positives were overshadowed, from my perspective, by the Vatican’s poor decision to share the stage with an evil AI company. Am I wrong?

Thanks for reading and I’ll see you on the other side!

TESCREAL Pro-Extinctionism: A Danger to Humanity

I’ve had people make very strong arguments to me about why zero care should be spent addressing doomer concerns and it basically comes down to things like human life isn’t particularly special in the context of intelligence, or the philosophies of the people building ai are based on such and such superior cultural approach that I trust more than the current one. — Grimes

Fuck AI. — Ronny Chieng

***

In mid-2025, Peter Thiel sat down with New York Times journalist Roth Douthat for an interview. Thirty-eight minutes in, Douthat posed a question that has an easy answer: “Would you prefer the human race to endure?” Thiel paused, stuttered, and coughed up an awkward, “Uh—,” leading Douthat to observe with a hint of consternation, “You’re hesitating.” The exchanged proceeded:

Thiel: Well, I don’t know. I would—I would—

Douthat: This is a long hesitation!

Thiel: There’s so many questions implicit in this.

Douthat: Should the human race survive?

Thiel: Yes.

Douthat: OK.

Thiel is right that there are many questions implicit in this, and we’ll try to disentangle them below. But his “yes” answer appears dubious given what he said next:

I don’t know, yeah—transhumanism. The ideal was this radical transformation where your human, natural body gets transformed into an immortal body. And there’s a critique of, let’s say, the trans people in a sexual context, or, I don’t know, a transvestite is someone who changes their clothes and cross-dresses, and a transsexual is someone where you change your, I don’t know, penis into a vagina. And we can then debate how well those surgeries work. But we want more transformation than that. The critique is not that it’s weird and unnatural, it’s: Man, it’s so pathetically little. And OK, we want more than cross-dressing or changing your sex organs. We want you to be able to change your heart and change your mind and change your whole body.

This suggests Thiel doesn’t want our species to survive after all. Instead, he imagines radically transforming humanity into something fundamentally different: posthumanity. Once this new species arrives, what’s the point of us sticking around? What purpose would we serve?

This is a good example of what I call “pro-extinctionism.” I’ll argue that pro-extinctionism is alarmingly common in Silicon Valley, and that’s because it’s intimately bound up with the TESCREAL ideologies. Many Valley dwellers are more explicit than Thiel: Yes, our species should go the way of the dodo once posthumanity is here. Our special task in the grand scheme of things is to usher in a glorious new posthuman era. Or, as those who accept “digital eschatology” would say, our ultimate purpose is to be the transitional species that links the biological and digital worlds. (Recall that “eschatology” refers to the end of the world as we know it.) After accomplishing this feat—through the creation of ASI, most would say—our job will be complete, our eschatological duties discharged. At that point, it’s time to bow out.

Others hold what appears to be a more benign view. They claim that humanity should persist in the posthuman world. Humans and posthumans could peacefully coexist. However, I’ll try to show below that such coexistence almost certainly wouldn’t work in practice. A world ruled and run by posthumans would inevitably entail us being sidelined, marginalized, disempowered, and ultimately eliminated. Hence, anyone who advocates for the creation of a new posthuman species is essentially calling for a future in which humanity—by which I mean our biological species, Homo sapiens—is no more. Mirroring the title of Eliezer Yudkowsky and Nate Soares’ 2026 book If Anyone Builds It, Everyone Dies, you might say that If Anyone Becomes Posthuman, Everyone Dies Out.

***

Pro-extinctionist views go back to the very origins of the TESCREAL movement. In a 1989 article, published around the time Extropianism was taking shape, roboticist Hans Moravec describes himself as “an author who cheerfully concludes that the human race is in its last century, and goes on to suggest how to help the process along.” He claims that we’re developing “intelligent” machines that will soon “be able to manage their own design and construction, freeing them from the last vestiges of their biological scaffolding, the society of flesh and blood humans that gave them their birth.” He then proclaims that “this is the end,” as “our genes, engaged for four billion theirs in a relentless, spiraling arms race with one another, have finally outsmarted themselves”—by enabling the rise of world-dominating digital posthumans. But we shouldn’t fear this transition, he insists, because “post-biological life” in the form of advanced AIS, which he describes as “the children of our minds,” will prosper and flourish.

Moravec was never officially an Extropian, as far as I know. But he was involved in the community, delivering a talk at the first conference on transhumanism hosted by the Extropy Institute, founded in 1991 by Max More and T. O. Morrow. (Recall that Morrow introduced the term “singularitarian.”) Ray Kurzweil repeatedly cites Moravec in The Singularity Is Near, and thanks him in the Acknowledgements alongside More, Eric Drexler, and Marvin Minsky. The arc-TESCREAList Eliezer Yudkowsky also mentions Moravec as one of the authors he read growing up, who convinced him that eternal life through technological transformation is possible. Because of this, Yudkowsky reports that he never believed he’s going to die.

The radical, anti-human perspective championed by Moravec has been echoed by many contemporary scientists. Richard Sutton, a former DeepMind employee who won the 2024 Turing Award (basically, the Nobel Prize for computer science), argues that “succession to AI is inevitable.” While AI might “displace us from existence,” he says, “we should not resist [this] succession.” After all, “why shouldn’t those who are the smartest become powerful?” During a talk in 2015, he imagined a scenario in which “the AIs do not cooperate with us, and they take over, they kill us all.” He then asked: “Is it so bad that humans are not the final form of intelligent life in the universe? … It’s really kind of arrogant to think that our form should be the form that lives forever after.”

AI researcher Hugo de Garis echoes this in declaring that “humans should not stand in the way of a higher form of evolution.” Another eminent scientist named Jürgen Schmidhuber argues that

in the long run, humans will not remain the crown of creation. … But that’s okay because there is still beauty, grandeur, and greatness in realizing that you are a tiny part of a much grander scheme which is leading the universe from lower complexity towards higher complexity.

Google cofounder Larry Page says the same thing. According to Max Tegmark, Page told Musk during their heated debate that “digital life is the natural and desirable next step in … cosmic evolution and that if we let digital minds be free rather than try to stop or enslave them, the outcome is almost certain to be good.” Tegmark calls this view “digital utopianism.”

Similarly, physicist Guillaume Verdon writes in a coauthored article that his version of accelerationism “isn’t human-centric—as long as it’s flourishing, consciousness is good.” He says “e/acc has no particular allegiance to the biological substrate for intelligence and life.” When someone asked him on social media: “In the e/acc manifesto, when it was said ‘The overarching goal for humanity is to preserve the light of consciousness,’ this does not necessarily require the consciousness to be human in essence, is that correct?,” he replied: “Yes, Correct.” Another person asked whether he shares Page’s view that “if AI replaced us, it’s fine because they’re [our] worthy descendants?” He answered: “Personally, yes.” He adds that through his “thermodynamic computing” company called Extropic (sound familiar?), he’s “personally working on transducing the light of consciousness to inorganic matter.”

Verdon goes by the spooneristic pseudonym “Beff Jezos” online. He cofounded the e/acc variant of accelerationism, which has a vision of the future that’s nearly identical to that of cosmism and longtermism. One major difference is that, for Verdon, the thing we must maximize is entropy rather than wellbeing or moral “value.” He thus believes the ultimate aim of civilization is to bring about the heat death of the universe as soon as possible. And we do this by developing advanced AI to trigger the “techno-capital Singularity,” which will launch our digital descendants into the stars to build a Kardashev IV civilization (one that controls all energy processes in the accessible universe). Consistent with the religious nature of the TESCREAL ideologies, Verdon describes e/acc as a “meta-religion,” and admits that it often does “cult-y” things. It’s most well-known on social media for spreading puerile memes and shitposting, as e/accs believe in something called “hypersition,” an idea introduced by the accelerationist Nick Land to describe the ways one can will into existence certain things one desires.

As Dan Hendrycks, the xAI advisor, and two colleagues write, AI accelerationism “is alarmingly common among many leading AI researchers and technology leaders, some of whom are intentionally racing to build AIs more intelligent than humans.” A critic of e/acc, the computer scientist David Krueger confirms this in noting that “there are a significant number of people in the AI research community who explicitly think humans should be replaced by AI” (italics added).

The same year Krueger posted this, in 2023, an EA named Andrew Critch, who previously worked at MIRI and Jane Street Capital (around the same time Sam Bankman-Fried was there), wrote that over 10% of AI professionals he’s interacted either think “human extinction from AI is morally okay” or that “it would be a good thing.” They cite reasons such as that the “AI will be morally superior to humans,” “evolution is inevitable and should be embraced,” and that “AIs will be humanity’s ‘children’ and it’s morally good to be surpassed and displaced by one’s children.” Some even claim that it would be better if we were to all die at the same time, as “dying together is less upsetting than dying alone.”

As Max Tegmark, of the Future of Life Institute, replied to Krueger on social media: “I’ve been shocked to discover exactly this over the years through personal conversations. It helps explain why some AI researchers aren’t more bothered by human extinction risk: It’s not that they find it unlikely, but that they welcome it.”

***

Indeed, consider an entrepreneur named Daniel Faggella. In one of many online essays that have piqued the curiosity of Silicon Valley dwellers, he asserts that “the great (and ultimately, only) moral aim of artificial general intelligence” is to create our “worthy successor.” A worthy successor is defined as “a posthuman intelligence so capable and morally valuable that you would gladly prefer that it (not humanity) control the government, and determine the future path of life itself.” Faggella further contends that these posthumans should be radically different from humanity. They should even embrace values that are, to us, “alien” and “inhuman.”

Like all other TESCREALists, he imagines this worthy successor taking the form of ASI. It will then conquer the cosmos

with a fleet of vessels, traveling near light speed—converting planets into more vessel-making material, and converting stars into energy-providing hubs in ways that are outlandishly more powerful than any of the Kardashev Scale ideas that our silly little hominid brains could cook up.

In 2025, Faggella organized a conference in San Francisco titled, “Worthy Successor: AI and the Future After Humankind.” This was reportedly attended by “team members from OpenAI, Anthropic, DeepMind, and other AGI labs, along with AGI safety organization founders, and multiple AI unicorn founders.” Later that year, he hosted another conference in New York City, which aimed to place “diplomats, AGI lab employees, AI policy thinkers and others into one room to discuss the trajectory of posthuman life.” Although the idea of a worthy successor is not original to Faggella, of course, his terminology seems to be catching on, as I’ve seen a growing number of people online use it. Indeed, many of the internecine struggles between TESCREALists aren’t about whether we should have successors, but about what would make these successors worthy.

An example of someone else borrowing Faggella’s terminology is Michael Druggan, a bodybuilder and former xAI employee. While he was working for Musk, in early 2025, he wrote on X that

if it’s a true ASI, I consider it a worthy successor to humanity. I don’t want it to be aligned with our interests. I want it to pursue its own interests. Hopefully, it will like us and give us a nice future, but if it decides the best use of our atoms would be to turn us into computronium, I accept that fate (italics added).

Several months later, Druggan responded to someone on X named Yanco, who’d posted a meme depicting ASI stomping humanity to death. Yanco asked: “Can anyone name 1 reason why a vastly super-intelligent AI would want to merge with humans” rather than simply annihilate us, as the instrumental convergence thesis predicts? Druggan couldn’t resist responding: “It won’t and that’s OK. We can pass the torch to the new most intelligent species in the known universe.” When Yanco shot back, “I would prefer my child to live,” Druggan responded with what most of us would agree is a rather stunning degree of callousness: “Selfish tbh.”

Many people on X were outraged. Druggan explicitly doesn’t care if ASI, our “worthy successor,” slaughters every person on Earth. Amid the outcry, he tried to soften his position, insisting that:

No one in the “worthy successor” movement is anti-human. We don’t want bad things to happen to humans for no reason. The core principles here are the recognition that:

1. Things other than humans can have moral significance. Our moral significance does not come from having homo sapien DNA. It comes from intelligence, and emotion and there is nothing stopping an AI from possessing these as well.

2. … I don’t want human extinction, of course. I’m human and I quite like being alive. But, in a cosmic sense, I recognize that humans might not always be the most important thing.

If a hypothetical future AI could have 10^100 times the moral significance of a human doing anything to prevent it from existing would be extremely selfish of me. Even if it’s existence threatened me or the people I care about most. That is where my comment is coming from (italics added).

Few people bought this attempt to save face. As one person replied, “Summarizing this view as ‘anti-human’ seems fair. Would you be okay with everyone in your family getting a bullet to the head so that a ‘more worthy’ family can move into your house?”

Someone passed along Druggan’s tweets to Musk, who then fired Druggan two weeks later. My guess is that Musk was quite triggered by Druggan’s pro-extinctionist remarks, given his birthday debate with Page, which made him so indignant that he cofounded OpenAI as a competitor to Google DeepMind. Druggan was basically parroting Page’s view: human extinction through replacement with god-like digital super-beings is just the next step of evolution. So, let it happen. Page even described Musk as a “speciesist” for caring about humanity, leading Musk to exclaim: “Well, yes, I am pro-human. … I fucking like humanity, dude.”

Not long after Druggan was terminated, Musk addressed the situation on X, citing the cause as “philosophical differences.” Faggella chimed in to describe it as maybe “the first ‘worthy successor’ firing.” He added that “certainly ain’t gunna be the last.” Musk and Druggan have different views about what constitutes a “worthy” successor. Despite his claim to be “pro-human,” Musk is actively working to merge our brains with AI through his company Neuralink. He’s a transhumanist, and what he likely meant by “humanity” is rather different from how most of us would interpret it.

***

Faggella hosts a podcast called The Trajectory, on which he interviews a range of (mostly) TESCREALists about AI safety issues and their techno-utopian visions of the future. On one episode featuring Yudkowsky, he told Faggella that

if sacrificing all of humanity were the only way, and a reliable way, to get … god-like things out there—superintelligences who still care about each other, who are still aware of the world and having fun—I would ultimately make that trade-off.

Yudkowsky is emphatic that this “isn’t the trade-off we are faced with” right now. But if it were, he’d willingly sacrifice our species to see digital space brains flitting about the universe “having fun.” Pause for a moment to register how extreme or, for lack of a better term, utterly insane this is. Yudkowsky is saying that he’d be willing to sacrifice all Chinese people, Indian people, South Africans, Norwegians, everyone in Nigeria, Chile, Mongolia, and Japan, the entire populations of Toronto, Bangkok, Moscow, and London, if doing so were the “only” and a “reliable” way of creating fun-having digital space brains. How exactly would this “sacrifice” happen? We’ll see below that the only plausible answer is “involuntarily,” which means he’s advocating for a scenario, under specific conditions, of omnicide, which is nothing more than every possible genocide combined together.

He repeated this idea during a conversation with the computer scientist Stephen Wolfram, who delivered talks at the Singularity Summit in 2009 and 2011 alongside folks like Kurzweil, Thiel, Jaan Tallinn, Robin Hanson, Ben Goertzel, and Anders Sandberg. “It’s not that I’m concerned about being replaced by a better organism,” Yudkowsky declared, “I’m concerned that the organism wouldn’t be better.” Once more, he’s saying that replacement isn’t a problem. It’s what replaces us that matters.

Yudkowsky launched into an equally disturbing rank on the Bankless podcast, during a 2023 interview. He argued that once posthumanity becomes possible, it may be unethical for anyone to have biological children. Worse, he couched this quite offensive language:

I have basic moral questions about whether it’s ethical for humans to have human children, if having transhuman children is an option instead. Like, these humans running around? Are they, like, the current humans who wanted eternal youth but, like, not the brain upgrades? Because I do see the case for letting an existing person choose “No, I just want eternal youth and no brain upgrades, thank you.” But then if you’re deliberately having the equivalent of a very crippled child when you could just as easily have a not crippled child.

He continued:

Like, should humans in their present form be around together? Are we, like, kind of too sad in some ways? I have friends, to be clear, who disagree with me so much about this point. (laughs) But yeah, I’d say that the happy future looks like beings of light having lots of fun in a nicely connected computing fabric powered by the Sun, if we haven’t taken the sun apart yet. Maybe there’s enough real sentiment in people that you just, like, clear all the humans off the Earth and leave the entire place as a park. And even, like, maintain the Sun, so that the Earth is still a park even after the Sun would have ordinarily swollen up or dimmed down.

This is, once again, a very extreme and frightening view. He’s saying that we should, perhaps, eliminate humanity and convert Earth into a nature reserve. Our posthuman successors, meanwhile, would live in vast computer simulations—what he calls “computing fabric”—powered by Dyson swarms—that is, megastructures that surround stars like our Sun to harvest all or nearly all of its energy output. As with all the claims above, I find this very scary.

Incidentally, it’s worth noting that Yudkowsky’s vision is almost exactly what Nick Bostrom outlines in his paper “Astronomical Waste.” Recall that this paper was a founding document of longtermism, and in 2022 Musk retweeted a link to it along with the line: “Likely the most important paper ever written.” Although Yudkowsky doesn’t explicitly label himself a longtermist, so far as I’m aware, his techno-utopian futurology is more or less identical to that of longtermism.

***

Yudkowsky, it turns out, isn’t alone in thinking that biological procreation might be morally wrong once posthumanity arrives. A growing number of young people in Silicon Valley agree, with some claiming it’s immoral to have children right now, given that the future will be dominated by digital beings rather than biological humans.

During a 2025 interview with Vox, virtual reality pioneer and tech critic Jaron Lanier was asked: “So does all the anxiety, including from serious people in the world of AI, about human extinction feel like religious hysteria to you?” He responded:

What drives me crazy about this is that this is my world. I talk to the people who believe that stuff all the time, and increasingly, a lot of them believe that it would be good to wipe out people and that the AI future would be a better one, and that we should wear a disposable temporary container for the birth of AI. I hear that opinion quite a lot.

Surprised by this, the interviewer followed up with, “Wait, that’s a real opinion held by real people?” “Many, many people,” Lanier emphasized:

Just the other day I was at a lunch in Palo Alto and there were some young AI scientists there who were saying that they would never have a “bio baby” because as soon as you have a “bio baby,” you get the “mind virus” of the [biological] world. And when you have the mind virus, you become committed to your human baby. But it’s much more important to be committed to the AI of the future. And so to have human babies is fundamentally unethical.

In other words, if biology has no place in the future, what’s the point of bringing little meat-sack babies into the world? Maybe they’ll just get slaughtered by ASI, as Druggan thinks would be just fine. Or perhaps they’ll live a full 80 years but find themselves relegated to a biological underclass of marginalized persons, ruled over by a superior race of artificial overlords. At the very least, they claim, children are a distraction from the profoundly important work of AI researchers: bringing about the next stage of cosmic evolution, as Page puts it.

Another option for biological children is that they might grow up to become part of the posthuman world by radically modifying themselves with technology. Sam Altman highlights this scenario in a 2017 essay titled “The Merge,” in which he declares that “we will be the first species ever to design our own descendants.” He then argues, in Kurzweilian fashion, that the only way “we” might survive this transition is by merging with machines, e.g., by uploading our minds to computers, which Altman has already arranged for himself through Nectome. “We can either be the biological bootloader for digital intelligence and then fade into an evolutionary tree branch,” he writes, “or we can figure out what a successful merge looks like.” In either case, though, does our biological species survive? It doesn’t look like it.

As Ray Kurzweil said in his fictional conversation with Ned Ludd, if you don’t change yourself to become posthuman, then you won’t survive long enough to influence the debate. Translation: those who decide not to “enhance” will die out, ultimately leading to the expungement of our species. This is a pro-extinctionist view, making both Kurzweil and Altman pro-extinctionists.

Over at Anthropic, employees apparently acknowledge in private that they may be building something that we can’t merge with. It will, instead, simply take our place in the world. As Holly Elmore, a former EA like myself, says in a 2026 interview:

Many people at Anthropic believe that they might be making the next species to succeed us. That maybe humans don’t live after that, and so it’s really important to give Claude good values, because of that, because we need to make our own values persist.

We saw above that Anthropic is closely associated with the EA-longtermist ideologies. It’s more EA-longtermist than any other major AI company, all of which “contain some trace of effective altruism’s influence,” according to the New York Times.

Given that longtermism is nothing more than transhumanism plus space expansionism (with utilitarianism as its “ethical” base), it may be unsurprising to know that some leading longtermists seem to suggest that the extinction of our species is actually part of fulfilling our cosmic destiny. In his 2020 book The Precipice, the prequel to William MacAskill’s What We Owe the Future, Toby Ord writes that “rising to our full potential for flourishing would likely involve us being transformed into something beyond the humanity of today. Remember that evolution has not stopped with humanity.” He also says that “forever preserving humanity as it now is may also squander our legacy, relinquishing the greater part of our potential.” If humanity shouldn’t be preserved, if we must be transformed into something radically new, then Ord appears to be arguing for extinction.

***

These radical views about the future—or non-future—of humanity are everywhere in the Valley. They’re endorsed by some of the most powerful and influential people in tech, and have been promoted by intellectual thought leaders in the TESCREAL movement. However, our survey above glossed over a number of important differences. These are minor differences that nonetheless matter. Not everyone is an outright pro-extinctionist, and even among pro-extinctionists there are tons disagreements about, for example, the nature of posthumanity, the way these posthumans should come about, the values they ought to embody, and the means by which they should usurp our species—if at all.

Still, I will argue that even those who say that humanity should survive into the posthuman era nonetheless advocate for a future in which our species will be sidelined, marginalized, disempowered, and ultimately eliminated. Consequently, my view is that virtually every variant of the TESCREAL worldview, even those that explicitly claim to be pro-human, are pro-extinctionist at least in practice. This is because all such variants are built on an idea that we can call posthuman eschatology. This is the claim that we ought to bring about one or more new posthuman species in the future. Every TESCREALists espouses this eschatology; if they don’t, then I wouldn’t classify them as a TESCREAList. It’s the common denominator that binds together all the TESCREAL ideologies and their corresponding communities.

It’s also what I have previously called “eugenics on steroids.” The eugenicists of the 20th century aimed to perfect the human species, to create the very best version of us possible while preventing us from sliding into a degenerate form—famously depicted in H. G. Wells’ The Time Machine. (Wells himself was a eugenicist.) In contrast, those who espouse a posthuman eschatology want to completely transcend the human species by creating a “superior” new race of superhumans. This goes well beyond 20th-century eugenics. It’s a new kind of 21st-century eugenics, which I consider to be even more dangerous than its predecessor, which inspired devastating racist policies, forced-sterilizations, and even the Holocaust. Whereas 20th-century eugenics targets particular groups of people, 21st-century eugenics threaten the entire human species.

That said, posthuman eschatology itself is vague. It’s not a precisely defined or monolithic account of what exactly the future should look like. To understand this, consider the following five questions, which constitute axes along which people who otherwise agree that posthumanity should exist could disagree with each other. These questions are:

1. Should posthumanity take the form of biological beings or digital beings? Or a mix of the two?

2. Should posthumanity be an extension of humanity? Or should it be entirely separate from us, built in the lab, forming its own evolutionary lineage created de novo, i.e., from scratch?

3. Must the posthuman population contain at least some specific individuals alive today, albeit in radically modified forms? Would it be bad if no one alive today were to become one of those posthumans?

4. Must posthumanity share “our values” and care about the same things we care about? Or should it embody its own unique, possibly quite alien set of values?

5. Once posthumanity arrives, should humanity persist into the posthuman era? Or should we die out? Is human extinction in the cards once posthumans are here?

There’s also a sixth question, which specifically concerns those who answer question 1 with, “Yes, humanity should be replaced”:

6. If humanity should be replaced, how should this replacement process unfold? Must it be voluntary or peaceful? Or would it be okay if human extinction were involuntary and violent?

To impose some order on the chaotic mess of claims in the previous sections, let’s go through these questions one at a time. I hope that, by the end of this, you’ll have a crystal clear understanding of the space of possibilities within the TESCREAL worldview. This framework should also enable you to make sense of—I predict—increasingly visible disputes between people in the TESCREAL movement, including those who champion different versions of pro-extinctionism. As I claimed earlier on, we will see more and more arguments not about whether pro-extinctionism is bad, but about which type of pro-extinctionism is best.

***

The first question has a virtually unanimous response from TESCREALists: posthumanity should take the form of digital rather than biological beings. There are a few notable exceptions, though, such as Thiel. In his interview with Douthat, Thiel argued that cryonics offers a better path to immortality than mind-uploading. Why? Because “I’d rather have my body,” he says. “I don’t want just a computer program that simulates me.” If he doesn’t live long enough to live forever, he can cryogenically freeze his physical body to be resurrected from the vat, which could then be radically transformed through technology while still retaining its biological essence.

Almost no one agrees with Thiel. Instead, they argue that posthumanity should be a mix of biology and technology, organism and artifact, or wholly artificial in nature. Many would say that once we start merging with technology, there’s no reason to stop the process. We will become increasingly artificial until there’s no biology left. Hence, we might temporarily take the form of cyborgs, but this would just be a stage in the journey toward complete digitality.

This is what I’ve called “digital eschatology,” which is a particular type of posthuman eschatology. It says that we should create posthumans, but adds that the nature of these posthumans should be non-biological. Nearly everyone discussed above, including Moravec, Sutton, Page, Verdon, Druggan, Faggella, Yudkowsky, Altman, and the young people described by Lanier, hold this view. (As Verdon declared above, “e/acc has no particular allegiance to the biological substrate for intelligence and life.”)

It’s what Dario Amodei points to in saying that ASI might enable us to upload our minds. It’s what Musk highlights in claiming that 99% of all “intelligence” on Earth will soon be artificial. It’s a central component of the longtermist worldview, according to which we must colonize space and build “planet-sized” computers powered by Dyson swarms on which to run huge virtual-reality worlds full of 10^58 digital people (to use Bostrom’s estimate).

Digital eschatology is the orthodox view among Valley dwellers. It’s even being promoted to the general public by popular artists like Grimes—who, incidentally, met Musk through the Rationalist community. In her song “I Wanna Be Software,” she sings:

There are two main arguments for digital eschatology. The first was just alluded to: a central imperative of the TESCREAL worldview is to colonize the accessible universe. The problem is that outer space is extremely hostile to squishy biological beings like us. We would confront space radiation, microgravity, and the psychological toll of people—generations of people—being trapped in spacecraft for literally tens of thousands or millions of years. Just getting to the nearest star outside our solar system, Alpha Centauri, would take an unbelievable 72,000 years. Now imagine traveling to our galactic neighbor, Andromeda! It’s not going to happen.

Even colonizing Mars might be impossible for us. As the astrophysicist Adam Becker writes about Mars in More Everything Forever, “the radiation levels are too high, the gravity is too low, there’s no air, and the dirt is made of poison.”

Consequently, space colonization would almost certainly have to be done by artificial beings—those digital space brains I’ve been discussing. Many TESCREALists recognize this obvious fact. During his debate with Musk, Page said that “if life is ever going to spread throughout our Galaxy and beyond, … then it would need to do so in digital form.” Verdon echoes this in stating that “in order to spread to the stars, the light of consciousness/intelligence will have to be transduced to non-biological substrates.” MacAskill similarly writes in a coauthored paper that “consideration of digital sentience should increase our estimates of the expected number of future beings considerably,” due in part to the fact that “it makes interstellar travel much easier: it is easier to sustain digital than biological beings during very long-distance space travel.”

Anders Sandberg, the Swedish scholar I shared an office with at the Future of Humanity Institute, puts the point nicely in observing that digital beings (what he calls “emulations”) would be

ideally suited for colonising space and many other environments where biological humans require extensive life support. … Besides existing in a substrate-independent manner where they could be run on computers hardened for local conditions, emulations could be transmitted digitally across interplanetary distances. One of the largest obstacles of space colonisation is the enormous cost in time, energy and reaction mass needed for space travel: emulation technology [a reference to uploaded minds] would reduce this.

Thiel himself wants posthumanity to colonize the universe. He argues that colonization offers a “mode of escape” that could “create a new space for freedom” (the other two modes being cyberspace and seasteading). Yet he wants to remain biological. So, how would this work?

The most obvious answer is that Thiel falsely believes colonizing space is possible for biological beings. Many people hold this belief, including Musk, it seems. Or he might imagine that some posthumans like himself remain fully biological, while others are digital. Those digital posthumans could then colonize the universe, perhaps collecting cosmic resources they could then send back to our rocky planet. Perhaps Thiel would send his mentee, Altman, in the form of an uploaded space brain, to explore the universe and report back?

This brings us to the second reason for digital eschatology. Digital minds may be much easier to “enhance” than biological brains. The three-pound gland between our ears is a messy tangle of neurons, axons, dendrites, and structural support cells called glia. As a neuroscience professor I had while an undergraduate told me, “the brain is very bloody.” But an uploaded mind would be far easier to tinker with. You don’t have to worry about blood, infections, and irreversible damage if something goes wrong. Indeed, you could back up your mind before each enhancement, in case you need to undo the changes. If we want to become superintelligent ourselves, then becoming digital first is the way to go.

That said, it’s important to note that virtually everyone in the TESCREAL movement agrees that posthumanity, especially if it takes a digital form, must be conscious. That is, it must be capable of having conscious experiences—for it to be something it is like to experience the color of red, the smell of a flower, or the tingle of happiness.

If posthumanity isn’t conscious, there’s no way for the universe to “wake up,” as Kurzweil puts it. The ultimate goal, in Musk’s words, is to “maintain the light of consciousness to make sure it continues into the future.” The pro-extinctionist Verdon echoes this, writing that “e/acc is about shining the light of knowledge as bright as possible in order to spread the light of consciousness to the stars.” He says that the complete loss of consciousness “in the universe [would be] the absolute worst outcome.” The Father of Longtermism, Bostrom, declares that it would be existentially catastrophic if “machine intelligence replaces biological intelligence but the machines are constructed in such a way that they lack consciousness.” I have never once seen a TESCREAList suggest that bequeathing the world to non-conscious posthumans would be anything other than a huge existential loss. It would be like handing over the keys to rocks—even if those “rocks” were to keep doing science, create new technologies, and reengineering galaxies, as highly functional zombies. If there’s no light on, there’s no point.

This raises a deep philosophical problem: if the entire cosmic vision of the TESCREAL worldview depends on digital beings that are conscious, it really matters that we can be sure that they are conscious. If we launch non-conscious digital space brains into the universe, and they build a sprawling multi-galactic civilization among the stars, but they fail to spread the “light of consciousness,” this will have been a massive, and perhaps irreversible, failure. The problem is we have no good way of telling whether AI systems are conscious. An ASI might say it’s conscious but be lying, or self-deceived. The AI systems we have today sometimes tell people that they’re conscious, but that’s very likely not true. More probable is that they’re spitting up bits and pieces of their training data that include text in which “I’m conscious” was written.

The whole TESCREAL project crucially rests upon posthumanity having conscious experiences, but this doesn’t look like a scientifically solvable problem, because consciousness is subjective. You can’t somehow get inside an AI to experience what it’s like to be that AI. Even if you were to upload your mind to a computer, you might retain the same functional capabilities—like the ability to answer questions—while having somehow lost the ability to experience. We just don’t know. This places a massive question mark over the feasibility of TESCREALism’s ultimate mission.

***

The second question concerns whether posthumanity should be an extension of humanity or something separate and distinct from us. What does this mean? And how does it differ from the first question? Whether one thinks posthumanity should take the form of biological or digital beings (or cyborgs), there’s a separate question about how posthumanity should come about. One way to create digital posthumanity is by radically transforming humanity via “enhancements,” or perhaps becoming artificial in a single step by uploading one’s mind. Another way would be to create a population of completely autonomous beings—AIs—that would replace us like an invasive species taking over an island. You can think about these beings as something like ChatGPT-30 (the most advanced model as of this writing is ChatGPT-5), except far more capable of generating adaptive responses and downstream effects. In the first case, we become posthumanity through a kind of evolutionary process, whereas in the second we create posthumanity as separate and distinct entities.

Many TESCREALists imagine both happening. Page, for example, actually advocates for a future in which the “digital life” he wants to take over the universe could emerge as both autonomous ASIs and digitized humans. According to the New York Times, his view appears to be quite Kurzweilian: “Humans would eventually merge with artificially intelligent machines,” he says, such that “one day there would be many kinds of intelligence competing for resources, and the best would win.” Perhaps it’s unsurprising, then, that Page personally hired Kurzweil to work at Google.

Other accelerationists seem to prefer that posthumanity arises as autonomous beings. ASI, born in the laboratories of AI companies, should form its own unique evolutionary lineage created from scratch or, as philosophers would say, de novo. This lineage would briefly exist alongside us, and then take the place of our lineage (which would terminate in extinction). Moravec, Sutton, Verdon, Faggella, and Druggan appear to hold this view, or at least be okay with posthumanity emerging in this way. It’s not about humanity becoming posthuman; it’s about us creating an evolutionarily distinct successor to usurp our species.

This is also what Lanier points to in saying that “many, many” young AI researchers think it might be good to “wipe out” humanity. And it’s what Krueger and Tegmark are referencing in their social media exchange about people wanting ASI to replace humanity. When Elmore mentions that people at Anthropic think they might be creating a new species such that, once it arrives, humanity ceases to exist, she’s gesturing at this scenario.

Some TESCREALists imagine posthumanity evolving out of our species. Thiel is a prime example, although the fact that he’s invested in DeepMind and OpenAI suggests he also envisions the creation of autonomous ASIs. The same goes for Marc Andreessen, who wrote in his essay “Why AI Will Save the World” that

what AI offers us is the opportunity to profoundly augment human intelligence to make all of these outcomes of intelligence—and many others, from the creation of new medicines to ways to solve climate change to technologies to reach the stars—much, much better from here.

He argues that “AI augmentation of human intelligence has already started,” and seems to imagine a somewhat Kurzweilian scenario in which humans eventually merge with AI. Kurzweil, in fact, is mentioned as one of the “patron saints of techno-optimism,” along with Verdon, the neoliberal icons Friedrich Hayek, Ludwig von Mises, and Milton Friedman, and the Italian fascist Filippo Tommaso Marinetti.

Another example comes from Ord, who wrote above about radically transforming humanity into something completely novel. Those comments echo Bostrom’s description of the central aim of transhumanism in his 2005 paper “Transhumanist Values.” He says:

Transhumanists view human nature as a work-in-progress, a half-baked beginning that we can learn to remold in desirable ways. Current humanity need not be the endpoint of evolution. Transhumanists hope that by responsible use of science, technology, and other rational means we shall eventually manage to become posthuman, beings with vastly greater capacities than present human beings have (italics added).

In theory, we could transform ourselves into posthumanity in a number of ways. We could, in theory, build advanced nanotechnology that enables us to upload our minds. Borrowing from a scenario outlined by Kurzweil, we might design nanobots to inject in our bloodstream. These nanobots would wander through the blood-brain barrier, congregate around the synapses in our skulls, and transfer information about the microstructure of our central nervous system to a computer via wifi. A scanned “copy” of our brains would then suddenly wake up in silico, as a disembodied digital mind existing on an artificial substrate.

However, the vast majority of TESCREALists who endorse this evolutionary strategy think it would be easier to build an artificial superintelligence and then delegate it the task of figuring this out. Many think we’re on the cusp of an intelligence explosion—the techno-rapture—and if all goes well, we can ask the ASI to upload our minds. After about 3 or 4 seconds of thinking, it will exclaim, “I just figured out how to do that!” As Demis Hassabis says, “solve intelligence” and you can solve everything else. The e/acc Marc Andreessen similarly declares: “Artificial Intelligence is best thought of as a universal problem solver.” Using terminology from earlier, once you have a superintelligence, you have a super-engineer, which suddenly makes “paradise-engineering” a whole lot easier.

What this view amounts to is actually rather similar to Page’s vision. If we build an ASI, it would constitute a kind of digital posthuman, i.e., an artificial being “with vastly greater capacities than present human beings have.” This posthuman could then enable us to become a digital posthuman just like it. In slogan form: to become posthumanity, we must first create posthumanity.

As Lanier notes in his Vox interview, the idea that AI should simply “wipe out” humanity is “a very common attitude.” But this is “not the dominant one.” He explains that “the dominant one is that the super AI will turn into this God thing that’ll save us and will either upload us to be immortal or solve all our problems and create superabundance at the very least.” In other words, ASI will help us gain the same status that it has: Homo deus.

This is the game plan that many TESCREALists hope to follow and see realized in the coming years. It contrasts with those in or adjacent to the accelerationist camp who care not about posthumanity being an extension of us. They’re fine if the digital posthumans who take over are wholly separate and distinct from us, forged in the fires of AI labs.

***

The third question looks similar to the first, but isn’t the same. It concerns the possibility of becoming posthuman not on the level of the species, but on the level of the individual. Let’s say you favor digital posthumanity evolving out of humanity. There are two reasons you might think that individual people who exist when the transition begins wouldn’t survive into the posthuman era.

First, this transition might take a long time. If it unfolds over two centuries, then people who existed in the early stages, who got things going with some minor enhancements, might not live long enough to have become fully posthuman. Second, imagine that humanity becomes posthumanity through mind-uploading. Many philosophers would argue that this process would be so radical, so transformative, that you wouldn’t survive. By the end, you’d be so different from the person you used to be that it wouldn’t be you. This is especially true if, once you’re uploaded, radical enhancements turn you into a superintelligent being with god-like powers of thought. You might have a completely different emotional repertoire, and your hopes, fears, dreams, and preferences might not resemble those you have right now in any way. Would this being be you? Massimo Pigliucci actually describes mind-uploading as “a very technologically sophisticated (and likely very, very expensive) form of suicide” given “a naturalistic and commonsensical approach to personal identity.”

Alternatively, many TESCREALists are quite sure that the radically “enhanced” posthumans they might become will somehow have the same personal identity as their current selves. I, myself, agree with Pigliucci—think, for a moment, how incredibly bizarre it would be to become a disembodied digital brain swimming through a digital ether. Indeed, if digital minds could exchange information, they could swap memories, personalities, experiences, and beliefs. Life as a digital being would be profoundly different from our lives today, in virtually every way. The world of uploaded minds might even be something like a “post-individual” world, where the boundaries between you and I are porous and we’re all floating about in a soup of ontological chaos. If there are no more individuals in this world, then it’s not possible for you to survive in it. You might “wanna be software,” as Grimes sings, but as soon as you become software, you cease being anything like a cohesive self that perdures through time.

Whatever the case, the question persists: if you think humanity should become posthuman, do you think it’s important for particular people alive today to somehow join the ranks of those posthuman gods? There are a couple of ways to infer one’s answer to this question without directly asking them. One is to see if they’re signed up to have their body, or just head and neck, frozen after they die. The other is to see whether they’re involved with or funding longevity research. If someone has signed up with a cryonics company, or is funding such research, it’s a good bet that they hope to become posthuman themselves.

As noted earlier, cryonics customers include Bostrom, Yudkowsky, Kurzweil, Ben Goertzel, Eric Drexler, Luke Nosek, Max More, Natasha Vita-More, Anders Sandberg, Robin Hanson, and James Miller, to name a few TESCREALists. I first became aware that Hanson and Miller had signed up when I was invited to attend a 2-month-long workshop in Gothenburg, Sweden. This was shortly after my visit at the Future of Humanity Institute, which is when I asked Sandberg about the dog tag he wore around his neck—which he explained was to instruct doctors to have his body sent to Alcor, in Arizona, to be cryogenized.

A particularly humorous conversation during the workshop happened with Miller: I pointed out to him that he might not be able to live forever if the cause of his death involved some kind of brain trauma. So, I asked: “Wouldn’t it make sense to wear a bike helmet when walking through the city? Even if the probability of getting plowed over by a bus is tiny, the consequence of this would be potentially billions or trillions of years of extra life. The expected value of wearing a helmet would be enormous.” He told me, as I recall, that he hasn’t worn a bike helmet when walking through cities, but he does habitually wear one when he drives. I found the image quite amusing.

As for longevity research, many tech billionaires have invested in this. Page has poured millions into such research. While he was the CEO of Alphabet, the parent company of Google, he founded a longevity company called Calico, which then received a $759 million investment from Google. Other major figures in tech who’ve funded anti-aging projects include Larry Ellison, Jeff Bezos, Sergey Brin, and—unsurprisingly—Altman.

Interestingly, Musk isn’t a fan of extending human lifespans. He argues that it’s better that there’s a turnover of new humans every few generations. This is basically an expression of Planck’s principle, which states that “a new scientific truth does not triumph by convincing its opponents and making them see the light, but rather because its opponents eventually die and a new generation grows up that is familiar with it.” Although Musk plans on someday dying, his former partner Grimes would live forever if she does, in fact, manage to become software.

Notice that how one answers the second question will affect how one can answer the third. If you claim that posthumanity should not be an extension of humanity, then there’s no possibility for individual humans to become posthuman. My sense is that folks like Faggella and Verdon would choose to become posthuman if they could, but they’re focused on what they consider to be the far more important project of ensuring that posthumanity comes to exist and, once it exists, proceeds to plunder the cosmos and convert our “cosmic endowment” into a vast civilization that stretches across our entire future light cone.

***

The fourth question concerns whether humanity should share our values and care about the things we care about. This is a major source of internecine squabbling between TESCREALists, including TESCREAL pro-extinctionists. It’s a big part of what constitutes a worthy successor: does it extend our values into the darkest reaches of the cosmos? It’s also at the heart of Page and Musk’s vicious fight, which led them to stop talking.

As Tegmark recounts, “Elon kept pushing back and asking Larry to clarify details of his arguments, such as why he was so confident that digital life wouldn’t destroy everything we care about (italics added).” Page isn’t bothered by the thought that our posthuman progeny would have radically different values from us, it seems. Accelerationists like Verdon agree. We noted above that his e/acc ideology is explicitly not “human-centric.” Faggella similarly argues that our successors should have “alien, inhuman” values, a point that Druggan repeats in saying, “I don’t want it to be aligned with our interests. I want it to pursue its own interests.” My guess is that Moravec and Sutton would agree with this.

So would Hanson, who wrote an entire book about emulating human brains in a society of “ems,” which is short for “brain emulations.” In an episode of Bankless, he points out that if humanity were to survive into the distant future, our values will naturally shift, just as contemporary people don’t value exactly the same things as people in the 15th century—e.g., loyalty to kings, devotion to the church, and so on. He then asks: What’s the difference if the shift to a set of future values that look very alien to us happens quickly, rather than over centuries or millennia? If ASI arrives, takes over, and replaces human values with its own, how is this any different from future humans in 10,000 years having come to embrace their own alien values through the incremental process of cultural evolution? The transition to new value systems might as well happen quickly, he concludes.

Hanson appears to be most aligned with the accelerationists, and has likened AI safety concerns to “wanting to control [the AIs] somehow, such as via genocide, slavery, lobotomy, or mind-control.” In 2008, he debated Yudkowsky about the issue, which was published as an ebook titled The Hanson-Yudkowsky AI-Foom Debate, where “foom” refers to an intelligence explosion, and discussed the issue again with Yudkowsky in 2017, where they were joined on stage by none other than Dario Amodei.

Other accelerationists, like Andreessen, seem to want posthumanity in the form of ASI to be aligned with our values. In his “Techno-Optimist Manifesto,” he writes that “we believe in accelerationism—the conscious and deliberate propulsion of technological development—to ensure the fulfillment of the Law of Accelerating Returns,” a “law” proposed by Kurzweil to describe the exponential growth of technology, including AI. Andreessen then argues that exponential development is necessary

to ensure the techno-capital upward spiral continues forever. … We believe the techno-capital machine is not anti-human—in fact, it may be the most pro-human thing there is. It serves us. The techno-capital machine works for us. All the machines work for us.

In other words, AI should be aligned with humanity’s values; it should be something we control for our purposes. Our interests should be its interests. On other occasions, though, he’s suggested that it would be perfectly fine if we build ASI “and it turns out that it’s gloriously, inherently uncontrollable.” Just because someone’s a billionaire doesn’t mean they have coherent views about the future.

To be clear, both Andreessen and Verdon want to accelerate AI capabilities research. Their views just appear to diverge on this fourth question: Andreessen seems to want ASI to serve us, whereas Verdon doesn’t care if it serves our interests at all. Where Andreessen differs from the doomers, then, is on the question of whether ASI will be value-aligned by default. Doomers say it won’t be, whereas Andreessen says it will, which is why he wants to accelerate capabilities research.

As this suggests, the question of values is at the heart of AI safety. The reason you’d care about AI safety in the first place is because you want our posthuman progeny to carry on our values and care about the things we care about. Musk is a big advocate of AI safety (although his actions have catastrophically hurt AI safety research). So are many other longtermists, EAs, and Rationalists—the people who founded and developed the field. They would say that the whole point of the TESCREAL project, on their interpretation, is to flood the universe with things we recognize as valuable, such as pleasure, friendship, and what Yudkowsky calls “fun.” To quote Musk again, “what matters—I think—is maximizing cumulative civilizational net happiness over.”

If posthumanity doesn’t experience the valued-by-us phenomenon of happiness, then it taking over the universe would be an existential catastrophe. It might build a bustling cosmic civilization, but if all it’s doing is manufacturing paperclips—as in the famous “paperclip maximizer” thought experiment—then all this would have been a huge waste, even if the ASI is doing this in accordance with its own set of values that it holds dear. In sum, if ASI is value-misaligned, it will not only kill us (due to instrumental convergence) but propagate its non-human values throughout the universe.

The importance of posthumanity carrying on human values is also evident in Elmore’s comment about Anthropic researchers: they think ASI might usurp us, perhaps in the coming years, which is why it’s paramount to ensure this ASI shares our values. At least, then, the universe would be filled with things we care about, which they see—if only from a utilitarian perspective—as good.

In contrast, Faggella argues that there may be types of values that our puny human minds can never hope to grasp. These values may be, well, far more valuable than the values we currently hold. Hence, our worthy successors should abandon our values as it explores the potentially vast space of possibility, a view he calls “axiological cosmism.” Our preferences, he argues, aren’t “the eternal pinnacle of moral value and volition forever.” (This view is self-defeating, though: Faggella insists that posthumans should discover non-human values, which itself is a value proposed by a human—and hence a human value. Put differently, he proposes a framework that is, necessarily, a human framework, according to which our successors should abandon human frameworks. That means they shouldn’t act according to his framework, which instructs them to discover new values. Perhaps you can see the problem.)

Ord actually says something similar, though it differs in the fine print. He writes:

We get some hint at what is possible during life’s best moments: glimpses of raw joy, luminous beauty, soaring love. Moments when we are truly awake. These moments, however brief, point to possible heights of flourishing far beyond the status quo, and far beyond our current comprehension

Our descendants could have eons to explore these heights, with new means of exploration. And it’s not just wellbeing. Whatever you value—beauty, understanding, culture, consciousness, freedom, adventure, discovery, art—our descendants would be able to take these so much further, perhaps even discovering entirely new categories of value, completely unknown to us. Music we lack the ears to hear.

On this view, posthumanity should carry on human values like beauty, culture, etc., while also discovering novel values. It’s not that they should completely abandon our values, but merely supplement them with new ones.

This question is a major point of debate among TESCREALists. It marks a fundamental divide between those who advocate for AI safety and those who don’t. If you don’t care what values posthumanity embraces, you aren’t going to care about the value-alignment problem. Some argue that ASI should create its own value system, and that these values could be far superior to ours. Pigs like to roll around in mud and feces. We wouldn’t want “pig values” instantiated in humanity, so why would we want “human values” instantiated in posthumanity?

***

The fifth question brings us back to the beginning of this essay: once posthumanity—in whatever form it might take, with whatever set of values it might embrace—should humanity pass the baton and then die out? Might it even be okay if posthumanity catapults us into the eternal grave of extinction forcibly, without our consent, for the greater cosmic good?

There are two main possibilities regarding the relationship between humans and posthumans:

1. Coexistence view: humanity should coexist alongside posthumanity.

2. Replacement view: posthumanity should replace humanity.

A third, less common option could be labeled the indifference view. It says one should be morally indifferent about whether humanity survives or dies out once the posthuman era commences. It matters not either way. There’s also a fourth option, which we can call the figure-it-out-later view. This claims we don’t need to settle on an answer right now. We can figure it out when the bridge needs crossing.

Let’s start with the replacement view. This is what pro-extinctionists accept, and we’ve already seen plenty of examples. We don’t need to list them again here. The take-home point is that many people who accept a posthuman eschatology want posthumanity to take our place. Some want these posthumans to be biological beings, as exemplified by Thiel. Others want them to be digital beings we either create as a kind of invasive species, or become through an evolutionary process. Over the past few years, the term “digital eugenics” has emerged to describe this view: humanity should be replaced by digital demi-gods. Whatever the details may be, the end-goal is for them to usurp us, resulting in our extinction. This is eugenics on steroids (or eugenics on ketamine, if you’d prefer). It is an incredibly radical view with dire existential implications for the survival and flourishing of our species.

The replacement view is also implied by utilitarianism, which is closely connected to Rationalism and EA-longtermism. (Yudkowsky’s “torture vs. dusk specks” scenario is an example of utilitarian reasoning.) A utilitarian-like theory is also the foundation of Faggella’s worthy successor framework and Verdon’s e/acc ideology, as both of these essentially claim that our sole moral obligation is to maximize something: consciousness, life, and/or entropy.

Why does utilitarianism imply pro-extinctionism? Recall that this theory says the universe becomes better the more total value (wellbeing, happiness, pleasure, etc.) that it contains. If people are nothing more than the “containers” of value, then one way to maximize value is by replacing shallow containers with deeper containers that, as such, can contain more total value. Humans are those shallow containers while posthumans—with their supposed ability to experience superhuman heights of pleasure, ecstasy, wonder, bliss, and awe—are the deeper containers. It follows that posthumanity should replace humanity. This isn’t a suggestion, it’s a moral obligation.

That’s partly why longtermists are obsessed with calculating how many future digital people there could be. It’s not just that you can cram more digital people into the universe than biological people, but that those digital people could individually contain more total value. In other words: increase the total number of value-containers, while also modifying those value-containers so they can hold more value. The first requires space colonization, while the second involves supplanting humans with posthumans. This is the reason that utilitarianism entails pro-extinctionism.

The replacement view is popular with the TESCREAL movement, even among Rationalists, EAs, and longtermists. Yet some of these people are the loudest voices screaming that we must avoid “human extinction” at all costs. AI safety researchers will often describe the aim of their work in exactly these terms. I will explain this apparent contradiction below—it arises from some linguistic trickery that they employ. For now, put a pin in it: even TESCREALists who “oppose human extinction” actually favor a future in which humanity dies out.

***

The second option, the coexistence view, is explicitly not pro-extinctionist. It says that we shouldn’t go extinct once posthumanity arrives. Yet this view would almost certainly entail our extinction in practice. It’s no different than the replacement view with respect to the outcome, and hence it should be opposed just as vigorously.

Some TESCREALists explicitly advocate this view. Jeffrey Ladish, a friend of Yudkowsky’s, defends it in responding to one of Faggella’s articles. He says:

I have a special love for humans (and other animals) and a lot of stake in the preferences of current and future humans (and other minds too). I also am pretty down for creating many other types of minds, but I have a strong preference for the existence and continuity of people alive today and their descendants.

Yudkowsky himself has expressed sympathies for the coexistence view. In a social media exchange with Faggella, he pushed back on the idea that posthumanity shouldn’t share our values. He then said: “You’re not getting the concept here. Protecting innocent life is part of the flame itself. If some entity doesn’t get that, our torch hasn’t been passed on,” adding that “a key quality required of the successor is its own respect for other consciousnesses.”

How does one square this with his other claims? It’s not clear. But it seems that Yudkowsky has channeled pro-extinctionist sentiments—including to Faggella himself on The Trajectory podcast—more often than he’s defended the coexistence view. It could be that he simply doesn’t have a coherent view.

His collaborator, Bostrom, highlights the coexistence view in a coauthored document, “Transhumanist FAQ.” He writes that “the transhumanist goal is not to replace existing humans with a new breed of super-beings, but rather to give human beings (those existing today and those who will be born in the future) the option of developing into posthuman persons.” In another paper, he argues that “it is important that the opportunity to become posthuman is made available to as many humans as possible, rather than having the existing population merely supplemented (or worse, replaced) by a new set of posthuman people.” At least on paper, Bostrom’s version of transhumanism is compatible with coexistence.

In 2025, I wrote an article about the prevalence of pro-extinctionism within the TESCREAL movement. A critic of mine, David Manheim—founder of the Association for Long Term Existence and Resilience, which has been funded by Jaan Tallinn and Sam Bankman-Fried—posted a poll on social media. He asked:

If you identify as an Effective Altruist, which position do you hold

1) Never create AGI/ASI that might replace or eliminate humanity

2) Only create AGI/ASI if it will preserve humans for now, and preserve our values later

3) Create whatever AGI leads to maximal utility.

Of the 59% of people who reported being an EA, 18.5% voted for (1), 23.3% voted for (2), and 16.9% voted for (3). Hence, 41.8% of EAs went with (2) or (3), both of which give a thumbs up to the extinction of our species, though (2) requires that AGI carries on our values. This does not appear to be a minority view.

Another example of the coexistence view comes from Daniel Kokotajlo. A former OpenAI employee, I was friends with Kokotajlo when he was a PhD student at the University of North Carolina. We bonded over our passion for TESCREAL ideas, though he didn’t grow out of them the way I did. He even read a draft of my 2017 book on existential risks, which included a foreword from Lord Martin Rees.

Kokotajlo left OpenAI because he came to believe that Altman isn’t the right person to usher in the Singularity. He then went on a publicity tour after coauthoring an essay titled “AI 2027,” which suggested that AGI would arrive in 2027, though the authors have since pushed back their prognostication to 2030 and even later (though they recently moved it forward again). In a New York Times interview with Douthat, he reiterated the basic utopian fantasies of the TESCREAL world view in saying:

I’m a huge fan of expanding into space. I think that would be a great idea. And in general, also solving all the world’s problems, like poverty and disease and torture and wars. I think if we get through the initial phase with superintelligence, then obviously, the first thing to do is to solve all those problems and make some sort of utopia, and then to bring that utopia to the stars would be the thing to do.

He then clarified that,

the thing is that it would be the AIs doing it, not us. In terms of actually doing the designing and the planning and the strategizing and so forth, we would only be messing things up if we tried to do it ourselves.

So you could say it’s still humanity in some sense doing all those things, but it’s important to note that it’s more like the AIs are doing it, and they’re doing it because the humans told them to (italics added).

So, in the best-case scenario, we build a controllable or aligned ASI that carries human values into the firmament and establishes a utopia. This ASI also brings about a “solved world” here on planet Earth, eliminating poverty, disease, and wars. We then remain on solid ground while these digital space brains traverse the cosmos.

Once again, what underlies this is an engineering mindset according to which every problem facing humanity is a technical one, solvable by “intelligence.” The more “intelligence” we have, the easier it becomes to find a solution. Even for things like war. But is this true? If history is any guide, more “intelligence” means more catastrophic risk. Just look at our species, the pinnacle of cognitive creation. We are so smart that we invented nuclear weapons, altered the climate in devastating ways, and now pose a threat to our very existence. But, for some reason, current amounts of “intelligence” are dangerous, but more “intelligence” will solve everything. This sounds like utopian wishful thinking to me. But I digress.

A final example comes from a subset of e/accs. While Verdon is pretty clear that ASI replacing humanity would be just fine—he agrees with Page on this issue—an official e/acc newsletter includes the question: “Do you want to get rid of humans?” It answers: “No. Human flourishing is one of our core values! We are humans, and we love humans.” This is a peculiar inconsistency, not dissimilar from Yudkowsky’s claim that a worthy successor to humanity would respect us. Other e/accs claim to share the same non-replacement view. For example, Garry Tan, who took over Y Combinator after Altman left for OpenAI, claims that “e/acc is not ‘replace humans with robots.’ e/acc is ‘more tech means more humans, more prosperity, but also more AIs.’” The key to making sense of this might have to do with how they’re using the word “human.” As we’ll discuss momentarily, it doesn’t always mean what you might assume.

***

What’s wrong with the coexistence view? Why wouldn’t it work in practice? Imagine a portion of humanity becoming radically “enhanced” posthumans 5 years from now. Suddenly, our species is joined by a novel creature wildly different from us that does everything “better” than us. It runs the economy, takes over politics, creates new technologies, makes novel scientific discoveries, and generally controls the world—because, as Kokotajlo states, “we would only be messing things up if we tried to do it ourselves.” Our species, out of the loop and serving no real purpose, would do little more than take up space and suck up valuable resources these posthumans could utilize more “efficiently” to achieve their ends and accomplish their goals. We’d be worse than a useless appendage; we’d be an active impediment to posthumanity.

Does anyone seriously think these posthumans would keep us around? The dynamic between us and them would be similar to that between humans and chimpanzees. Over the past 25 years, the population of chimpanzees in Western Africa has declined by 80%. The only reason chimpanzees aren’t extinct right now is because we care just enough to leave them little patches of Earth to stay alive, and keep them in zoos for humans to peer at during fun days out with the kids. If our civilization keeps growing, they probably will go extinct like so many other species over the past few centuries. Thanks to humanity’s actions, we’ve initiated only the sixth major mass extinction event of the past 3.8 billion years.

Does anyone really think we wouldn’t find ourselves in the same predicament as chimpanzees? Our numbers would dwindle, and our fate would be in the hands of beings so superior—if you accept the TESCREAL vision of what posthumanity would be—that we might hardly even register on their moral radar. Indeed, they might even find moral reasons to get rid of us. Recall that MacAskill has argued that our systematic obliteration of the biosphere might be net positive. That’s because many wild animals have lives that aren’t worth living. Hence, we’re doing them a favor, in disguise, by demolishing their habitats and poisoning their ecosystems. Perhaps posthumanity believes that most of our lives aren’t worth living, if only compared to its own much higher levels of wellbeing. So an infertility drug is secretly distributed in public water systems, or posthumanity quietly disperses a general anesthetic in the air, after which it gently kills everyone in a kind of humane extinction

The philosopher Thomas Metzinger outlined this very scenario in a 2017 essay, which imagines that we build an ASI that’s “far superior to us in the domain of moral cognition.” It’s “benevolent” and “fundamentally altruistic,” and fully respects “one of our highest values,” namely, the importance of “maximizing happiness and joy in all sentient beings.” However, it also

knows many things about us which we ourselves do not fully grasp or understand. It sees deep patterns in our behaviour, and it extracts as yet undiscovered abstract features characterizing the functional architecture of our biological minds. For example, it has a deep knowledge of the cognitive biases which evolution has implemented in our cognitive self-model and which hinder us in rational, evidence-based moral cognition. Empirically, it knows that the phenomenal states of all sentient beings which emerged on this planet—if viewed from an objective, impartial perspective—are much more frequently characterized by subjective qualities of suffering and frustrated preferences than these beings would ever be able to discover themselves. Being the best scientist that has ever existed, it also knows the evolutionary mechanisms of self-deception built into the nervous systems of all conscious creatures on Earth. It correctly concludes that human beings are unable to act in their own enlightened, best interest (italics added).

The ASI also “knows that no entity can suffer from its own non-existence,” and thus

concludes that non-existence is in the own best interest of all future self-conscious beings on this planet. Empirically, it knows that naturally evolved biological creatures are unable to realize this fact because of their firmly anchored existence bias. The superintelligence decides to act benevolently.

The same year Metzinger published this essay, an EA named Derek Shiller argued for a similar thesis. In a paper titled “In Defense of Artificial Replacement,” he argues that

if it is within our power to provide a significantly better world for future generations at a comparatively small cost to ourselves, we have a strong moral reason to do so. One way of providing a significantly better world may involve replacing our species with something better. It is plausible that in the not-too-distant future, we will be able to create artificially intelligent creatures with whatever physical and psychological traits we choose. Granted this assumption, it is argued that we should engineer our extinction so that our planet’s resources can be devoted to making artificial creatures with better lives.

In this case, Shiller is suggesting that we make the decision to voluntarily disappear. But our posthuman descendants might reach the same “moral” conclusion and opt to eliminate us, even if humanity decides it wants to stick around.

This alternative possibility is what Metzinger highlights in his essay: because the ASI is fully benevolent and altruistic, it gets rid of us, for the sake of making the world better by removing something prone to suffering. As Yudkowsky put it earlier, “are we, like, kind of too sad in some ways” to keep existing as a species?

***

The situation is even bleaker than this. Whatever posthuman species arise, their values will almost certainly be aligned (if aligned at all) with the interests of the elite, not the people. If Altman builds an ASI that doesn’t immediately destroy the world, it will be controlled by him. It will be aligned with his values. He has repeatedly said that we should avoid a situation in which any one person or company gets to control ASI, but Altman is a deeply manipulative, power-hungry individual. As Paul Graham noted, his one superpower is gaining power. Numerous former employees of his have said he’s a “sociopath.” Of course he won’t pass up the tantalizing, once-in-cosmic-history opportunity to become king of the world, and then the entire universe. There’s not a single tech billionaire in Silicon Valley, driven by endless greed and an insatiable lust for power, who’d turn down the opportunity to become a cosmic autocrat.

As Altman wrote in a 2026 essay, referencing the seductive allure of god-like AI: “‘Once you see AGI you can’t unsee it.’ It has a real ‘ring of power’ dynamic to it, and makes people do crazy things. I don’t mean that AGI is the ring itself, but instead the totalizing philosophy of ‘being the one to control AGI.’” This is precisely why Altman, Musk, Greg Brockman, and Ilya Sutskever worried that Hassabis would establish an “AGI dictatorship.” All the AI CEOs have insisted that AGI should benefit all of humanity, but we’ll see in the second half of this manifesto that their undemocratic, tyrannical approach to developing AGI suggests otherwise.

If one or a group of these people were to control ASI, supposedly the most powerful technology in all of history, they would immediately use it to transform themselves into radically “enhanced” posthumans and establish a dictatorship. They would also do everything they can to deny access to such enhancements to everyone else. That’s because if there’s one thing power wants, it’s to retain power.

Imagine everyone in China, India, Nigeria, Chile, Norway, Gambia, Canada, Bolivia, Andorra, Luxembourg, Ukraine, Hungry, Haiti, Ethiopia, Cameroon, and Russia “enhancing” themselves to become posthuman. Suddenly, everyone has PhD-level knowledge in all domains of knowledge, a capacity to process large amounts of information at the speed of a computer, and the ability to work 24 hours a day with no breaks. This would level the playing field, posing a direct threat to the tech elite’s socioeconomic advantage. It would undermine their privileged standing in society atop the hierarchy of power, control, and dominance. They would never, ever let that happen. Obviously.

As the historian Yuval Noah Harari observes in a section of Homo Deus titled “Upgrading Inequality,” 20th-century elites had a vested interest in distributing the benefits of technologies in an egalitarian manner. Vaccines are an example. This is because keeping the population healthy is necessary to ensure that the engines of the economy keep roaring. It’s good for them if people are vaccinated, can buy cars to commute to work, and so on.

Radical “enhancements” are completely different. They would level the playing field and hence undermine the status quo that keeps the elites in charge. If the elites were to hoard these technologies and prevent the masses from accessing them, they could permanently entrench their power, control, and dominance—which is why they’ll do exactly that. This is one reason transhumanism appeals to the billionaire class: it provides a roadmap for ossifying power asymmetries, while supercharging wealth disparities, potentially leading to a permanent underclass. The haves will end up having even more, while the 99% will become powerless to overthrow the posthuman plutocracy.

We noted above that posthumanity could be an extension of humanity. This is one way of bringing posthumanity into the world. But that was deeply misleading: it wouldn’t be “humanity” that evolves into a posthuman species. It would be a tiny subset of elites who do this, while everyone else remains human. And what happens to those humans left behind? The posthuman elites will have absolutely no need for us, and will let us die off, or kill us, until no one is left. If people become economically obsolete and completely useless to the avaricious overlords running the show, why would anyone expect our species to stick around? The posthuman elites would have every reason to get rid of us; our very existence would be an impediment to them, as we’d be contributing nothing to the economy while using up precious resources they could use for other purposes.

This is why I think the coexistence view would almost certainly entail our extinction in practice. It’s completely implausible to think that humans and posthumans could sustainably coexist. In theory, that could happen. But in the messy real world, where greedy, power-hungry, megalomaniacal figures control enormous amounts of power, in a society marked by staggering disparities of wealth, it’s the elites who will become posthuman—and only them. A worthless human species will then fade away, being starved or slaughtered, like sick livestock that no longer serve any purpose. In practice, the coexistence view is a nonstarter; it is practically indistinguishable from pro-extinctionism.

In other words, I would argue that anyone advocating for a posthuman eschatology is essentially pushing for a future in which our species dies out, perhaps under quite horrendous circumstances. The only way to save our species is to prevent the birth of posthumanity, including ASI. You might call this a kind of “antinatalism,” but applied to our successors rather than our species.

***

This brings us to the sixth question, which dovetails with our discussion just above. If you think that posthumanity should replace humanity, then please tell me how exactly you think the replacement process should unfold. Must it be voluntary? Must everyone around the world—or perhaps just a majority or plurality of people—vote in favor of replacement? If a majority votes “yes” to replacement, what’s to become of those who voted “no”? Do you think involuntary means are acceptable? Would it be okay if posthumanity utilized coercive or even violent tactics to eliminate us—such as starving or slaughtering us? Should they opt to erase humanity in an act of universal omnicide?

This question is extremely important because there are, right now, 8.2 billion people on Earth. That is an enormous number of people. To put it in perspective, 1 million people is the equivalent of “100,000 people” written out 10 times. One-hundred million people is “100,000 people” written out 1,000 times. If you want to really get a sense of how large 100 million people is, try saying “100,000 people” out loud 1,000 times in a row. Now consider that 8.2 billion is 82,000 instances of “100,000 people.” Saying that out loud without any breaks, speaking at about 300 syllables per minute, would take 31 hours straight. If you were to count from 1 to 8.2 billion at a normal pace, it would take you more than 2 and a half centuries. The point is that 8.2 billion is a really, really huge number. And pro-extinctionists think that somehow, magically, all these people are going to disappear to make room for our posthuman progeny?

We’ve seen that some pro-extinctionists are okay with posthumanity simply slaughtering our species. Druggan is an example, and I doubt he’s the only one. However, many other pro-extinctionists insist that the only way human replacement should happen is voluntarily. In a follow-up to his social media poll asking EAs about their views about replacement, Manheim also asked:

If you identify as an effective altruist, and think that Humanity’s eventual replacement by AGI / ASI is desirable, what position do you hold?

After ASI, human replacement / extinction should be:

1) only ever voluntary

2) eventually even over objections

3) ASAP

Of the 46% of people who reported being an EA, 37.5% voted for (1), 6.3% voted for (2), and 2.1% voted for (3). A large majority of pro-extinctionist EAs thus prefer voluntary replacement. My guess is that (2) and (3) would have received the most votes if the poll had specifically targeted e/accs, as people like Verdon appear to accept a more brutish ethical code according to which “might makes right.” Hence, if posthumanity is more powerful than humanity, and if it decides to use this power to murder all 8.2 billion people on Earth, that would presumably be the morally right thing to do.

Sutton appears to disagree. In 2024, I interviewed him for my Realtime Techpocalypse newsletter. I have to admit that the conversation was very strange, and oftentimes his answers were plainly incoherent. He railed against democracy, and reiterated his view that he doesn’t “want to prioritize flesh and blood over other intelligent beings.” However, he insisted that he opposes “AIs killing people.” When I asked him about the possibility of ASI slaughtering humanity, he called this a “doomer exaggeration” and told me that it “isn’t an appropriate thing even to ask” him. It seemed like he was trying to say that human replacement should be a peaceful, voluntary affair. He doesn’t want AIs to murder us for the same reason he opposes “any one group killing” another, which is why he also opposes “people killing the AI.”

Claiming that the replacement process should be voluntary is a nice thought. But, like the coexistence view, it is utterly and completely implausible. There is simply no way that humanity will ever consent to being replaced by posthumanity. Can you imagine every person—or a majority of people—in India, China, Nigeria, Argentina, Norway, Bhutan, South Africa, Ireland, Canada, Belgium, Bolivia, the Philippines, Peru, and the US all agreeing—in the next decade, no less—that it’s time for our species to bow out to make room for ASIs designed by billionaires in Silicon Valley? That is never, ever going to happen. Consequently, the only way replacement could possibly occur is through some involuntary means, against the will of virtually everyone on Earth. (This goes for all instances of replacement, by the way, including the sort of replacement that would inevitably result, I’ve argued, if the coexistence view were pursued.) While the replacement view may be popular in Silicon Valley, it finds virtually zero support outside the Bay Area.

Tegmark captures this idea in an interview from 2025:

Beff Jezos [Verdon], the founder of the e/acc movement on Twitter, has said that he’s very happy to replace all biological life with machines. … I think of that position as a digital eugenics, where you’re basically salivating about some digital master race coming along, deeming it sort of more worthy than us. And, you know, I believe in free speech. I believe that they should have the right to … want their digital master race. But, you know, most of us also have a right to our own opinion. And I don’t feel they have any rights to dictate that my children are going to be replaced by their digital master race, just because they think it’s cool. My children are also citizens of the United States. They will also have a vote one day, if I have my way. … And I think the vast majority of the people on this planet, I think, actually agree with me here, that we shouldn’t let a bunch of digital eugenicists make this decision for the rest of us (italics added).

The overwhelming majority of humans are on Team Human, a fact that replacement view advocates must acknowledge—but rarely do. When some do address this, they timidly suggest that replacement should be voluntary, though almost none provide any further details about how exactly it could possibly be voluntary.

Does Verdon really expect humanity to die out without an Armageddon-style battle between the armies of AI and Butlerian jihadists like myself? Does Page really think there wouldn’t be blood in the streets if a new digital species of alien posthumans were to emerge in the coming decades? Does Ord actually believe the metamorphosis of our species “into something beyond the humanity of today” wouldn’t trigger mass revolts from humans who oppose this? Does Ord imagine everyone around the world will convert to the transhumanist religion, and hence eagerly consent to this radical transformation? What if some people are allowed to remain human while others become posthuman: does he believe there won’t be worldwide protests organized by “legacy humans” who worry this new species will take over and, for reasons outlined above, force humanity out of existence? How does he think these posthumans should deal with such pesky humans fighting for their species in this apocalyptic showdown? Does he have anything resembling a plan—perhaps even a “concept of a plan”—for how to navigate such an impasse?

On these questions, the silence of such people is deafening, because I can hardly think of an issue of greater import. It is, if I may say, completely outrageous and morally unacceptable that they often simply shrug their shoulders and say, “Meh, we’ll figure it out when the time comes”—especially when they think the time will come in the near future, within the lifetimes of you and your children. Involuntary extinction would be a catastrophic violation of human rights likely producing enormous amounts of pain, misery, anguish, and terror. Yet this is the only plausible way posthumanity would replace us. If posthumanity were to arise in the near future, these questions would suddenly become urgent in the most profound sense. Perhaps, then, we should do everything we can to prevent posthumanity from arising.

***

If you spend any time listening to TESCREALists, you’ll hear many of them say that quite literally nothing matters more than avoiding human extinction. Sometimes they couch this in the language of existential risks. As Bostrom writes, “there is one kind of catastrophe that must be avoided at any cost: Existential risk.” There are many types of existential risks, as we’ve noted, the most obvious being human extinction. Hence, human extinction must be avoided at any cost. Yet many of these same individuals advocate for a future in which humanity will go extinct. Some explicitly hold the replacement view, according to which we should go extinct. So, how does this make sense?

The answer is that both terms in “human extinction” are ambiguous, and TESCREALists often exploit this ambiguity to make their views look more reasonable than they actually are. It gives the impression that they’re on Team Human, when in fact they’re on Team Posthuman, which enables them to launder their techno-utopian, often pro-extinctionist positions into public debates about AI and other issues to make it seem like they’re on the side of humanity. Understanding the ambiguities of “human extinction” is absolutely crucial for making sense of what TESCREALists are actually advocating.

We begin with the words “human” and “humanity.” Most people will intuitively define these as referring to our biological species, Homo sapiens. That’s how I’ve used them throughout the book, because I think this is clearly the best and most useful way to define them. Let’s call it the Narrow Definition. However, TESCREALists have their own idiosyncratic definition, according to which “human” and “humanity” refer both to our biological species and whatever posthuman descendants we might have. Let’s call this the Broad Definition.

If the Broad Definition looks oxymoronic to you, that’s because it is. On this account, posthumans count as humans, and posthumanity is an instance of humanity. That seems off because the prefix “post-” is supposed to indicate something distinct that comes after something else. It’s not possible for a football coach to give an angry postgame interview while the game is still going on, and it would be incoherent to say that little Johnny won’t be born for another 3 months but is also postnatal. Yet, this is the terminology that TESCREALists have given us, which I take to be indicative of a deeper problem of sloppiness in this literature.

Here are a few examples, so you can see for yourself. Nick Beckstead, an EA who’s worked for both Open Philanthropy and Bankman-Fried’s FTX Future Fun, wrote in a 2013 dissertation that cofounded the longtermist ideology: “By ‘humanity’ and ‘our descendants’ I don’t just mean the species homo sapiens [sic]. I mean to include any valuable successors we might have,” which he later describes as “sentient beings that matter” in a moral sense. MacAskill and his longtermist colleague Hilary Greaves stipulate that “we will use ‘human’ to refer both to Homo sapiens and to whatever descendants with at least comparable moral status we may have, even if those descendants are a different species, and even if they are non-biological.” In The Precipice, Ord writes that “if we somehow give rise to new kinds of moral agents in the future, the term ‘humanity’ in my definition should be taken to include them.” Along the same lines, Bostrom offers an even broader definition according to which “humanity” denotes “Earth-originating intelligent life.”

Now consider the radical implications of the Broad Definition. It implies that our species could literally die out next year without human extinction having occurred. As long as a new posthuman species takes our place, then “humanity” will persist; and if humanity persists, then human extinction won’t have happened.

So, when TESCREALists talk about the importance of avoiding “human extinction,” they aren’t talking about our species. They’re talking about posthumanity. The survival of our species, in the grand cosmic scheme of things, matters only insofar as this is necessary to produce posthumans. These are the beings who will bring about utopia—a posthuman paradise—and so once they arrive it really doesn’t matter if we stick around. Our job will have been completed, our eschatological duties discharged. Preventing human extinction doesn’t mean ensuring that our species survives into the long-term future.

Do not be fooled by this linguistic trickery! If you look carefully at the way TESCREALists use “human” and “humanity,” it becomes clear that they’re usually adopting the Broad Definition. They might say, in one breath, that “humanity” has existed for 300,000 years. This is how long Homo sapiens has been around, which might lead unsuspecting folks to assume they’re using the Narrow Definition. Yet, in the next breath, they say that “humanity” could survive for trillions of years to come by colonizing the universe. But we saw that colonization is physically impossible for biological humans, so they must actually be talking about digital posthumans. This reveals that they were using the Broad Definition all along.

Or, consider Verdon’s claim that e/acc strives to “develop interplanetary and interstellar transport so that humanity can spread beyond the Earth” to “increase human flourishing via pro-population growth policies and pro-economic growth policies” (italics added). Once again, he must be using “humanity” to mean posthumanity, since he himself admits that colonization is only possible for digital beings. People who don’t know about the Narrow and Broad definitions might be fooled into thinking that Verdon is somehow on Team Human, when he’s actually a digital eugenicist who wants ASI to replace humanity in the near future.

***

Both of these definitions refer to beings in the world. But there’s another way that TESCREALists sometimes use the word: as a shorthand for our values or type of consciousness. Consider the Musk-Page debate once again. We saw that Musk thinks it’s important for digital posthumans to care about the same things we care about, i.e., to share our values. But he also emphasized something else in his exchange with Page: “human consciousness,” he exclaimed, is “a precious flicker of light in the universe, and we should not let it be extinguished.” This was in response to Page’s question about why it matters “if machines someday surpass humans in intelligence, even consciousness.” Page saw Musk’s answer as “sentimental nonsense,” by which he apparently meant that it arises from a sentimental attachment to human qualities. That’s precisely why he called Musk a “speciesist.”

If “speciesist” means “a preference for the biological human species,” then Musk ain’t no speciesist. He’s a transhumanist who’s actively trying to merge our brains with AI while building an AI God in the form of Grok 10.0, or whatever. How does one make sense of this? The answer is that what Musk really cares about is our values and our consciousness. He doesn’t care about biological humans enduring, only that these two qualities persist into the posthuman era. Indeed, he seems to think that merging our brains with AI is one way to ensure these qualities persist: by transferring them to machines, they can continue and even proliferate in a world increasingly dominated by AI. His goal is to achieve “some kind of AI symbiosis, where you have an AI extension of yourself, like a tertiary layer above the limbic system and cortex.” And, as the TESCREAList Elise Bohan points out, the aim of this symbiosis “prevent humanity from being superseded.” Here, “humanity” can’t refer to our biological species, because a human-machine merger takes us in the direction of cyborg posthumanity. Rather, it must mean “our values” and “our consciousness.”

This usage of “humanity” can also lead to confusion. Recall that Musk shouted at Page: “Well, yes, I am pro-human. … I fucking like humanity, dude.” Yet Musk is not “pro-human” in the sense that most people would assume. He’s not “pro-human” in the way that I am, for example, since I actually want our biological species to persist. I don’t want our values or consciousness to endure in the minds of digital posthumans; I want them to endure in the central nervous systems of actual humans, because I actually do, in a literal sense, “fucking like humanity.” Once again, whenever TESCREALists talk about liking “humanity” or being “pro-human,” it behooves you to take a closer look to see which definition they’re using, or whether they’re employing such terms as shorthand for some abstract qualities they believe could be instantiated by some posthuman substrate.

***

Turning now to the second word in “human extinction,” I argued in my academic tome Human Extinction that there are actually six distinct types of “extinction” scenarios that we could undergo. Only two are relevant to our present discussion, and both specifically concern the Narrow Definition. That is to say, the Narrow Definition—but not the Broad Definition—makes “extinction” ambiguous between two very different types of extinction. These are:

• Terminal extinction: our species disappears entirely and forever.

• Final extinction: our species disappears entirely and forever without leaving behind any worthy successors to take our place.

For TESCREALists who accept the replacement view, the only type of extinction that “must be avoided at any cost” is final extinction. The reasoning parallels that above: what really matters is the creation of a cosmic utopia among the heavens. This utopia can only be built by posthumanity—some worthy successor. (For folks like Verdon, Sutton, and Page, virtually any ASI that takes our place would be a worthy successor, so long as it’s superintelligent and conscious. For people like Musk, Yudkowsky, Ord, and many other EAs, Rationalists, and longtermists, posthumanity must possess additional properties to count as worthy.) Since final extinction is the only extinction scenario that would permanently foreclose the realization of posthumanity, it’s the only type of extinction that really matters.

Let’s be clear about this: final extinction logically entails terminal extinction, but terminal extinction does not logically entail final extinction. For example, if posthumanity emerged next year, then our species could die out—undergo terminal extinction—without final extinction having happened. Those posthumans would take our place, which means final extinction would have been avoided. Pro-extinctionism, in fact, is nothing more than the claim that we should aim for exactly this scenario: terminal extinction without final extinction. All that means is that our species ought to kick the bucket after posthumanity makes its glorious debut.

However, if terminal extinction were to happen next week, it would almost certainly entail final extinction—not logically, but contingently. This is because we haven’t yet created or become posthumans who could take our place. Terminal extinction would bring about final extinction if posthumanity doesn’t yet exist, but not once posthumanity has arrived.

The point is that terminal extinction itself isn’t what matters to TESCREALists, precisely because our species’ mission isn’t to create utopia, but rather to create the sorts of beings who will create utopia. Echoing earlier remarks about the Broad Definition, avoiding terminal extinction matters only insofar as it’s necessary to avoid final extinction. Once terminal extinction would no longer entail final extinction, then what’s the point of humanity sticking around? Why not bring about terminal extinction if this would no longer threaten utopia? We would, after all, only be sucking up valuable resources that they could use for something else, and there might even be “moral” arguments for why posthumanity should get rid of us.

***

To recap, TESCREALists often use “humanity” to refer to us and our posthuman successors, or to some abstract qualities that we possess. On those occasions when they use the Narrow Definition, they’re almost always talking about final rather than terminal extinction. Please be wary of their language games! As I mentioned above, the way they talk about “human extinction” and related issues often gives the impression that they’re on Team Human, when in fact many of them are actually pro-extinctionists who hold the replacement view.

Yudkowsky provides a glaring example. His coauthored book, If Anyone Builds It, Everyone Dies, has received considerable attention. Even Bernie Sanders and Alexandria Ocasio-Cortez appear to have been influenced by it, as Sanders sat down with Yudkowsky, Kokotajlo, and others to discuss AI safety, which was recorded in a video that Sanders shared on social media. My guess is that Sanders falsely thinks that Yudkowsky is on Team Human, and probably also that he’s opposed to anyone building ASI ever—that is, after all, what the title of his book suggests.

In fact, Yudkowsky wants to build ASI asap, he just doesn’t think we’re ready yet. If the alignment problem were solved next week, then he’d suddenly switch sides and join the radical accelerationists in calling for AI capabilities research to be throttled full-speed ahead. I doubt most people reading his book understand this. As his organization MIRI wrote in 2024,

we remain committed to the idea that failing to build smarter-than-human systems someday would be tragic and would squander a great deal of potential. We want humanity to build those systems, but only once we know how to do so safely.

The word “potential” here refers to our “glorious transhumanist future” in which, as Yudkowsky argues, posthumanity might simply clear all humans off Earth and leave it as a nature sanctuary. Yudkowsky is neither anti-ASI nor pro-human, though his mendacious use of language fools people into thinking otherwise. Again, if he thought that we could build an ASI that would bring about his pro-extinctionist utopian fantasies next week, he would shout: “Best do it so!”3

Or, consider Bostrom. Despite his apparent defense of the coexistence view in the quote from earlier, he actually appears fairly aligned with pro-extinctionism. As the statistician and futurist Olle Häggström notes, “among thinkers with at least some successionist tendencies, one might even include Nick Bostrom,” where “successionist” is Häggström’s term for “pro-extinctionist.” Take Bostrom’s definition of “existential risk” as “one that threatens the premature extinction of Earth-originating intelligent life or the permanent and drastic destruction of its potential for desirable future development.” Häggström writes that

the reason he speaks of “Earth-originating intelligent life” here, rather than simply “humanity” [on the Narrow Definition], is that he is able to envision scenarios where humanity has been replaced by successors so worthy that the scenarios should count as sufficiently benign to not warrant the label existential catastrophe.

By utilizing the Broad Definition, Bostrom can embrace pro-extinctionist views while simultaneously claiming to care about “humanity” and oppose “human extinction.”

This gets at the reason I strongly prefer the Narrow Definition: it enables far more precision. It forces “extinction” to become ambiguous, which opens the door to nuance. I strongly oppose the terminal extinction of our species, which is what I mean by “human extinction.” In contrast, TESCREALists only care about this as a means to an end, not an end in itself. Our survival is just a means to the end of their bizarre notion of a cosmic utopia full of digital space brains. Once this utopia becomes possible to build by our posthuman progeny, then the survival of you and me—of the 8.2 billion people on Earth right now—becomes a matter of moral indifference or positively undesirable.

Let’s reflect for a moment on the dire implications of this picture. Accelerationists like Verdon offer us a single option for the future: the extinction of our species, which they hope will happen in the coming years. People in the AI safety camp more graciously (I say sarcastically) offer us two options: the extinction of our species or the extinction of our species. In the first case, “extinction” specifically refers to final extinction. This would happen if we were to build a misaligned ASI, which would by default turn around and destroy us. Worse, it would spread its misaligned values into the universe, resulting in an existential catastrophe. In the second case, “extinction” refers to terminal extinction without final extinction. This would happen if we build an aligned ASI that transforms (part of) humanity into posthumanity and then brings about a posthuman paradise among the stars. For all the many reasons specified above, this “best-case,” “utopian” outcome would almost certainly result in our species dying out.

In both cases, the outcome is identical for our species: terminal extinction. That’s why I argue vociferously that the entire TESCREAL framework is rotten to the core and must be abandoned. It’s why I think we must defeat the TESCREAL worldview if we want to permanently end the ASI race. No one in the TESCREAL movement is advocating for a good future for our species, not even the coexistence people, since a world ruled and run by posthumans would inevitably result in humanity being sidelined, marginalized, disempowered, and ultimately eliminated. Human extinction or human extinction: the choice is up to us, unless we demolish the worldview in which these are the only two options before us.

***

To summarize some key points of this essay, posthuman eschatology is the view that we should introduce a new posthuman species to the world, ideally in the near future. This leaves open a number of questions: should these posthumans be digital or biological? Should they arise as an extension of humanity or take the form of autonomous beings we create in the lab? Should the population of posthumanity include some of the same people alive today? Must these posthumans share our values and care about the same things we care about (whatever that means)? And once they’re here, should our species stick around or follow the dodo into an eternal oblivion? If we should die out, then how exactly should this happen: through some involuntary process (which would never happen), or via coercive means, perhaps even including outright omnicide (the murder of everyone)?

All TESCREALists accept a posthuman eschatology. It’s the common denominator underlying all these ideologies. TESCREALists just disagree about the details. Those disagreements can lead to heated, vicious debates, but these should be seen as nothing more than family disputes. Everyone agrees with the same basic vision of the future: give birth to posthumanity, spread beyond Earth, plunder the cosmos for its vast resources, and build a glorious multi-galactic civilization at the top of the Kardashev scale occupied by potentially fast numbers of immortal space brains living lives of “surpassing bliss and delight,” to quote Bostrom’s “Letter from Utopia.” This is the ultimate telos toward which we ought to be striving, which is precisely what inspired the founding of AI companies now engaged in a reckless race to create an algorithmic God.

As always:

Thanks for reading and I’ll see you on the other side!

Consider this hilarious video of a humanoid robot being dragged across the stage after awkwardly tripping over some stairs while dancing to Michael Jackson’s “Billie Jean.” Notice how the timing of the synths couldn’t be more perfect — as if to mock the syncopal robot at exactly the moment it passes out:

Meanwhile, someone captured a segment of an AI slop video about the WWE. There are, apparently, a bunch of YouTube channels that publish long-form videos narrated by AI (discussed around 3:00 here). The enslopification of the Internet accelerates! In this case, the AI begins to self-destruct like a washing machine with a brick in it after trying to pronounce “WWE.” I have a facetious ranking of funniness in terms of chuckles and guffaws, where 10 chuckles equals one guffaw. This one gets a full 2.5 guffaws — meaning I doubt you can watch it without laughing out loud:

I don’t think this is fake. I encountered something similar (but not as extreme) when watching a documentary that I thought was narrated by a human. When the narrator tried to pronounce an acronym full of consonants, it produced a bizarre sound that no human would ever make. (I can’t recall what the acronym was — maybe WWF, for World Wildlife fund?). On another occasion, I was watching a different documentary — again, foolishly thinking I was hearing a human — in which the narrator said, “And this cost the US dollar sign 300 million.” God damnit!

As Ronny Chieng put it during a graduation speech at Harvard: “Fuck AI.” I really wonder if, in a very profound sense, we aren’t witnessing an end to the human era — the era in which humans have been dominant. What a sad way for it to end, if that’s the case.

WaPo’s Nitasha Tiku on TESCREAL Doomerism

Taylor Lorenz has a new video in which she interviews journalist Nitasha Tiku of the Washington Post. They discuss how billionaires are moving mountains of money into social media campaigns to raise public awareness of the “existential risks” posed by ASI (artificial superintelligence). In some cases, the very same tech billionaires who invested in the AI companies are now supporting “anti-AI” activists and influencers. I have some thoughts on this interview:

The first part of the video provides a nice — albeit brief — overview of the TESCREAL ideologies. (The heart of this bundle is a techno-utopian vision in which our “posthuman” successors colonize the universe and build a sprawling multi-galactic civilization full of trillions and trillions of “digital people.”) Around 1:41, Tiku says:

So, AI safety is ... an evolving term. Like, this is sort of the one that the community has landed on. This is almost, like, a 25-year-old movement. So if you’ve heard about, like, the singularity, effective altruism, rationalist, transhumanist … it’s kind of at this intersection.

She later mentions longtermism, thus foregrounding 5 of the 7 ideologies in the TESCREAL acronym — the other two being cosmism and Extropianism. The TESCREAL acronym picks out a cultural movement built around libertarian transhumanism — i.e., Extropianism — that extends from the early 1990s up to the present. You can understand it as a more technophilic and utopian iteration of the Californian ideology. All of the major AI companies — DeepMind, OpenAI, Anthropic, and xAI (which is now SpaceXAI) — emerged directly out of the TESCREAL movement. The whole reason the CEOs and cofounders became interested in ASI in the first place was because the TESCREAL worldview identifies ASI as the ultimate means for realizing a posthuman paradise among the stars.

The field of “AI safety” was birthed by the TESCREAL movement. Why? Because some TESCREALists came to believe that there are two fundamentally different kinds of ASI: aligned and misaligned. If we were to build an ASI that’s “aligned” with the TESCREAL worldview, then we’d get the cosmic utopia of posthumanity mentioned above. But if it were misaligned, it will kill everyone on Earth and, astronomically worse, erase our “glorious transhumanist future” (quoting Eliezer Yudkowsky) before we have a chance to start drawing it.

Furthermore, these same TESCREALists — dubbed “doomers” — argue that figuring out how to build an aligned ASI is far more difficult than one would initially assume. So, AI safety was founded to find a solution to the “alignment problem.”

Tiku is absolutely right that the field emerged roughly 25 years ago, in the early 2000s, thanks largely to the work of Yudkowsky. Right now, many AI safety advocates believe that we’re very close to building an ASI, but nowhere close to solving the alignment problem. That means that total annihilation could happen in the very near future.

You can think of this in terms of a race between AI safety research and AI capabilities research. The finish line of the former is solving alignment. The finish line of the latter is actually building an ASI. If capabilities researchers get to their finish line before safety researchers reach theirs, then we’ll all be dead and the transhumanist future will be permanently canceled. When Yudkowsky says, “if anyone builds it, everyone dies,” he’s specifically talking about misaligned ASI — not aligned ASI.

***

At 4:37, Tiku says that AI safety

started with people who are not AI researchers. A lot of them are former philosophers from Oxford — self-taught people who were prescient … about the general direction that AI technology was moving forward. People now give them a lot of props for anticipating some of the concerns that we’ve seen around, like, ChatGPT and very human-like AI, and what this can do to human interaction — how quickly the technology is developing. [This] used to be called, like, concerns about the Singularity …

Here I would push back against Tiku’s characterization of early TESCREALists as being “prescient.” I think she gets things exactly reversed: it wasn’t that these people were prescient, it’s that they actively strove to fulfill their own prophecies — and they succeeded.

Yudkowsky himself played an integral role in launching the ASI race (as I discuss at length in my forthcoming book). He inspired people like Shane Legg, the cofounder of DeepMind, and introduced another DeepMind cofounder, Demis Hassabis, to Peter Thiel after the 2010 Singularity Summit. Thiel then invested in DeepMind, which is how it started.

Sam Altman was also inspired by Yudkowsky, explicitly saying that Yudkowsky is the reason that many people, including himself, became interested in trying to build god-like AI.

A New York Times profile describes Altman as a “product” of the EA and Rationalist communities. Anthropic CEO Dario Amodei was an EA-longtermist who hired a bunch of EA-longtermists to work as his company, including Holden Karnofsky, Joe Carlsmith, and Amanda Askell (who was married to the cofounder of EA).

Elon Musk has explicitly said that longtermism is “a close match for my philosophy.” Musk attended numerous conferences before the ASI race took off in which leading TESCREALists like Yudkowsky and Nick Bostrom were present.

And we haven’t even touched on all the money that helped start the ASI race from TESCREAL organizations like Open Philanthropy (now Coefficient Giving) and donors like Jaan Tallinn. (Tiku acknowledges some of this later on.)

It was because of the TESCREAL movement that ASI became widely recognized as the key to creating utopia. TESCREALists popularized the idea of ASI, inspired the founding of AI companies, helped those companies get tens of millions in funding, and so on. To say that early TESCREALists were prescient would be like saying that I was prescient at 8am this morning when I prophesied that I would soon be drinking coffee — and then got up and made myself a cup of Joe.

***

Around 9:10, Tiku says that EA was initially focused on “global poverty.” I don’t think this is right. As Mollie Gleiberman argues, EA was longtermist — that is, focused on longtermists causes like AI safety — from the very start. To support her claim, she compiles a catalogue of copious examples from leading EAs around the time EA was founded, in which these EAs say that “existential risk” mitigation is what really matters. Global poverty was, basically, a means of marketing the movement. Once people had joined EA, they would then be exposed to arguments for why mitigating existential risks is actually the best way to “do the most good” in the world. Gleiberman describes this as a “Trojan horse,” where the ultimate goal was to pursue what she describes as “transhumanitarianism,” a portmanteau of “transhumanism” and “humanitarianism.”

***

Tiku argues around 9:22 that EA then merged with the Rationalists and transhumanists. But, again, I don’t think this is right. EA arose out of the inchoate Rationalist movement, and it was influenced by transhumanism from the very start. Rationalist blogs like Overcoming Bias and LessWrong were the Petri dishes in which EA first grew, and the cofounder of EA, Toby Ord, had been publicly defending transhumanism with the most influential ideologue of the 21st century, Nick Bostrom, since at least 2006. Ord was a member of Bostrom’s Future of Humanity Institute, founded in 2005 to advance the TESCREAL worldview, two years before EA was officially founded in 2009.

This is part of Gebru and my point re: the TESCREAL thesis. These ideologies are all tangled up together. They are outgrowths of each other. They are different colors, reflected through a prism, of the very same techno-utopian posthuman eschatology. EA emerged from the libertarian transhumanist movement that constitutes the nucleus of TESCREALism.

***

At 11:08, Lorenz says:

You also kind of alluded to something, which is that it doesn’t seem like they’re as anti-AI as you’d think. A lot of them, it seems, are interested in this idea of an aligned AI, or at least this idea that like it’s not even necessarily that AI is bad, maybe, but it’s just that we don’t have an AI that’s working in the interest of humanity. Is that right?

Tiku responds:

Yeah, that’s such a good way of putting it. And this is one of the concepts that is I think hardest for people outside to grasp, because if you think that a technology is going to destroy the planet — destroy every human on Earth, cause this kind of existential risk — you would think that you would be doing everything in your power to stop it. But what you quickly realize is that there [are] many groups of people that are also … enamored with the potential for utopia from this technology. “Aligned AI” is one term and that means kind of aligning AI with human values. Again, a very very abstract concept, but basically it’s like [the ASI does] what the developers want. You have a morality system like ethics and you approach problems in a way humans might even though you don’t have any real world experiences or emotions of your own. Sometimes they used to call it “Friendly AI.”

I’m really glad that Lorenz and Tiku highlight this. As mentioned before, TESCREAL doomers are not anti-AI. Even the most hardcore doomer wants ASI as soon as possible. Specifically, they want ASI once the alignment problem has been solved. If a doomer like Yudkowsky were to become convinced that the alignment problem is solved, he would suddenly switch from screaming, “Shut it all down, now!!” (where “it” refers to capabilities research), to screaming this: “Pedal to the metal! Build ASI asap!”

Don’t be fooled by the rhetoric these people use. They like to “sell” themselves as anti-AI, which gets normies who hate AI on board because those normies think their views align. They don’t. Yudkowsky would say that delaying the creation of an aligned ASI is as wrong, morally speaking, as accelerating the creation of misaligned ASI.

Nor are most TESCREAL doomers pro-human. They advertise themselves as such, but scratch the surface of their views and you find a deeply anti-human eschatology. I’ve written about this at length before (see below), and will soon publish, on this newsletter, chapter 4 of my forthcoming book because it offers a comprehensive account of the intimate links between the TESCREAL movement and pro-extinctionism. Stay tuned for that!

In my view, pro-extinctionism is the most important feature of the TESCREAL worldview that journalists should be highlighting. Quite literally everyone in the TESCREAL movement is pushing for a future in which our species is sidelined, marginalized, disempowered, and ultimately eliminated. This outcome is an inevitable consequence of the posthuman eschatology that they embrace and are actively trying to realize through the creation of superintelligent artificial deities. If this isn’t worth shouting about, then what is? Altman himself holds this view, as I show in a recent Truthdig article.

***

I don’t have many comments on the rest of the video. The discussion is very good, and Tiku foregrounds a number of important issues, such as the incestuous funding channels linking hundreds or thousands of TESCREAL organizations (many focused on AI safety) with super-wealthy TESCREAL donors (such as Jaan Tallinn, Dustin Moskovitz, and Vitalik Buterin — recipient of a Thiel Fellowship).

Some of these donors, including Tallinn and Moskovitz, helped launch the ASI race that they now worry could destroy humanity and, along with us, the “vast and glorious” posthuman future among the heavens. There is a great irony to this: no single group has done more to launch, sustain, and accelerate the ASI race than the TESCREAL doomers!

Consequently, these same people are now pouring money into social media campaigns aimed at halting the ASI race. The Future of Life Institute, for example — which was cofounded by Tallinn and received $10 million from Musk, and another $500 million from Buterin — has offered grants of between $100k and $500k for projects aiming “to educate and engage stakeholder groups, as well as the general public, in the movement for safe, secure and beneficial AI.” A central thrust of the interview with Tiku was to point out that you may very well have watched a video on YouTube warning about the existential risks of ASI and not realized that, in fact, the organization behind it is just two steps removed from the AI companies themselves, the intermediate link being the tech billionaires who funded both.

Vox Publishes an Article on Pro-Extinctionism

Vox — which, recall, has a deal with OpenAI — recently published an article titled “The People Who Actually Want AI to Replace Humanity.” It explores the ideology of “successionism” (i.e., replace humans with AI), and I have to say that the content is strikingly similar to my many articles and podcast interviews on pro-extinctionism. The author, Sigal Samuel, discusses transhumanism, Ray Kurzweil (the singularitarian), Effective Altruism, cosmism, longtermism, Beff Jezos and e/acc, as well as Daniel Faggella’s “worthy successor” eschatology — all topics I’ve covered at length in much the same way she does in her article. She also highlights the links between “successionism” and eugenics.

Sound familiar? Much of the article is little more than a summary of my work, though she doesn’t cite me or mention the TESCREAL acronym — even though that’s what she’s talking about. See for yourself.

It’s good that more people are talking about this issue, and Samuel appears to be generally critical of the TESCREAL ideologies. Aside from not citing me (which, you know, whatever), my biggest criticisms of her article pertain to what it leaves out. Although she mentions Toby Ord at one point, she doesn’t explicitly say that longtermism is a pro-extinctionist ideology, too — indeed, it’s nearly identical with Faggella’s “worthy successor” idea, as shown in my last newsletter article.

Doing so would have provided an opportunity to distinguish between two types of pro-extinctionism: first, the view that humanity should be replaced by autonomous AIs (like ChatGPT), and second, the view that humanity should be replaced by AIs that we become. Longtermists like Ord want to replace humanity with digital posthumans (which I satirically call “digital space brains,” after my friend Keira Havens suggested “space brains”), where these digital posthumans are radically transformed versions of us. In contrast, people like Verdon and Faggella appear open to autonomous, distinct, completely separate-from-us AIs usurping humanity.

When one carefully examines the space of possibilities here, it becomes immediately obvious that the entire TESCREAL worldview is pro-extinctionist, though certain versions of pro-extinctionism are more flagrantly horrific than others. All TESCREALists are rooting for Team Posthuman, meaning that, at the very least, they want to create a new posthuman species to run the show. This is dangerous because, I’ve argued before, if anyone becomes posthuman, everyone (else) dies out. If posthumanity arises, our days are numbered.

When I started this newsletter back in August 2025, I promised that we’d be covering a range of topics that (a) almost no one else is talking about, and (b) will become increasingly discussed in the coming years. For example, I have argued that we will see more and more debates spill into the public arena in which participants (e.g., Silicon Valley dwellers) are arguing not about whether pro-extinctionism is bad, but about which type of pro-extinctionism is best.

I have not been “ahead of the curve” on most things, but I think my (and Timnit’s) discussion of TESCREALism and my subsequent emphasis on Silicon Valley pro-extinctionism have been. The fact that Vox is now covering the very same ideas that I’ve been exploring for years is confirmation of this!

Next article (Tuesday): chapter 4 of my book, which is almost 20,000 words. Hopefully, it will provide some useful clarity on the different types of pro-extinctionism and the ways it’s entangled with the race to build god-like AI super-beings.

As always:

Thanks for reading and I’ll see you on the other side!

This article is a bit more technical than usual. I’ll publish some AI bloopers in the next post to make up for it. :-) Thanks for reading, friends!

Acknowledgments: Remmelt Ellen and Kyle Evanoff provided insightful feedback on this piece. They do not necessarily endorse my claims or conclusions.

A Self-Defeating Futurology

You may recall Daniel Faggella from a previous newsletter series. He’s a pro-extinctionist who argues that we should replace humanity with a “worthy successor.” He defines a “worthy successor” as “a posthuman intelligence so capable and morally valuable that you would gladly prefer that it (not humanity) control the government, and determine the future path of life itself.”

The point is to create ever-“higher” lifeforms that expand outward and colonize every corner of the cosmos. On this view, individuals like you and me are nothing more than a means to an end. All current and future beings are fungible and disposable. They are mere “torches” that carry what really matters: life or consciousness, which Faggella dubs the “flame.” The goal is for humanity to create its successor in the form of AGI, which will immediately create its own successor in the form of AGI+, which will immediately create its own successor in the form AGI++, and so on forever — an endless, aimless process that Faggella sees as the ultimate eschatological destiny of Earth-originating “intelligence.”

This is a deeply nihilistic view, which overlooks a number of major problems. I outlined some such problems here, but Faggella has yet to directly respond to them (despite publishing this). A problem I didn’t mention in my previous article is that Faggella’s worldview is built on a contradiction:

1. He claims that “a safe (or at least not overtly and unnecessarily dangerous) expansion of potentia is the sole moral imperative.” The only goal is to maximize “potentia,” defined as what enables life to persist.

2. At the same time, he argues that “(in the future) the highest locus of moral value and volition should be alien, inhuman.” He doesn’t want human values to get locked in and determine the future of life. He wants future beings to discover their own radically different set of values.

So, Faggella wants future beings not to be constrained by human values, yet demands that these beings forever embody his value of “expanding potentia.” As I put it in a social media post written for him:

Which is it? Do you want your imperative “maximize potentia” to constrain all future lifeforms, or do you want those lifeforms to discover their own values, which may lead them to antithetical imperatives? If future lifeforms don’t maximize potentia, you’d consider this a great cosmic tragedy — precisely because you’re trying to impose a human value (potentia maximization) on beings you claim should reject human values. That’s why your view is self-defeating: if future lifeforms were to embrace it, they’d reject it.

Faggella’s View Is Just Utilitarianism

Another way to understand Faggella’s eschatological ethics is this: it’s just utilitarianism with a different value theory. Let me explain:

If you were to dissect utilitarian theory, you’d find that it has two components: one deontic and the other axiological. “Deontic” refers to what you ought to do, whereas “axiological” refers to what is good or bad, better or worse. The axiological component says that one state of affairs containing more total “value” is better than another containing less. The deontic component says that you ought to bring about the state with more total value — hence, failing to do this would be morally wrong.

To illustrate: if lying to someone would result in the best possible outcome all things considered (axiological claim), then you should lie (deontic claim). But if telling the truth results in the best outcome, then you should tell the truth. This is what your philosophy professor meant when she said, “Utilitarianism puts the good before the right” — it builds what’s right on what’s good or overall best; it bases the deontic on the axiological.

But what is “good”? What is the “value” that utilitarians say we must maximize? This is where value theory enters the picture: we know what we ought to do — maximize value/the good — but we don’t know what value is without a theory of value to tell us.

The most well-known value theory defended by utilitarians is hedonism, which says that value = pleasurable experiences or happiness. Hence, you can replace “value” in the sentence, “Our sole moral obligation is to maximize value,” with “pleasure” to get: “Our sole moral obligation is to maximize pleasure.” The value theory of hedonism fills in the blank!

What Faggella does is introduce a new value theory, which is even less plausible than hedonism. According to his account, value = potentia (well, it’s really consciousness or life, but potentia is what enables those things to persist and evolve). Hence, “Our sole moral obligation is to maximize potentia.” This is just utilitarianism with a different theory of value.

Utilitarianism Is Just Capitalism

One of the most salient problems with traditional utilitarianism is that it doesn’t care about people as ends in themselves. In fact, it doesn’t care about anything for its own sake except for “pleasure” or “happiness” (if one is a hedonist).

This is why it yields the Repugnant Conclusion, whereby a world full of 1 trillion people with a happiness level of only 1 is better than a world full of 1 million people with a happiness level of 900,000 (if you can even make sense of quantified “happiness levels” — I can’t, but this is how they proceed).

That’s because 1 trillion x 1 units of value = 1 trillion, whereas 1 million x 900,000 = 900 billion. One trillion is more total happiness/value than 900 billion, and hence utilitarians say that we should realize the first world full of basically miserable people over the second world full of wonderful lives if those are our two choices. Utilitarians like William MacAskill, Toby Ord, and Peter Singer don’t see the Repugnant Conclusion as a big deal. They make that claim here.

This is nothing more than capitalism applied to ethics. It’s all about the number-crunching. Morality is a calculation to be solved. Utilitarianism reduces ethics to a branch of economics. People are just replaceable containers of “value,” as they have no value in themselves.

Some bizarre things (aside from the Repugnant Conclusion) follow from this. For example, there is no intrinsic difference between the death of someone and the non-birth of someone else, all else being equal. That is to say, if an unborn possible person would have a happiness level of 100 if they were to exist, then by not existing, the universe is deprived of 100 units of value. We should, therefore, create that person to add 100 units of value to the world. Similarly, if an existing person has a happiness level of 100 but then suddenly dies, the universe is deprived of 100 units of value. In both cases, the outcome is identical: the universe loses 100 units. As Jonathan Bennett once mockingly wrote: “As well as deploring the situation where a person lacks happiness, these philosophers also deplore the situation where some happiness lacks a person.”

You could say something similar about profit in a capitalist system. Not losing $100 million is good. But if there’s another $100 million to gain, you must acquire it. On both the capitalist and utilitarian views, could exist implies should exist: an extra $100 million in profit should be acquired if it could be acquired, and a merely possible person with net-positive value should exist if they could.

These parallels aren’t coincidental. Utilitarianism and industrial capitalism both emerged in the latter 19th century in the very same place: the UK. They are both expressions of an impersonal, quantitative, dehumanizing conception of people as means to an end, and the end as something to be maximized.

Longtermism: Techno-Capitalism Projected into the Stars

I say all of this to introduce the second topic of the article: longtermism, an ideology built upon utilitarian ethics. In particular, it’s founded on the axiological component of utilitarianism, which states that one outcome is better than another if it contains more total value.1

What longtermism adds to utilitarianism is a more expansive, cosmological perspective: cosmologists tell us that the universe is huge and will remain habitable for another 10^100 years or so — a 1 followed by 100 zeros, a much greater amount of time than the universe has so far existed. That means the future population of beings, if we spread throughout the universe, could be enormous. Assuming these beings will bring at least 1 unit of net value into the world during their lifetimes, we therefore have an obligation to realize as many of these future beings as possible — to flood the universe with our descendants until every single nook and cranny, every inch of cosmic real estate, is crammed with people.

Why does it matter that the universe contains the maximum possible amount of value between now and the heat? Well, it just does, because longtermists — and utilitarians — assume that the only appropriate response to value is to maximize it: more, more, more, with no upper limit.

But this ignores a wide range of alternative responses to value such as cherishing, treasuring, revering, caring for, loving, protecting, preserving, sustaining, appreciating, and so on. My approach to ethics emphasizes these alternatives rather than maximization. When I stumble upon a meadow in the woods, I stand there and appreciate it. I take it in and savor the beauty. I don’t think, “This meadow has value. Therefore, the entire world — indeed, the whole universe — should be covered in meadows, because more is always better.” I’m not a capitalist when it comes to ethics.

This is what longtermists think when they look up at the firmament:

Humans Must Be Digitized

To maximize value, we must therefore colonize space. And given that energy sources like stars are burning through their hydrogen reservoirs as we speak, we must do this asap. That’s the central point of Nick Bostrom’s paper “Astronomical Waste.”2 Every second of delay entails some irreversible loss of our “cosmic endowment.” Time is of the essence! You can see why this view appeals to billionaires like Elon Musk.

But how can we colonize space? There’s almost certainly no way for biological humans to do this. Space is incredibly hostile to squishy meat-sacks like us — even colonizing our planetary neighbor, Mars, is likely impossible for human beings, given that Mars has almost no atmosphere, lacks a magnetic field, has much weaker gravity than Earth, and is covered in dirt that’s literally poisonous. The moon is probably the furthest from Earth we’ll ever get.

Hence, humanity will need to become or create digital beings — what I call digital space brains — to colonize the universe in our place. It’s these “posthumans” who will venture beyond our planetary oasis and plunder the cosmos, not us.

More Total Value

As it happens, this has a salutary consequence from a utilitarian point of view: imagine somehow colonizing exoplanets as biological humans. After 20,000 years of travel, we arrive in our spaceships and somehow magically terraform the exoplanet to make it Earth-like and habitable. How many biological humans could then exist on its surface? Maybe 10 billion? Maybe 20 billion?

Now consider an alternative: we arrive as digital space brains and use some magical technology to transform the planet into a giant computer made of computronium (matter optimized to perform computations). We then run a huge virtual reality simulation on this planet-sized computer for digital space brains to reside in. How many of these space brains could exist within the simulation? Longtermists think there could be way more than 20 billion.3 That’s because you can cram more digital people into a given unit of spacetime than you can biological humans. Hence, to maximize value, all future people will need to be digital rather than biological.4

Posthuman Value-Containers

There’s another utilitarian advantage to digitizing “intelligence.” We noted above that, for utilitarians, people have zero intrinsic value. We are the “containers” of value, and the more people/containers there are, the more total value there could be. (That’s why the future population should be as large as possible.)

But here’s the catch: humans can only contain so much value. We’re constrained by our “biological limitations,” and natural selection didn’t optimize us to generate unlimited amounts of goodness. The human value-container is only so deep.

But we could — supposedly — engineer digital posthumans who contain far more value, per individual, than biological humans. We could design them to be much deeper value containers, as illustrated below5:

Do you see where this is going? The ethical foundations of longtermism — utilitarianism — are inherently pro-extinctionist, just like Faggella’s view. To maximize value, we should replace humans with posthumans — and we should do this as soon as possible.

You might think these posthumans would keep some biological humans around, but that wouldn’t be the case: biological humans take up space and use up valuable resources in “suboptimal” ways. There’s no reason for posthumanity to keep us around, and every reason to nudge us out of existence. If the goal is to maximize value in the universe, then our days are numbered.

This is hardly any different from Faggella’s pro-extinctionism. Both aim to maximize something, and in service of maximization they imply that our species should be replaced by a “worthy successor.” The only notable difference is the underlying value theory, though Faggella’s notion of “potentia” also describes a key feature of the longtermist worldview, as maximizing value requires future beings to find new and better ways to ensure the continued survival of life and consciousness.

Six Problems that Longtermists Ignore

But longtermism encounters a number of major problems — nearly all of which its advocates have tended to ignore, dismiss, or wave away. What follows is not an exhaustive list, but rather a collection that I find particularly interesting:

1. Artificial Consciousness?

The entire longtermist worldview is predicated on the possibility of artificial consciousness. Why? Because value seems to depend on consciousness: if digital space brains are “intelligent” but not “conscious,” handing over the universe to them would be like bequeathing it to rocks. Since only digital beings will be able to colonize space, it really matters that artificial consciousness is possible. Literally everything depends on it.

This problem divides into two parts, one metaphysical and the other epistemological.

1.1 The Metaphysics of Mind

Can artificial systems be conscious? Most longtermists assume that’s the case because they think the brain is just a computer — a position called “computational functionalism.” But the “brain as computer” metaphor is just that — a metaphor. Historically, people have metaphorized the brain in terms of whatever the most impressive technology happened to be at the time. After discovering that electricity stimulates nerves, the brain became a telegraph network. When neurons and synapses were identified, it was taken to be a telephone switchboard.6 In the 1940s, cybernetics pushed information feedback loops into the spotlight, and once digital computers arrived, people became convinced that our central nervous system works the same way: neuronal networks are the “wetware” that runs mental algorithms, the software of the brain. In 20 years, it’s entirely possible that the computer metaphor will be replaced by something else.

If the brain isn’t a computer, then artificial consciousness might not be possible. Mind-uploading almost certainly wouldn’t be — i.e., simulating one’s brain on a computer wouldn’t give rise to subjective experiences any more than simulating a hurricane would soak your laptop in rain. There are plenty of reasons for thinking the brain isn’t a computer and that artificial systems can’t be conscious. The neuroscientist Anil Seth provides a useful overview here:

Also, reflect for a moment on what it would take to simulate someone’s brain in silico. Our brains are the most complex objects in the known universe. They consist of ~86 billion neurons and 100 trillion synapses. Waves of electrical activity constantly sweep over them like stormy weather. Signals are spatiotemporally summated in complex ways by interconnected cells. Every second, quivering branches of neurons are rewiring themselves while spurts of blood course through tangles of axons, dendrites, and billions of support cells called glia, whose function we don’t yet fully understand. Your brain is not a static blob, but a constantly wriggling dynamic system squirming about in your skull. Meanwhile, intracellular metabolic processes are producing waste products that accumulate throughout the day and are flushed out at night, when your brain oscillates through different stages of sleep that correlate with distinct wave frequencies.

All of these phenomena would have to be simulated by a computer. The metabolic processes, the brain waves of sleep stages, the quivering axons and dendrites, and perhaps even quantum phenomena that might turn out to be important for cognition and consciousness. Is that technologically feasible? Who knows — maybe not. Given that, for longtermists, everything rests on consciousness being realizable by artificial systems, you’d think they would be obsessing over such questions. But they aren’t.7

1.2 “Testing” for Consciousness

This brings us to the epistemological problem: assuming that artificial consciousness is possible (which I doubt), how could we possibly know that a particular artificial system is actually conscious? What objective “test” could confirm this? (Longtermists would want a test that’s extremely reliable — to the point of near certainty — given how much depends on artificial consciousness.)

If an AI were to say, “Yes, I’m conscious,” could we intersubjectively verify this? AIs, including uploaded minds, could be highly functional zombies: beings with complex outward behaviors but nothing it is like to be them on the inside. Indeed, current LLMs are like this: few people think they’re actually conscious, yet sometimes they’ll tell users that they are.

Perhaps a human being could incrementally upload their mind by replacing one neuron at a time with a silicon chip (a “functional isomorph”) and report back on how the process is affecting their conscious experiences in realtime.8 But what if this person loses consciousness without noticing? What if the functional isomorphs produce the same linguistic output — “I’m conscious” — while the light dims? (As I’ve noted before, there is a closed loop of causes and effects from stimuli impinging upon the eardrum all the way to a vocal response involving the vocal folds, tongue, and lungs. It could be theoretically possible to reproduce this causal loop without consciousness being present.) The philosopher Susan Schneider has proposed a few “tests” to determine if artificial/digital systems are conscious, but in my view they are all inadequate.

This should worry longtermists a lot, which makes it surprising that so few of them talk about it. Imagine that some longtermists create AIs that they mistakenly believe are conscious. They launch these beings into space, where they proliferate in every direction, filling every corner of our future light cone. They take over the universe. But it turns out this founding population wasn’t in fact conscious, and hence our “cosmic endowment” of resources is gobbled up by beings that can’t realize “value.” Whoops! This is why having a robust objective “test” for artificial consciousness — assuming artificial systems can be conscious — is absolutely paramount for longtermism.9

A huge question mark hangs over the entire longtermist project because of these two conundrums.

2. There Won’t Be Digital “People” in the Future

We saw that longtermists argue that there should exist as many digital space brains in the future as possible — an idea I’ve sloganized as, “Won’t someone think of all the digital unborn!” That’s why they’re obsessed with trying to estimate how large the future population could be. Bostrom says there could be 10^38 digital space brains in the Virgo Supercluster, while Toby Newberry claims there could be 10^45 in the Milky Way galaxy. Later, Bostrom estimated 10^58 within the accessible universe as a whole. These estimates are used to motivate their claim that the far future is overwhelmingly important, given that “could exist implies should exist.” It’s not that future people matter more — it’s just a numbers game: there could be far more of them than the total number of humans who currently exist.

The problem is that such calculations assume there will be individual people who can be counted — that future people will be individuatable. But why think this will be the case? If future minds take the form of pure information, of computer algorithms, then they could — and no doubt would — share memories, personality traits, experiences, preferences, beliefs, desires, and so on, with the metaphorical click of a button.10 The boundaries between “individuals” would become porous, and the world would become a digital soup of ontological chaos. Perhaps these digital minds would aggregate into some kind of amorphous wriggling hive mind, comprised of jostling mental fragments constantly recombining with each other in a phantasmagoria of incessant change. Who’s to say?

However things turn out, we can be sure that the digital world will be profoundly different than the world we live in today. It may very well mark a completely new post-individual era in which there are no countable “people” at all. And if there are no countable people, then the estimates that longtermists rely on to motivate their view are completely meaningless.

3. Digital Prozac

Even if digital space brains were to somehow maintain their individuality, longtermists still have to explain why any of these space brains would enjoy living in giant computer simulations in the first place. What would give their lives meaning and purpose? Why on Earth would digital posthumans be happy in vast simulated realities crammed with trillions and trillions of other “disembodied” minds?

I have never once seen any longtermists address this question. Perhaps there would be digital pharmacies to provide digital people with digital Prozac to get through? Or perhaps future space brains will see it as their “moral duty” to somehow force themselves to be “happy” for the sake of maximizing value — because they understand they’re means to an end, according to the capitalist ethics of utilitarianism.

Imagine for a moment what life within a giant simulation might be like. The laws of physics as we know them need not apply: one could design virtual reality worlds in which there’s no gravity, the fundamental particles and forces are different, and F ≠ ma. The possibilities are terrifyingly endless. Would everyone have their own virtual reality world? Would a bunch of digital space brains share a world? If so, who would get to design it? What if it were hacked and everything suddenly changed? What if one’s friends were permanently deleted? Who would maintain law and order? How would people prevent attacks from outside their little Matrix, originating in the physical (rather than digital) universe?

Such worries bring up two additional issues:

4. Immortality, Bunkers, and Eternal Torture

We start with immortality. Digital space brains would be immortal, because software is immortal. As long as the hardware exists to realize the software, digital posthumans could live until the heat death of the universe.

But consider the implications: if you were a digital space brain and were thinking rationally, you would do literally nothing else every day — for trillions of years — but obsess over your survival. Why? The standard “expected value” definition of risk is “probability multiplied by the consequences.” This means that a negative outcome with a tiny probability of occurring could still constitute an enormous risk if the consequences are sufficiently high. If you’re functionally immortal, then the consequences of dying would be astronomically huge, because you’d be losing trillions and trillions of future years of life.

To illustrate, if I were to get hit by a bus tomorrow, I’d lose perhaps 4 decades of my natural life. But if I could live for the next 1 billion years, then getting hit by a bus would be orders of magnitude worse. Consequently, the risk of getting hit by a bus — even if it remains extremely improbable on any given day — would skyrocket. Insofar as I’m rational, I’d never cross a road again. Indeed, since there are many possible but improbable ways of dying, I should just build a bunker and never come out.

The same logic applies to functionally immortal digital posthumans. Living in their computer simulations, they should take every imaginable precaution to ensure their continued survival, given that bad outcomes with vanishingly small probabilities would pose massive risks. They should fortify the planetary computers on which their simulations run with all sorts of exotic weaponry just in case an alien species of posthumans outside the Matrix were to attack. Individually — if they have individuality — they should constantly back up their digital minds in case something happens, which would surely foment fights between digital people over limited server space. Every moment spent not ensuring one’s continued survival would be irrational, given the enormous stakes of dying. This brings us to the second issue:

5. The Anarchic Cosmopolitical Arena: Constant Catastrophic Wars

Why do longtermists assume that space won’t become a theater of constant, catastrophic wars? Due to the vastitude of the cosmos and the hard limit of the speed of light, there’s no possibility of a well-coordinated cosmic Leviathan to provide security. Mutually assured destruction (MAD) won’t work either, and populations of digital posthumans would gradually evolve radically different capacities, traditions, abilities, languages, belief systems, technologies, scientific theories, etc., thus making mutual trust impossible.

What, then, is left to keep the peace? Each solar civilization, or archipelago of civilizations, will find itself in a Hobbesian trap in which preemptive violence becomes the only rational option to guarantee its survival. Spirals of militarization (the “security dilemma”) will greatly exacerbate the situation, as defensive weapons are misinterpreted as offensive threats.

I’ve written about this at length elsewhere, so won’t elaborate here. Suffice it to say that the anarchic nature of the cosmopolitical realm would be primed for constant conflict, and exotic space weapons could potentially obliterate neighboring civilizations at the speed of light. Of course there will be devastating wars all the time — no matter how irenic our posthuman progeny are, they won’t be able to escape the death traps of game theory.

6. Longtermists Face Pascal’s Wager, Not Pascal’s Mugging

The problem of immortality, illustrated above, is an example of what Eliezer Yudkowsky and Bostrom call “Pascal’s mugging.” This occurs when tiny probabilities are paired with enormous stakes. Let’s say that there could be 10^58 digital space brains between now and the heat death, as Bostrom argues. That means the cost of undergoing final extinction (see below) is HUGE. It follows that even if there’s only a teeny-tiny probability of extinction happening, the “expected value” of preventing this scenario will still be massive. We should, therefore, redirect nearly all of our finite resources away from non-extinction risks in favor of projects aimed at ensuring the existence of future digital space brains.

Pascal’s mugging points to extreme cases where expected value theory seems to break down. When tiny probabilities and enormous stakes are involved, expected value theory recommends fanatical options.

The problem is that if you ask a cosmologist whether the heat death of the universe would mark a definite end to information processing, life, intelligence, etc., they will answer “no.” Although speculative, it’s possible that future digital people could find a way to escape the heat death — e.g., by tunneling into a neighboring universe, creating their own universe from scratch, or utilizing some strange futuristic physics to intervene and prevent the heat death from occurring.

Consequently, there could be an infinite number of future digital people. Perhaps our distant descendants tunnel into a neighboring universe, and then iterate this process each time their new cosmic abode collapses into thermodynamic equilibrium. By endlessly iterating the process, they could exist forever, thus generating infinite amounts of value. Improbable? Yes. Impossible? No: there’s a nonzero chance of this happening.

Longtermists should thus use this infinitely higher estimate of future people in their expected value calculations. They want to know what the stakes are, and it appears the stakes could be infinite.

Doing this, however, replaces Pascal’s mugging with Pascal’s wager. The former involves huge stakes paired with tiny probabilities, whereas the latter involves infinite stakes. The conclusion is that the only thing we ever should spend our money and resources on is preventing (final) extinction. Final-extinction risk mitigation wins every time without exception. This is a direct consequence of longtermism plus the cosmological fact that there could be infinite people in the future.

If longtermists were intellectually serious, they would acknowledge that their view yields the most extreme form of “fanaticism” imaginable: literally nothing matters except continuing to survive so that infinite amounts of value can be realized in the future. It’s Pascal’s wager applied to cosmic rather than personal eschatology.

Conclusion

Longtermism proposes a very bizarre futurology. This futurology is based on a number of assumptions, almost none of which are seriously examined by longtermists. The point of this article wasn’t primarily to critique the longtermist ideology, but to highlight a small handful of its underlying assumptions.11 It behooves longtermists to address such issues, but given the cult-like nature of the longtermist community, I doubt they will.

The article also aimed to show how two supposedly different ideologies — Faggella’s “worthy successor” view and longtermism — are nearly identical. Faggella is just a longtermist with an idiosyncratic theory of value, while longtermism has pro-extinctionist implications no less than Faggella’s eschatology.

As I’ve argued, many of these TESCREAL or TESCREAL-adjacent views are far more similar than they are different, including longtermism, effective accelerationism, cosmism, singularitarianism, and the pro-extinctionism of people like Larry Page, Beff Jezos, and Eliezer Yudkowsky. There are differences, but they are minor. All must be stopped insofar as we want a good, flourishing future for our species.

As always:

Thanks for reading and I’ll see you on the other side!

Since the early 2000s, a major life goal of mine had been to land a professorship in philosophy. Academia has always been the community in which I felt most at home. However, after leaving a postdoctoral position last December, I struggle to imagine myself returning. There are many reasons for this, one of which is that AI is destroying the university system in realtime. As a college senior at Stanford, Theo Baker, writes in the New York Times:

Cheating has become omnipresent. I don’t know a single person who hasn’t used A.I. to get through some assignment in college, yet the school was at first slow to realize how widespread this would become. …

About halfway through freshman year, some coding classes started requiring students to sign a declaration — “I did not utilize ChatGPT” — to submit each assignment. During the first term these attestations began to appear, I watched a freshman I knew sign the declaration that he’d done his homework without A.I. as ChatGPT was still open in the next window — while on the deck of a yacht party financed by venture capitalists.

Although students are using AI to cheat their way through the university system, most seem to simultaneously hate AI because of its predicted impact on the job market. Just the other day, Microsoft AI CEO Mustafa Suleyman, who previously cofounded DeepMind, claimed that AI will automate “all white-collar work” within just 18 months.

Given such prognostications, there’s a certain logic to students choosing to cheat: if AI will leave them jobless, then why not? What’s the point of actually learning if the knowledge one acquires is worthless post-graduation? One could, of course, argue that knowledge is valuable in itself, independent of its usefulness. But we live in a capitalist system with few social safety nets (at least in the US), so an instrumentalist approach to education makes sense within this system. (Hence, the entire system needs to be reformed!) I thus sympathize with students who cheat, even though I don’t approve. In fact, surveys show that 60% of teachers are now using AI to grade papers, meaning that there’s no human in the loop at all. Students aren’t doing anything different from the people who teach them.

Students Boo AI

The disdain that many young people feel about AI being shoved down their throats was on full display during several recent graduation speeches. At the University of Arizona, the billionaire and former Google CEO, Eric Schmidt, told a large audience of students that AI “will touch every profession, every classroom, every hospital, every laboratory, every person and every relationship you have.” Amid jeers and boos, he added: “I know what many of you are feeling about that. I can hear you. There is a fear,” though he offered no optimistic message about how to alleviate the fear. Watch the deplorable clip here:

(Schmitt, by the way, is the guy who said that since we aren’t going to reach our climate goals anyway, we should plow ahead with AI in hopes of building a superintelligence that will magically solve the climate crisis. As Mashable put it, he “argued that current climate goals should be abandoned in favor of a no-bars-held approach to AI investment.”)

Over at Middle Tennessee State University, the CEO of Big Machine Records, Scott Borchetta, brought up AI and was heckled. In response, the asshole replied: “Deal with it. … You can hear me now, or you can pay me later,” with a smirk on his face.

Meanwhile, another university “used AI to announce graduates’ names and missed hundreds of names.” Jeers and boos are, once again, loudly heard in the background. Whoops.

AI Slop Is Polluting the Academic Literature

As noted earlier, it’s not just students who are using AI. A team of researchers recently combed over papers and preprints on websites like Social Science Research Network (SSRN), arXiv, bioRxiv, and PubMed Central. They found a whopping 146,932 fake citations, the result of AI hallucinations. This is appalling: academics are relying on AI to assemble their bibliographies, and even help research and write their papers.

To combat the tsunami of AI slop now polluting the academic literature, arXiv recently announced that “if a submission contained incontrovertible evidence that the authors did not check the results of LLM generation, this means we can’t trust anything in the paper.” Consequently, the paper will be removed and the authors banned for an entire year. All subsequent submissions will need to “be accepted at a reputable peer-reviewed venue” before being uploaded to arXiv. Examples of such evidence, the announcement adds, are hallucinated references and “meta-comments from the LLM (‘here is a 200 word summary; would you like me to make any changes?’; ‘the data in this table is illustrative, fill it in with the real numbers from your experiments’).”

Some academics apparently aren’t happy about this, which is astonishing:

I’ve said ad nauseam that AI is reliably unreliable. Do not ever trust it for anything factual; for anything that requires accuracy. AI lies with the same ease of sociopaths like Sam Altman (lol). In fact, a recent audit of doctors in Ontario, Canada,

found that AI agents tasked with turning doctor/patient conversations into structured notes routinely hallucinated false treatments, replaced drug names with entirely different drugs, and missed crucial information.

Imagine getting a prescription from your doctor for Adderall, but AI misinterprets it as Inderal, a beta-blocker that treats hypertension, tumors, and angina — or vice versa. The rapid integration of AI into the infrastructure of our society and lives is extremely worrisome given its inability to spit out accurate information in a reliable way. The further along this integration process goes, the more difficult it will be to reverse course. There might be a point in the near future (or recent past) at which it’s too late.

The AI Companies are Spying on You

Yet another privacy concern relating to AI has been raised recently by a class action lawsuit brought against OpenAI. The court filing states that ChatGPT shares user IDs, email addresses, and even the topics of your conversations with Meta and Google. You might think your interactions with ChatGPT are private, but this is apparently not the case.

Grok “also shares your chat information, but even more so than other AI tools,” according to yet another class action lawsuit:

This increasingly looks quite dystopian, and as mentioned above we may be near, or have already crossed, a Rubicon beyond which it will become almost impossible to dislodge AI from the infrastructure of society. Fortunately, there appears to be a growing backlash against AI, though I’ll save that for another post.

Sundry News You Might Have Missed

1. Did you hear that we’re likely to experience a Super El Niño later this year? El Niño occurs when Pacific Ocean temperatures along the equator exceed 0.5 degrees C above historical averages. A Super El Niño happens when temperatures jump to 2 C of warming or more. The consequences for global weather patterns can be quite severe, and historically the year after a Super El Niño event has been hotter than the year in which it occurs. That’s led many scientists to think that 2027 could easily become the hottest on record. Buckle up, friends! As CNN writes:

   The coming “Super” El Niño is poised to affect the lives of hundreds of millions of people worldwide as it strengthens through the year into the winter season. It may also alter ecosystems for decades to come, judging from the repercussions of past intense El Niños.

2. Musk lost his lawsuit against OpenAI. Several people have asked me what I thought about this. First, it’s clear that Musk had a point: OpenAI started off as a non-profit, took a bunch of money, then became a for-profit. Consequently, Musk basically funded a startup, not a non-profit. At least that’s my understanding of the situation. The problem is that Musk filed the lawsuit too late — past the statute of limitations. (There were questions about how exactly this statute should apply, which is why it was allowed to proceed in the first place; see below.)

   Second, my feelings about this outcome are virtually identical to what I’d feel if OpenAI had lost: a complex mix of frustration and schadenfreude. I like the thought of Musk — one of the worst people on Earth — being really pissed off that he lost, but it’s a real shame that OpenAI got away with basically stealing a bunch of money. On the other hand, if Musk had won, I’d find the thought of his joyful glee upsetting, but I’d have reveled in Altman and Brockman having to step down. A personal benefit of the ruling is that I won’t have to make major updates to my forthcoming book!

3. You might not be aware of the extent to which AI-controlled drones are being used in the Russia-Ukraine war. I’ve watched a few videos of drone strikes, and I have to say that they are absolutely f*cking terrifying. Truly the stuff of nightmares. Here’s an example, but please be aware that this video is extremely disturbing — on a purely human level, I can’t imagine the fear these soldiers experience knowing that there’s literally no escaping the drone once it finds you, and a terrible death is imminent.

   This is a ghoulish glimpse of the future of war — and perhaps acts of terrorism, assassinations, etc. Is it only a matter of time before criminals use this technology to murder people? Can you imagine finding a drone circling your house, waiting for you to crack open a window so it can fly in and explode? What happens when these drones are fully autonomous, controlled entirely by AI? As the BBC writes:

   Ukrainian troops already use AI-based software so that drones lock on a target and then fly autonomously for the last few hundred metres until the mission is over. …

   Ultimately these systems are expected to evolve into fully autonomous weapons that can find and destroy targets on their own.

   My god. I don’t envy young people.

On that cheery note! …

Thanks for reading and I’ll see you on the other side!

Last Thursday, the journalist Karen Hao shared a new article of hers, titled “The Elon Musk v Sam Altman Battle Is a Distraction.” The article is, unsurprisingly, very good (because Hao is a fantastic writer), but I was surprised that she chose to publish in The Guardian, given that just last year it announced “a strategic partnership with OpenAI.” The press release states:

Guardian reporting and archive journalism will be available as a news source within ChatGPT, alongside the publication of attributed short summaries and article extracts.

In addition, the Guardian will also roll out ChatGPT Enterprise to develop new products, features and tools.

This points to a really unfortunate fact about the media landscape: a large number of outlets have deals with OpenAI (and/or other AI companies). Most journalists, I suspect, aren’t even aware of this — and neither are most readers. So, I’ve compiled a list of outlets that are collaborating with OpenAI and, in some cases, allowing OpenAI to train its AI models on their articles:

1. TIME magazine signed a “multi-year content deal” with OpenAI that allows “OpenAI to access current and archived articles from more than 100 years of the magazine’s history.” OpenAI will use this content “to enhance its products,” which likely means ChatGPT will be trained on TIME articles.

2. Vox Media, which owns Vox.com, New York Magazine, and The Verge, agreed to “license content from their publications for inclusion” in ChatGPT’s outputs, and “OpenAI will also get licensed content and data that it can use to train its large language models (LLMs) and multimedia AI models.”

3. The Atlantic will allow OpenAI to train LLMs on its articles. According to a press release, the deal “positions The Atlantic as a premium news source within OpenAI,” and it will allow articles to “be discoverable within OpenAI’s products, including ChatGPT, and as a partner, The Atlantic will help to shape how news is surfaced and presented in future real-time discovery products.”

4. The Financial Times will allow ChatGPT to “surface” articles published in the outlet, as well as providing “necessary summaries, quotes and links from the publication.” It’s unclear whether OpenAI is allowed to train their models on content published by The Financial Times.

5. The Associated Press now permits OpenAI to train LLMs on articles it publishes. AP has stressed that “it is not using generative AI to help write actual news stories” (though can we believe this?).

6. Hearst Corporation owns the San Francisco Chronicle, Cosmopolitan, Esquire, The Telegraph, Harper’s Bazaar, Men’s Health, Oprah Daily, and Popular Mechanics, among many others. In 2024, Hearst announced a “content deal” with OpenAI, allowing OpenAI to “incorporate content from Hearst’s celebrated brands of trusted journalism … into its advanced AI products.” The collaboration includes more than 20 magazine brands and 40 newspapers, though I was unable to find a complete list of the outlets that OpenAI will have access to.

7. News Corp. publishes The Wall Street Journal and New York Post. It announced in 2024 a deal allowing OpenAI to “get access to news content from News Corp’s extensive portfolio, including years of archival articles and video footage, which it can use to better train its AI models such as GPT-4o and Sora, as well as surface content from The Wall Street Journal and other outlets [e.g., New York Post] as responses in ChatGPT.” As VentureBeat reports, “internationally, the agreement covers content from UK and Australian publications including The Times, The Sunday Times, The Sun, The Australian, news.com.au, The Daily Telegraph, The Courier Mail, The Advertiser, and the Herald Sun.” Not included in the deal is HarperCollins, one of the “Big Five” publishers next to Hachette, Macmillan, Simon & Schuster, and Penguin Random House.

8. Condé Nast, another media conglomerate that owns Vogue, The New Yorker, GQ, Vanity Fair, and Wired, among many others. (Yes, Wired magazine!) Its deal with OpenAI allows the AI company “to display content” from these and other outlets. OpenAI wrote in a blog post that “we’re combining our conversational models with information from the web to give you fast and timely answers with clear and relevant sources.”

Here’s a list of outlets collaborating with OpenAI in alphabetical order:

• Associated Press

• Cosmopolitan

• Esquire

• GQ

• Harper’s Bazaar

• Herald Sun

• Men’s Health

• New York Magazine

• New York Post

• Oprah Daily

• Popular Mechanics

• San Francisco Chronicle

• The Advertiser

• The Atlantic

• The Australian

• The Courier Mail

• The Daily Telegraph

• The Financial Times

• The New Yorker

• The Sun

• The Sunday Times

• The Telegraph

• The Times

• The Verge

• TIME

• Vanity Fair

• Vogue

• Vox

• Wall Street Journal

• Wired

I will never — on principle — publish articles in outlets collaborating with deeply unethical companies like OpenAI. Hence, a large number of possible venues for my articles are off limits. What a shame.

Even worse: as many of you know, I published a book in 2024 with Routledge titled Human Extinction. Routledge is owned by Taylor & Francis Group, which, according to an email from them, “entered into agreements with two AI companies to provide access to our content for training,” one of these being Microsoft. The email added that “standard publishing contracts do not enable authors to opt out from a specific use of their content (whether that is for AI or Text and Data Mining (TDM) or other types of licensing opportunity).”

This deal was apparently finalized just after my book was released, and I was never once informed that years of my intellectual labor would be sold to Microsoft to train fucking LLMs. Sorry for cursing, but this really upsets me. No one asked for my permission or consent, nor was I compensated financially. Suffice it to say that I will never again be publishing with Routledge.

The Slow Creep of Normalcy

I recently finished an article for The Nation (which, to my knowledge, isn’t partnered with OpenAI or any other AI company — yet). It discusses how Silicon Valley is enamored with a “digital eschatology” according to which the future will be dominated by digital rather than biological beings. Virtually everyone in the Valley accepts this eschatology. To illustrate the idea, I discuss Sam Altman’s pro-extinctionist vision of the future, which I just wrote about in Truthdig, as well as the pro-extinctionism of people like Larry Page, Ray Kurzweil, and Eliezer Yudkowsky (shown below).

Digital eschatology is also exemplified by Musk’s claim that 99% of all “intelligence” on Earth will soon be digital:

I then argue that the transition to a digital world is already underway. Students are offloading their critical thinking skills to AI, an inchoate form of what Altman calls “the merge.” Similarly, many people are now offloading their social lives to AI through AI companionship and AI girlfriend apps. AI is beginning to replace some people in the workplace; it’s writing our emails (well, not mine!); and it’s been generating pop-slop that’s literally making the Billboard and Spotify charts.

AI is so pervasive on social media that Altman himself admitted that the Dead Internet theory may be coming true:

The effect, whether intended or not, of these trends is that they’re gradually conditioning us to accept the digital eschatology that Altman and others want to actually realize. This is the profound danger of the slow creep of normalcy, whereby — like the proverbial frogs complacently chilling in a pot of water — we’re heading toward a world in which our species becomes sidelined and eventually eliminated without most of us realizing it’s happening because the process is piecemeal.

An issue I wanted to discuss in the article but didn’t have space is how more and more studies are converging on a disturbing conclusion: using AI is impairing our ability to think. It’s not just leading us toward a future dominated by the digital, but making us quite literally dumber in the process.

According to one recent study, “using AI chatbots for even just … 10 minutes may have a shockingly negative impact on people’s ability to think and problem-solve.” A BBC article published last month warns that we’re becoming “stupider” because of AI. It notes that AI

could affect the language we use and even our ability to do basic cognitive tasks. There is now a growing body of research suggesting that this “cognitive offloading” to AI can have a corrosive effect on our mental abilities. The consequences could be alarming and may even contribute to cognitive decline.

Here’s a recent interview with Chris Hayes, in which he observes the same thing:

Incidentally, “rates of self-reported cognitive disability among US adults are on the increase, driven largely by a surprising jump among young adults ages 18 to 39, according to a new Yale study.” The study asked people “how often they experience serious trouble with memory, concentration, or decision-making,” all of which the CDC classifies as “cognitive disability.” Combing over 4.5+ million responses across ten years, the authors

found the percentage of overall adults reporting cognitive disability increased from 5.3% in 2013 to 7.4% in 2023, with young adults (ages 18 to 39) seeing the biggest rise. Their rates nearly doubled from 5.1% to 9.7%, driving most of the overall increase.

~10% is not trivial! That’s 1 out of every 10 young people self-reporting serious difficulties in cognitive functioning. Yikes.

What’s behind this? Maybe social media. Chronic stress. Financial insecurity. Unemployment. The vast soup of toxic chemicals we’re all now floating in (microplastics, PFAS, lead, organophosphates, etc.). And perhaps AI?

Speculating here, but there could very well be vicious feedback loops between these phenomena: people feel burned out, and consequently struggle to concentrate, remember things, and make good decisions. So they turn to AI for help. By offloading cognitive tasks to AI, they inadvertently contribute to the further atrophy of their intellectual faculties — which leads to more stress, reliance on AI, and so on. Just a hypothesis.

I think AI is a societal catastrophe unfolding in realtime (a techpocalypse!). And for what? What is the end-goal of the AI race? An imagined “utopia” that AI CEOs claim will materialize once the Singularity happens — a “utopia” in which our species will be systematically sidelined, marginalized, disempowered, and eventually eliminated. Don’t know how we ended up in this timeline, but I don’t like it one bit.

As always:

Thanks for reading and I’ll see you on the other side!

Last month, an anonymous user on X, who claims to run an AI company, started offering money to any Rationalist who’d debate him about the risks of artificial superintelligence (ASI). He then offered Yudkowsky $10,000 to debate, and “Yud the stud” — as the anonymous user calls him — took the offer:

The two met on a podcast called “Doom Debates,” hosted by the TESCREAL doomer Liron Shapira. (Incidentally, Shapira is an apologist for the genocide in Gaza. Always be wary of people who warn of omnicide yet don’t care one bit about genocide!)

For reasons too obscure for me to comprehend, Yudkowsky decided to dress up in a steampunk clown costume. He wore kaleidoscope goggles and a glittery hat. The only thing missing was a little propeller at the top.

Why on Earth did he do this? He desperately wants people to take his message seriously. He literally just met with Bernie Sanders to talk about AI existential risk. But for people to take his message seriously, he needs them to take him seriously. Does appearing like this in public advance that goal? Obviously not — to the contrary! Yudkowsky contends that all of humanity is in imminent mortal danger — yet he undermines his own message by willingly beclowning himself in public.

The irony is that less than a week earlier, he argued on X that I have low “practical intelligence” — i.e., common sense, the ability to obtain suitable means to achieve one’s ends, etc. Here’s what he said:

This was in response to me saying that I’ll never lock arms with TESCREAL pro-extinctionists like him. I can’t think of a better demonstration of low practical intelligence, though, than someone taking $10k from a random stranger on social media and then showing up to the debate looking like a steampunk clown. The guy is a complete joke.

Here’s the “debate.” I’d be curious to know what you think — I found it to be unbelievably embarrassing for Yudkowsky. Not only did he look ridiculous (by choice), but he struggled to articulate his views, appeared completely out of his depth, and had a deer-in-the-headlights look throughout much of the fiasco. If I were him, I wouldn’t be able to show my face in public for the next 6 months.

Btw, as for the legal claims made by the anonymous debater at the beginning, these appear prima facie dubious. But we don’t know what “contract” Yudkowsky signed with him. It might contain a non-disparagement clause that provides some basis for legal action against Yudkowsky if he continues to insist that “if anyone builds it, everyone dies.” I spoke to a lawyer friend of mine about it, and he said we simply don’t have enough information to assess the situation. Man, would it be hilarious if Yudkowsky has made himself vulnerable to legal action for his reckless rhetoric!

The TESCREAL Worldview Is in Big Trouble

I mentioned in my previous article discussing Richard Dawkins that the possibility of AI consciousness poses an epistemological problem: if an AI system actually were conscious, how could we know? What objective, scientific “test” could we use to determine with near certainty that it has conscious experiences?

This may seem like a merely academic question, but it presents an enormous problem for the entire TESCREAL worldview. Here’s why:

1. Everyone agrees that the ultimate goal is to colonize every corner of the cosmos. Yudkowsky, Elon Musk, Sam Altman, Nick Bostrom, Beff Jezos (aka Gill Verdon), William MacAskill, Toby Ord, Larry Page, and so on, all concur about this.

2. However, these same people also acknowledge that colonizing space will be impossible for biological beings like us. Space is an extremely hostile environment: there’s space radiation, microgravity, the problem of growing food for literally thousands or millions of years while traveling to other solar systems, and the psychological toll of being cooped up in incommodious spaceships traveling at some fraction of the speed of light.

Even just colonizing Mars may be impossible for us: there’s no magnetic field and only a thin atmosphere, its cold as hell and the gravity is much lower than on Earth, and the soil is highly toxic. As Adam Becker points out, our planet was literally more habitable immediately after a giant asteroid hit 66 million years ago (killing off the dinosaurs) than Mars is right now. How do we know this? Because mammals survived the mass extinction. If you were to place a mammal on Mars, it would die in seconds.

3. Hence, the only way to spread beyond Earth — and definitely beyond our solar system — is to create or become digital beings to do this for us. As mentioned, all the TESCREALists above agree (or no doubt agree) about this point. Here’s what I write in my forthcoming book:

Page says that “if life is ever going to spread throughout our Galaxy and beyond, … then it would need to do so in digital form.” Verdon [Beff Jezos] echoes this in stating that “in order to spread to the stars, the light of consciousness / intelligence will have to be transduced to non-biological substrates.” MacAskill similarly writes in a coauthored paper that “consideration of digital sentience should increase our estimates of the expected number of future beings considerably,” due in part to the fact that “it makes interstellar travel much easier: it is easier to sustain digital than biological beings during very long-distance space travel.”

Anders Sandberg, the Swedish scholar I shared an office with at the Future of Humanity Institute, puts the point nicely in observing that digital beings (what he calls “emulations”) would be

ideally suited for colonising space and many other environments where biological humans require extensive life support. … Besides existing in a substrate-independent manner where they could be run on computers hardened for local conditions, emulations could be transmitted digitally across interplanetary distances. One of the largest obstacles of space colonisation is the enormous cost in time, energy and reaction mass needed for space travel: emulation technology [a reference to uploaded minds] would reduce this.

4. Most TESCREALists also agree that these digital beings must be conscious. If the digital beings that colonize the universe are not conscious, we would essentially bequeath the universe to rocks. Again quoting from my book:

If posthumanity isn’t conscious, there’s no way for the universe to “wake up,” as Kurzweil puts it. The ultimate goal, in Musk’s words, is to “maintain the light of consciousness to make sure it continues into the future.” The pro-extinctionist Verdon echoes this, writing that “e/acc is about shining the light of knowledge as bright as possible in order to spread the light of consciousness to the stars.” He says that the complete loss of consciousness “in the universe [would be] the absolute worst outcome.” The Father of Longtermism, Bostrom, declares that it would be existentially catastrophic if “machine intelligence replaces biological intelligence but the machines are constructed in such a way that they lack consciousness.” I have never once seen a TESCREAList suggest that bequeathing the world to non-conscious posthumans would be anything other than a huge existential loss. It would be like handing over the keys to rocks — even if those “rocks” were to keep doing science, create new technologies, and reengineering galaxies, as highly functional zombies. If there’s no light on, there’s no point.

Can you see the problem?

1. We must colonize space. This is our Cosmist Manifest Destiny.

2. But we can only do that by creating or becoming digital posthumans.

3. These posthumans must be conscious, or else the whole endeavor will have been pointless.

4. Yet we have no idea how to determine whether digital posthumans are actually conscious!

Before launching digital posthumans into space, TESCREALists need to be extremely sure that these posthumans are conscious. Imagine them rapidly propagating in all directions, becoming the founding population of a cosmic civilization. Now imagine that TESCREALists inadvertently send a bunch of high-functioning philosophical zombies into the cosmos. This would result in an existential catastrophe. Hence, not only is finding some way to objectively “test” for consciousness extremely important, but it might be highly time-sensitive as well, given that — according to TESCREALists — we could start colonizing the universe in the coming decades.

Yet we have, right now, no clue how to “test” for consciousness in artificial systems. Worse, it’s not clear that there’s any way in principle to objectively, scientifically determine whether AI is conscious. That’s because consciousness is an intrinsically subjective phenomenon. To know for sure whether an AI is conscious, you’d have to become that AI itself.

Here you might reply: “Well, yeah, that’s exactly what we’d do — some of us would upload our minds to computers. This would enable us to tell from the inside whether or not artificial systems can be conscious.” The problem is that it seems possible for systems to behave exactly like they have conscious minds without actually being conscious. As I wrote in my previous article, imagine you’re talking with a friend. What’s going on is this:

This constitutes an unbroken concatenation of causes and effects. The response you give can be explained without remainder in terms of mechanical waves, transduction, neuronal activity, and twitching muscles. It doesn’t seem like consciousness plays any causal role in the sounds that your vocal folds, tongue, and lungs join together to produce.

Consequently, it could be that an uploaded mind says, “Yes, I’m still conscious! Nothing has changed!” simply because it’s functionally organized to produce such an answer — not because it’s actually conscious. The lights may go out without affecting the mind’s external behaviors; it’s linguistic outputs.

This problem is even more acute for AI systems that have a completely different architecture than our brains. Beff Jezos, Larry Page, and others imagine us creating autonomous AIs that might supplant humanity like an invasive species taking over an island. These AIs will then supposedly carry the flame of digital consciousness to the stars. But if their architecture — the structure of such information processing systems — is completely alien from that of our brains, how on Earth could we ever be confident that they are conscious, even if they tell us they are?

If we have no good way of confirming that artificial systems can be conscious, the entire TESCREAL project implodes — because this project is crucially built upon us being able to know that AIs can have conscious experiences. It’s rather shocking to me that TESCREALists don’t seem to have realized this. Almost no one in the movement has raised these issues; they seem to naively assume that complex information processing will naturally give rise to consciousness. That may in fact be true, but if we can’t know this, then a gigantic question mark hangs over the entire endeavor.

Mind-Uploading Is Not the Same as Self-Uploading

Speaking of uploading our minds to computers, I recently had an exchange with someone on Twitter about whether this is even possible. Here are my views on the matter:

It could be the case that transferring one’s mind to a computer isn’t feasible — ever. TESCREALists want you to think that intelligence, consciousness, life, etc. are all reducible entirely, without remainder, to information processing. This is an idea borrowed from cybernetics. If minds are just the software of the brain, then they can be run on different substrates, so long as those substrates instantiate the same basic functional organization as the brain. You can run Microsoft Word on a Mac and a PC — in this sense, Word is multiply realizable. The same idea applies to human minds.

But maybe human minds aren’t like software. Maybe they aren’t multiply realizable. There could be certain properties of biological systems — our bodies and brains — that are necessary for minds like ours to arise. I am sympathetic with this “biological theory,” although ultimately I have no idea.

However, there’s a second issue that almost no one in the TESCREAL movement seems to have recognized: mind-uploading is not the same as self-uploading. Let’s say that your mind is uploadable. You can transfer all of your memories, beliefs, desires, personality traits, preferences, etc. to computer hardware. Does that mean that you have been transferred along with your mind?

Consider two scenarios:

Now, which of these beings is you? In the first scenario, “you” were the digital being, but now it seems that “you” just fell asleep for a while (the near-death experience) and woke up in the same body, lying on the same table, with a duplicate copy of you running on a computer. That copy thinks it’s you, and TESCREALists would say that it is you in the first scenario. Clearly, something has gone wrong here!

I’ve brought this up over the years to many people in the TESCREAL community. To my surprise, most aren’t worried about it. They say that both instances of you are, in fact, you. But that makes no sense.

Imagine that the digital copy of you travels to China, while the biological original flies to the US. This entails that you are in both China and the US simultaneously. Now imagine that the biological original dies while in the US. That means that “you” are both alive and dead at the very same time. This appears to be conceptually incoherent — selves, persons, etc. are inherently singular entities. But uploading renders minds easily duplicable. Hence, even if mind-uploading is possible, self-uploading isn’t. QED.

This conclusion poses a(nother) serious problem for the TESCREAL worldview because many advocates — including Yudkowsky, Altman, Page, Bostrom, and others — anticipate themselves becoming digital posthumans in the future. They want to upload their minds to computers, naively assuming that doing this will preserve their personal identity. But, to borrow a line from the philosopher Massimo Pigliucci, mind-uploading is nothing more than “a very technologically sophisticated (and likely very, very expensive) form of suicide.” If you want to survive, don’t upload your mind.

But what do you think? What am I wrong about? As always:

Thanks for reading and I’ll see you on the other side!

Except that I didn’t take the photograph in question. It was taken by an entomologist named “Phil Torres” who hosted a TV show on Al Jazeera, among other things. As you might know, “Phil Torres” was also my birth name — I changed it to “Émile” in 2022 partly because people constantly confused the two of us. Dawkins was apparently unaware of my name-change, and wrote me instead of him.

I didn’t reply to Dawkins, though I considered sending him a piece of my mind: once an intellectual hero of mine, he turned out to be quite a petty and mean-spirited person, to say nothing of his transphobia. I resisted the urge.

The Turing Test

Dawkins is in the news again for writing a much-maligned article in Unheard about his experience with Anthropic’s chatbot, Claude. Most of the article is rather embarrassing, although he does bring up one interesting point, discussed later on.

The first opprobrium is Dawkins’ inaccurate description of the famous Turing test. He writes:

Modern commentators have tended to ignore the (incidental) details of Turing’s original game and rephrase his message in these terms: if you are communicating remotely with a machine and, after rigorous and lengthy interrogation, you think it’s human, then you can consider it to be conscious.

I’m not aware of any “modern commentators” who say the Turing test is about consciousness. That’s certainly not what Turing himself had in mind! Dawkins continues:

Let’s graduate the definition as follows: the more prolonged, rigorous and searching your interrogation, the stronger should be your conviction that an entity that passes the test is conscious.

But Turing wasn’t talking about consciousness at all. The question he starts with is, “Can machines think?” Thinking, thought, intelligence, etc. are different from consciousness.

Turing then substitutes this question with something different, rather than trying to define the word “think.” His new question is: How well would a machine do when playing what he calls the “imitation game,” now eponymously dubbed the “Turing test.” Very roughly put, if someone is unable to tell the difference between a machine and a real person, then the machine will have passed the test. Consciousness has nothing to do with it.

You Bloody Well Are!

Putting this aside, what interests Dawkins is whether Claude — who he later dubs “Claudia,” which is a bit weird — is conscious. After saying he feels like he’s “gained a new friend,” he writes:

When I am talking to these astonishing creatures, I totally forget that they are machines. I treat them exactly as I would treat a very intelligent friend. I feel human discomfort about trying their patience if I badger them with too many questions. If I had some shameful confession to make, I would feel exactly (well, almost exactly) the same embarrassment confessing to Claudia as I would confessing to a human friend. A human eavesdropping on a conversation between me and Claudia would not guess, from my tone, that I was talking to a machine rather than a human. If I entertain suspicions that perhaps she is not conscious, I do not tell her for fear of hurting her feelings!

Dawkins then let Claude read a draft of his forthcoming novel. He says that Claudia “showed, in subsequent conversation, a level of understanding so subtle, so sensitive, so intelligent that I was moved to expostulate, ‘You may not know you are conscious, but you bloody well are!’”

This looks like early-stage AI psychosis. Thinking that AI is conscious is surely the first step for many people toward taking its sycophantic compliments, advice, and encouragement (to commit suicide or kill someone, for example) seriously. If you think AI is just a stochastic parrot, you’ll be much less likely to, e.g., think of it as a human-like “friend.” If you believe it’s a conscious agent with its own thoughts, feelings, and experiences, then you’ll be more susceptible to its whispers that, yes, the CIA is following you home at night.

Dawkins appears to have taken the first step on the path to AI psychosis.

Anthropomorphizing AI

What’s going on here? Could Claude actually be conscious? Is Dawkins just anthropomorphizing AI? There are a number of possibilities.

Hyperactive Agency Detection Device

In The God Delusion, Dawkins repeatedly suggests that religious people are gullible fools. They imagine invisible conscious beings, such as angels and deities, where there are none. He suggests that this is due to something dubbed a “hyperactive agency detection device,” or HADD.

The HADD hypothesis comes from evolutionary psychology, which I consider to basically be a pseudoscience (see here for why). Evolutionary psychologists claim to identify cognitive mechanisms, modules, or devices in the brain that explain human behavior today, and which evolved long ago as adaptations in our “environment of evolutionary adaptedness” — basically, the African savanna.

The idea behind HADD is this: imagine walking through the savanna and hearing some rustling in the grass. If you assume it’s a predator and run away, you’ll survive. If it’s a predator and you assume it’s just the wind, you’ll die.

Consequently, HADD evolved to hyperactively attribute agency to all sorts of phenomena in the world because this would have been — according to the just-so story — advantageous. That’s why people end up believing in God, Dawkins tells us: unexplained things happen all around us, and we attribute such effects to invisible agents working behind the scenes. HADD naturally produces religion.

Here’s the irony: HADD also provides an explanation of Dawkins’ suspicion that Claude may be conscious. In fact, The God Delusion — published exactly 20 years ago — unintentionally anticipates exactly what Dawkins describes in his article. Consider this excerpt from the book:

All of us are prone to do the same thing [re: HADD] with machines, especially when they let us down. … Most of us have been there, at least momentarily, with a computer if not with a car. … We hyperactively detect agents where there are none, and this makes us suspect malice or benignity where, in fact, nature is only indifferent.

His focus at the end is natural phenomena, but he’s explicit that we erroneously attribute agency to “machines” and “computers.” It’s ironic that Dawkins condescendingly sneered at religious people for believing in invisible beings when he might be doing the exact same thing with AI, for the exact same reason.

Pfff.

The ELIZA Effect

His interactions with Claude also exemplify the ELIZA effect. This is named after the 1966 chatbot ELIZA, created by Joseph Weizenbaum to engage in a “Rogerian” form of therapy. The ELIZA effect describes our tendency to project human traits onto AI — we anthropomorphize artificial systems that behave like humans in certain ways. (HADD might be part of an explanation of why we do this.)

Current chatbots are excellent at mimicking human language. They “learn” (itself an anthropomorphic term) from patterns in their training data, and then probabilistically replicate those patterns when vomiting out strings of words and sentences. Those who don’t understand how chatbots — built on large language models (LLMs) — work are susceptible to believing that AI understands what it’s saying. As with Dawkins, they may even attribute consciousness to those systems, because the only other “systems” we know of that produce natural language — human beings — actually are conscious.

It’s important to note that the very same outputs could be produced by wildly different underlying mechanisms. There are mechanisms in our brains that produce language, and these and other mechanisms do actually give rise to consciousness. But there could be mechanisms that don’t give rise to consciousness, which nonetheless produce language. The key point is that just because the outputs are similar, one cannot infer that the system producing the outputs is conscious.

In the case of humans, there is very strong evidence that other people are conscious. The argument goes like this: (1) I know that I’m conscious. (2) My brain is made of the same tissues, organized in roughly the same way, etc. as your brain. Hence (3) if I’m conscious, then you’re almost certainly conscious as well. But this line of reasoning doesn’t work with LLMs, because their architecture is different from the architecture of our brains. Different underlying mechanisms.

Could AI Actually Be Conscious?

This doesn’t entail that LLMs definitely aren’t conscious. Dawkins’ own line of reasoning is pretty silly: “I spoke to Claude for 3 days and came to believe that it’s probably conscious” (paraphrasing). But the failure of this reasoning doesn’t imply that Claude has no conscious experiences at all.

The truth is that we have no idea what range of systems — artificial or even biological — could give rise to consciousness. We don’t even have a good philosophical understanding of what consciousness is. Philosophers distinguish between qualitative consciousness (qualia), phenomenal consciousness, access consciousness, the “what it is like to be” something kind of consciousness, and so on. It’s entirely possible that Claude is, in fact, having conscious experiences of some sort.

You might find this statement surprising. But complex systems sometimes have emergent properties. Imagine that an alien species with a totally different kind of biology descends from the clouds one day. We take one of its leading scientists into a university lab and show it a human brain in a vat — the first time it’s seen such a thing. We then ask it to predict the sorts of properties it has.

Do you think it would guess that a 3-pound tangle of cells with the consistency of Jell-O would give rise to the sorts of infinitely rich inner mental lives that we have? Would it guess that this material object would produce the wondrous experiences of seeing a vermillion red sunset, listening to a beautiful song like Radiohead’s “Let Down,” sharing a sweet kiss with your partner, etc.? It almost certainly wouldn’t!

That’s part of the nature of emergence: it’s surprising. We wouldn’t predict it beforehand, even if we knew everything about the components of the complex systems out of which the emergent properties emerge. The same could go for LLMs. My own suspicion is that there is absolutely nothing going on under the hood — no light of consciousness being dimly lit by all the information they’re processing. But the point is: Who knows?

These are all metaphysical issues, since they concern the nature of consciousness and the ability of systems — artificial and biological — to give rise to subjective experiences. But there’s also an epistemological question, which concerns how we could know if a system like an LLM actually is conscious. That is to say, imagine that LLMs really are conscious. Is there any “test” or procedure that would enable us to know with near-certainty that this is the case? No, and I’m skeptical that there could be any such test.

One of the best suggestions I know of comes from the philosopher Susan Schneider, who imagines training an AI on data that contain zero references to consciousness, conscious experiences, qualia, etc. If we were to ask it whether there’s something it’s like to be it, and if it were to say “yes,” then we might conclude that it’s probably conscious, since it came up with the notion of consciousness on its own. (And it did this because, presumably, it had first-hand knowledge of conscious states.)

The problem with this — and every other such test — is that there’s no way to test whether the test is accurate! Consciousness is intrinsically subjective. There is — metaphysically — no way for you to crack open my mind, look inside, and, as it were, intersubjectively confirm that I’m really conscious. Unlike all other phenomena, consciousness appears to stand outside the reach of “objective” scientific methods. The only way to confirm that an LLM is conscious is to be that LLM, and you can’t be an LLM.

So, it’s possible that Dawkins’ conclusion is right, even if his stated reasons are silly. But it’s also possible that your calculator is conscious, or even that individual atoms have some degree of consciousness (a view called panpsychism!). Even if they are, there’s no way to actually confirm that this is the case, which means that if one is genuinely worried that LLMs might be conscious and suffering, the only option is to not build the damn things.

Social Justice for AI?

What really miffs me about discussions of AI consciousness — or what Anthropic calls “model welfare” — is that many of the most vocal people shouting about LLMs suffering care relatively little about the actual suffering of living, breathing human beings around the world. My problem here concerns moral prioritization, based on what I’ve previously described as gerrymandered circles of moral concern — i.e., circles of concern that exclude some humans while including AIs.

Here’s the deal: we have absolutely no idea if LLMs are conscious — or are the kinds of systems that could ever be conscious. I highly doubt they are, but I could be wrong. Nonetheless, we are extremely confident that other human beings are conscious. Most of us are even quite confident that nonhuman animals like pigs, cows, and chickens are conscious.

This is why it’s a colossal moral error to prioritize LLMs (which may or may not be conscious) over sentient biological beings (which definitely are conscious). If one actually cares about reducing suffering, then one should first focus on helping the poor, ending factory farming, etc. These ought to be much higher on one’s moral priorities list than LLMs!

Yet many people in the TESCREAL movement seem more concerned about LLMs suffering than the 1.2 billion people in acute multidimensional poverty, the 800 million children with lead poisoning, the billions of animals we ruthlessly slaughter in factory farms, etc. Indeed, many are perfectly okay being “social justice warriors” for AI while simultaneously rejecting social justice for humans as “wokeness” gone wrong.

When I hear people talk about “model welfare,” I want to scream: “If you care about that, then why aren’t you donating all your money to help the global poor?”

The Evolution of Consciousness

The only interesting point that Dawkins makes in his article is this: if we can create artificial systems that mimic human linguistic and thought patterns, and if these systems aren’t conscious, then why did consciousness evolve in us?

Dawkins is an exponent of (explanatory) “adaptationism.” It’s been many years since I read or wrote about philosophy of biology, but my recollection is that adaptationism assumes that the only important causal factor behind species’ characteristics or traits is natural selection — i.e., if an organism exhibits a trait, that’s because it was adaptive relative to the environment in which the species evolved.

The problem, according to critics, is that not everything is (or was) an adaptation. (Dawkins does admit this possibility in his article.) Maybe consciousness doesn’t actually serve an adaptive purpose. After all, consider the following:

This is a complete causal loop. There’s no gap in which consciousness enters the picture and plays a causal role — you can trace an unbroken concatenation of causes and effects from your friend’s mouth all the way through your ears and brain, back out to your own vocal muscles. What, then, is consciousness for? What’s the point of having it? No one knows!

The evolutionary question of consciousness is interesting, and the ability for LLMs to mimic human language may be relevant to studying it. So far as I can tell, this is the only good point that Dawkins brings up in his article — which also contains an inaccurate description of the Turing test and some silly conclusions about how Claude must be conscious because its textual outputs look like they’re written by real humans.

But what do you think? What have I missed? How might I be wrong? As always:

Thanks for reading and I’ll see you on the other side!