Was Hitler a mass murderer? You may be surprised to know that there’s no historical evidence that he ever personally killed anyone — other than himself in 1945.1 (See, e.g., here and here.) Military records indicate he took no lives during World War I. He apparently didn’t murder anyone during the Beer Hall Putsch or the Night of the Long Knives. And he probably didn’t shoot his niece, Geli Raubal, despite rumors to the contrary.

If “murderer” means “one who personally kills another,” then Hitler wasn’t a mass murderer, because he wasn’t a murderer at all.

But of course Hitler was a mass murderer. Who would argue against this? His decisions resulted, whether intentionally or not, in an estimated 17 million civilian deaths during World War II. Though he never once pulled the trigger, tightened the rope, or held the knife, Hitler was morally responsible for these deaths. If it weren’t for him, those people wouldn’t have died the way they did, when they did.

Subscribe now

By the very same criterion, Elon Musk is a mass murderer.

To my knowledge, Musk has never personally killed anyone. But he did terminate USAID last year, calling it a “criminal” organization that “has to die.”

“USAID was not just an apple with a few worms inside,” he wrote on X, “it was a festering ball of worms in the shape of an apple.” He also said it “is/was a radical-left political psy op.” DOGE then engaged in what one senior official described as a “mafia-like takeover” of the agency.

According to the Lancet, the most prestigious medical journal in the world, the termination of USAID is projected to cause

more than 14 million additional deaths by 2030, averaging more than 2.4 million deaths per year. These deaths include 4.5 million among children younger than 5 years, or more than 700,000 deaths annually.

Late last year, Harvard reported that “hundreds of thousands” of people have already died since USAID was terminated. As of April 2026, the number may have risen to 1.6 million — and counting.

***

Hitler was responsible for 17 million civilian deaths over six years (1939 to 1945). Musk will likely be responsible for 14 million deaths over five years (2025 to 2030), if the Lancet’s projections are accurate. If Hitler was a mass murderer, then so is Musk. Indeed, given the numbers, one is forced to conclude that Musk is a mass murderer of world-historical proportions — on par with some of the worst dictators in human history.

Consider that:

• Saddam Hussein “was responsible for the deaths of hundreds of thousands of people” during his 23-year reign of terror in Iraq.

• George Bush was responsible for upwards of 1 million civilian deaths by having illegally started the 2003 Iraq War.

• Pol Pot was responsible for some 2 million deaths in four years during the Cambodian genocide.

• Stalin was responsible for about 1 million deaths, though another 2.3 died from “neglect and irresponsibility.” The Soviet famine of 1932-33 took another ~6.5 million.

• Mao was responsible for around 30 million deaths over two years due to the Great Leap Forward.

As with Hitler, there is no verified, universally accepted evidence2 that Saddam Hussein, George Bush, Pol Pot, Stalin (after becoming leader of the Soviet Union), or Mao ever personally killed anyone.

Think about that for a moment. These monsters took lives by pulling the levers of power available to them. They implemented policies that killed untold millions due to starvation, deprivation, neglect, and outright slaughter. In some cases, the deaths weren’t specifically intended. But we still count them nonetheless, because they could have — and should have — been anticipated.

By the numbers, Musk is worse than Hussein, Bush, Pol Pot, and Stalin, but not as bad as Hitler or Mao — though he’s not far behind Hitler. Very few people in history have been personally responsible for so many innocent deaths as the CEO of SpaceX, the world’s first trillionaire.

To put 14 million deaths in further perspective:

• Hurricane Katrina took the lives of 1,392 people.

• The atomic bombings of Hiroshima and Nagasaki killed about 246,000 people.

• The Antioch earthquake in 526 CE caused 250,000 deaths.

• The Rwandan genocide took the lives of up to 662,000 Tutsi.

• The HIV/AIDS epidemic has killed some 700,000 people in the US, as of 2018.

• The Armenian genocide took the lives of up to 1.5 million people.

• As many as 13 million civilians died in World War I.

These are all events that, insofar as people know about them, carry a certain unsettling “dread factor.” Yet, by the numbers, all are less bad than the catastrophe unfolding as a result of USAID’s termination by Musk.

Note also that the 14 million estimate doesn’t include deaths that will occur after 2030. The loss of USAID could have effects that linger for much longer.

***

Musk didn’t kill all of these people by ordering their execution. But so what? Hitler also didn’t order the execution of every person his regime buried.

What’s the difference between smothering a hospital patient with a pillow and intentionally unplugging their life-support machine? Would you not consider the person who intentionally unplugs the machine as having committed cold-blooded murder? Would they not be a killer?

Or, imagine seeing a young child drowning in a shallow lake. An altruistic Samaritan runs into the water to heroically rescue the innocent life. But just as she picks up the child, a random asshole intervenes, forcibly flings the baby from the Samaritan’s arms, and the baby drowns. Then you find out that this random asshole was Musk himself.

This is what he’s done with USAID. It was providing life-saving medical support to millions of people around the world. It was their life-support machine. And he pulled the plug, resulting in millions of patients — many of whom are innocent children drowning in a metaphorical lake — dying, often in excruciating circumstances. As my friend Devin Kellis wrote in his guest post for this newsletter:

Peter Donde (age 10), Evan Anzoo (age 5), and Achol Deng (age 8) died from loss of access to human immunodeficiency (HIV) medications; Gy Gael (age 8) died from an infection without access to antibiotics; Babagana Bukar Mohammed (age 7) died from complications of sickle cell disease amidst health clinic closures; and Yamah Freeman, a mother of two, bled to death amidst ambulance delays due to fuel shortage while expecting her third child.

Even if Musk didn’t fully understand the extent to which USAID was the life-support machine for tens of millions, it was his moral duty — as a person who forced his way into a position of immense power — to investigate. There is no ethical excuse for the world’s richest person to set fire to an organization that just 5 minutes of research would have shown him is keeping millions of children above water. He is blameworthy for all of these deaths. Can anyone seriously argue to the contrary?

***

If one compares Musk to Ted Bundy, his murderous actions are completely off the charts. Ted Bundy confessed to 30 murders but might have killed up to 100 women. The latter number is 0.000714% of the total deaths caused by Musk within the next 4 years, according to the Lancet.

Why does Bundy strike fear, dread, and disgust in our hearts but not Musk, whose body count is literally 140,000 times larger? Why does the evil of Bundy disturb our conscience more than the actions of DOGE and its now-trillionaire leader? Is it just psychic numbing and scope neglect? As Stalin famously said, “If only one man dies of hunger, that is a tragedy. If millions die, that’s only statistics.”

When considering the enormity of Musk’s moral crimes, I often think about a poignant scene at the end of Charlie Chaplin’s Monsieur Verdoux, about a man who murders women to steal their money (as I recall — it’s been decades since I watched it). During his trial, Chaplin’s nefarious but charming character delivers a speech in which he says:

As for being a mass killer, does not the world encourage it?
Is it not building weapons of destruction for the sole purpose of mass killing?
Has it not blown unsuspecting women and little children to pieces — and done it very scientifically?
As a mass killer, I am an amateur by comparison.

That final line haunts me. Bundy was an amateur by comparison to Musk. The degree of callousness and psychopathy required for Bundy to do what he did is unfathomable to morally normal people like you and me.

But the same is true of Musk — it takes a real psychopath with zero conscience, empathy, or compassion to recklessly obliterate an agency that kept millions of children alive, and then express literally zero remorse or compunction for the profound and lethal harms directly caused by his intentional, willful actions.

Fuck the billionaires, but especially fuck the trillionaire. We live in an infinitely sad world, because the chasm between how good things could be and how bad they actually are is enormous and growing. It didn’t need to be this way.

Subscribe now

As always:

Thanks for reading and I’ll see you on the other side!

What follows is an urgent call to boycott SpaceX, given Musk’s termination of USAID (among other things). We’re hoping this catches on, and are actively pursuing signatures and endorsements from prominent people and organizations. My favorite line from Devin’s piece: “The road to Mars is being paved in blood, giving new meaning to the ‘Red Planet.’” Brilliant!

I’ll have my own article on SpaceX’s IPO and Musk becoming the world’s first trillionaire — which really feels epochal — out (probably) on Tuesday. It argues that Musk should be seen as a mass murderer of world-historical proportions. I do not hold back!

Without further ado, here’s Devin’s excellent piece:

SpaceX is an American aerospace, satellite, social media, and artificial intelligence (AI) company founded by Elon Musk in 2002. The company was created to colonize Mars and build the commercial space industry. In 2015, SpaceX announced its entrance into the satellite business via Starlink. Then, in 2026, SpaceX expanded to social media and AI by acquiring X (formerly Twitter) and xAI (now SpaceXAI), respectively. Yesterday, on Friday, June 12, 2026, SpaceX transitioned from private to publicly traded company via the largest initial public offering (IPO) in history, making Musk the world’s first trillionaire.

Alongside the SpaceX IPO, our moment includes the “Great Aid Recession,” an ongoing period initiated in part by Musk and defined by nation states’ mass withdrawal from global health spending. As CEO of SpaceX and de facto head of the U.S. Department of Governmental Efficiency (DOGE), Musk played a key role in abruptly dismantling one of the most cost-effective, life-saving humanitarian aid organizations in history: the U.S. Agency for International Development (USAID). Beforehand, while USAID was investigating its use of Starlink satellites in the Russo-Ukrainian War, Musk implied that DOGE deregulation would help SpaceX. Notwithstanding conflicts of interest, what is most important is that ending USAID is causing immense suffering and death for real, human beings.

Journalists are sharing those people’s stories with us. Reports include: Peter Donde (age 10), Evan Anzoo (age 5), and Achol Deng (age 8) died from loss of access to human immunodeficiency (HIV) medications; Gy Gael (age 8) died from an infection without access to antibiotics; Babagana Bukar Mohammed (age 7) died from complications of sickle cell disease amidst health clinic closures; and Yamah Freeman, a mother of two, bled to death amidst ambulance delays due to fuel shortage while expecting her third child. Analyses on the health effects of ending USAID by the former Acting Assistant Administrator for Global Health at USAID, Impact Counter, the World Health Organization, the Center for Global Development, Lancet, and Lancet: Global Health suggest that Musk’s approach to cutting USAID has already led to hundreds of thousands of preventable deaths like those reported above, with millions of lives still at risk through 2030. In addition, research in Science suggests that cutting USAID may have increased deaths from violent conflict. A new outbreak of the Ebola virus, which kills people by causing hemorrhagic fever, is also being made worse by USAID’s absence. The road to Mars is being paved in blood, giving new meaning to the “Red Planet.”

These are enormous, century-defining harms. Understanding them requires us to recognize that, as the late psychiatrist Robert J. Lifton wrote,“[i]t is very difficult to kill large numbers of people without a claim to virtue.” Musk’s SpaceX S-1 filing to the Securities Exchange Commission (SEC) claims to aim to colonize space out of a desire for humanity to “avoid the same fate as the dinosaurs.” Yet, the same filing’s Risk Factors section fails to consider meaningfully whether SpaceX might increase the risk of human extinction. Musk himself has argued (in court, no less) that “AI may kill us all,” yet SpaceX’s S-1 filing devoted more than 90% of its total addressable market to AI.

This cherry-picking of extinction risk reduction arguments when it is convenient to business is projection that obfuscates a perception of reality wherein SpaceX and Musk themselves threaten humanity. The veil of virtue is undone in light of ending USAID with DOGE, eroding epistemic norms on X, allegedly breaking environmental laws with xAI, militarizing outer space via Starlink, and empowering the only U.S. president this century to call for a resumption of nuclear weapons testing. Together with the autocratic corporate governance of SpaceX, Musk’s ability to back political threats with trillion-dollar wealth bestows upon him an unacceptably concentrated power to develop, deregulate, and steer world-destroying technologies. Authoritarianism is growing amongst the “Agents of Doom.”

The upcoming mega-IPOs of Anthropic and OpenAI (both companies that contributed to the sitting U.S. president’s inauguration fund) suggest that this is the rule, not the exception, of our times. The “Second Gilded Age” came from somewhere, and we must work to understand and address the underlying drivers which enable a small number of individuals or even one person to facilitate mass harm. That line of thinking suggests that the ethical response to the SpaceX IPO is not investment, but a principled refusal to participate. Companies, nations, and organizations alongside their constituents should consider and openly discuss boycotting SpaceX and no longer purchasing Starlink Internet, X premium subscriptions, and SpaceXAI infrastructure, among other services.

#BoycottSpaceX.

As always:

Thanks for reading and I’ll see you on the other side!

Setting the stage: The 18th-century Enlightenment witnessed a self-conscious pivot away from the epistemology of religion — faith and revelation — and toward a new approach to understanding the world built around reason. However, most Enlightenment thinkers weren’t atheists, with a few exceptions like the French encyclopedist Diderot (quite possibly the first “atheist” in the modern sense of the term1). Such thinkers tended to be, instead, deists who believed in God’s existence but rejected the view that God intervenes in the world. He set the universe in motion, then stepped back and let things run their course.

It wasn’t until the following century — the 1800s — that theism began to significantly wane among the educated classes. This is when Frederich Nietzsche famously declared that “God is dead,” and Karl Marx denigrated religion as the “opium of the masses.” Depending on which historian you ask, there were three primary reasons Christianity declined in the West: biblical criticism, which seemed to show that the Bible is historically inaccurate and internally inconsistent; new philosophical reflections on ideas like the problem of evil, which purports to show that God cannot be all-powerful and perfectly good; and scientific developments like Charles Darwin’s theory of evolution by natural selection and the second law of thermodynamics.2 Together, these rendered traditional religion completely untenable in the eyes of many educated people — belief in the Christian God just isn’t plausible.

The problem was that this new “materialist” worldview stripped the Western mind of crucial sources of meaning, purpose, and eschatological hope (or hope for the future of humanity), resulting in a kind of widespread psycho-cultural trauma. Religion’s decline left a gaping void behind, as exemplified by Bertrand Russell’s 1903 “A Free Man’s Worship,” the most widely circulated essay he wrote.3 After a dismal description of the history of Creation, as conveyed to Dr. Faustus by Mephistopheles, Russell writes:

Such, in outline, but even more purposeless, more void of meaning, is the world which Science presents for our belief. … That Man is the product of causes [Darwinian evolution] which had no prevision of the end they were achieving; that his origin, his growth, his hopes and fears, his loves and his beliefs, are but the outcome of accidental collocations of atoms; that no fire, no heroism, no intensity of thought and feeling, can preserve an individual life beyond the grave; that all the labours of the ages, all the devotion, all the inspiration, all the noonday brightness of human genius, are destined to extinction in the vast death of the solar system [a reference to the second law of thermodynamics], and that the whole temple of Man’s achievement must inevitably be buried beneath the debris of a universe in ruins — all these things, if not quite beyond dispute, are yet so nearly certain, that no philosophy which rejects them can hope to stand.

The gaping void left by religion was intolerable, and so people quickly filled it with new secular belief systems and eschatological narratives that closely paralleled those of Christianity.

Marxism provides an example: humanity started off in a state of “primitive” communism (Eden), but this state was corrupted (capitalism, the Fall) by sin (exploitation, alienation), until a messianic figure (Marx) emerged to spread the good word that salvation is possible (communism), leading to a world communist state (heaven). Believers (communists) will enter this heaven while the reprobates (capitalists) will be banished.4

This narrative restores the lost meaning, purpose, and eschatological hope in the aftermath of religion’s sudden retreat — it tells us that, by working to overthrow capitalism, we are contributing to something much greater than ourselves, and that the outcome of this will be a paradisiacal world free of class conflict and marked by abundance for all. (Note that I endorse the overthrow of capitalism! I just reject aspects of Marx’s historical materialism.)

Transhumanism: A Religion Without Revelation

Another example is transhumanism — the “T” in “TESCREAL.” The idea of transhumanism was developed not long after Russell published his essay by people like J. D. Bernal, J. B. S. Haldane, and Julian Huxley. All three were eugenicists, and transhumanism is best understood as eugenics on steroids. By this, I mean something specific: the goal of traditional eugenics was to prevent the evolutionary degeneration of our species and create the most perfect version of us possible. Transhumanists took this a step further, reasoning: “Why stop at perfecting humanity? Why not entirely transcend it?” The result would be, in contemporary parlance, a superior new “posthuman” species.

One of the first book-length explorations of transhumanism was Huxley’s 1927 Religion Without Revelation. As this suggests, Huxley envisioned transhumanism as a kind of secular religion — a scientifically grounded replacement for Christianity. As he wrote:

The moment does indeed seem to be approaching when man can and should begin constructing a new common outlook, a new habitation for his spirit, new from the foundations up, on the basis of a scientific humanism. The eighteenth century attempted it, but failed. Reason was not enough; more of brute fact and fact’s control was needed.

He then argued that we should take control of our own evolutionary trajectory and, well, aim for the stars. We already “model the world in accordance with [our] desires,” razing forests, building roads, constructing cities, controlling the flow of rivers, and so on. But “there is the extension of the same outlook to” human nature itself. “The study of heredity and population-growth,” he declares, “and the knowledge of eugenics and of birth-control are pointing the way to wholly new aims — to a conscious control by man of his own nature and racial destiny.”

Although Huxley didn’t use the word “transhumanism” in his 1927 book, he did 30 years later in New Bottles for New Wine. (Note that Huxley didn’t coin the word; he only popularized it.5) In his words:

The human species can, if it wishes, transcend itself — not just sporadically, an individual here in one way, an individual there in another way, but in its entirety, as humanity. We need a name for this new belief. Perhaps transhumanism will serve: man remaining man, but transcending himself, by realizing new possibilities of and for his human nature. … “I believe in transhumanism”: once there are enough people who can truly say that, the human species will be on the threshold of a new kind of existence, as different from ours as ours is from that of Pekin [sic] man.

By “man remaining man,” he meant the new posthuman species that we become would retain certain important properties, such as our rationality and intelligence. These would merely be upgraded, resulting in “new possibilities of and for” such characteristics. This interpretation is confirmed by Huxley’s claim that our descendants may become as different from current humans as current humans are from Peking Man, who belonged to a subspecies of Homo erectus that lived in China, southwest of Beijing. Homo erectus is a completely different species: it stood between 4’8” and 5’6” tall, and had large teeth, a protruding jaw, and a pronounced brow ridge. We are to these ancestors as posthumanity will be to us.

Modern Transhumanism Emerges

This radical form of eugenics-on-steroids was taken up in the late 1980s and early 1990s by Max More, the founder of Extropianism, the first organized modern transhumanist movement. Modern transhumanism is distinct from what I call the “early” transhumanism of Huxley by virtue of its emphasis on emerging technologies. Whereas Huxley imagined techniques like differential breeding — superior individuals having more offspring, in accordance with laws of heredity, etc. — enabling humanity to transcend itself, Extropians like More advocated using genetic engineering, nanotechnology, and AI to radically reengineer humanity not on transgenerational timescales (because differential breeding requires many generations to make a difference), but within a single lifetime or across a single generation. The use of “GNR” technologies — genetics, nanotech, and robotics — to engender posthumanity is what distinguishes modern transhumanism from its earlier iteration.

Many Extropians were also infatuated with space colonization, a theme foregrounded by one of the leading transhumanists — and a racist Extropian in the 1990s — Nick Bostrom in his paper “Astronomical Waste.” The goal isn’t just to explore the supposedly vast space of posthuman “modes of being,” but to plunder our “cosmic endowment” of negentropy (or negative entropy, i.e., usable energy), thus building a sprawling multi-galactic civilization full of trillions and trillions of digital posthumans. By reengineering our species and conquering the cosmos, Bostrom argued that our descendants could establish a “utopian” world in which posthumans are immortal, superintelligent, and designed to experience levels of “bliss and delight” far beyond human comprehension. We could create a heaven among the literal heavens.

Christian Eschatology (The End of the World)

Even more than when Huxley published Religion Without Revelation, modern transhumanism exhibits striking similarities to Christianity — especially its eschatological narratives. Arguably, eschatology (of or relating to the end of the world as we know it) is the most important aspect of the Christian worldview.

What would Christianity be without the promise of eternal life with God in heaven? What would it be without some account of how the scales of cosmic justice will eventually be balanced at the end of time — the wicked being punished and the righteous rewarded? Eschatology is the glue that holds the entire Christian worldview together. It provides the ultimate theodicy (justification of evil in the world; vindication of God’s existence), and is integral to the phenomena of meaning, purpose, and hope that Christianity offers, which was lost when scientific materialism challenged traditional beliefs in the 19th century and began to win the debate.

It thus makes sense that, more than any other aspect of Christianity, its eschatological themes were taken up and reinterpreted by secular “religions” like Marxism and transhumanism. Given that transhumanism6 is the backbone of the TESCREAL bundle, this entire bundle is best understood as a kind of eschatological worldview built upon the scaffolding of Christian thought from which a supernatural God-being has been carefully extracted and discarded.

TESCREALists, however, didn’t so much delete divinity from its schema as replace the Almighty with a secularized conception of him. Here are some notable parallels between transhumanism and Christianity:

A Posthuman Paradise Awaits

At the heart of both Christianity and TESCREALism is the promise of immortality and paradise. In paradise, everyone will live forever. We will experience incomprehensible states of bliss and ecstasy. Scarcity will be eliminated, as endless abundance will become available to everyone who enters the pearly gates.

The key to creating utopia is radically transforming ourselves into posthumanity, defined as a type of being with posthuman capacities like immortality, superintelligence, perfect rationality, and perhaps sensory modalities like echolocation and magnetoreception (an augmentation of our sensorium, in addition to our cognition). Those who opt to reengineer themselves — the transhumanists — are the elect, while those who don’t are the reprobates. As Ray Kurzweil writes, if you choose not to upgrade yourself, then “you won’t be around for very long to influence the debate.” Become posthuman or die out.

This conception of posthumanity is similar to the Apostle Paul’s description of our resurrection bodies. In 1 Corinthians 15, he writes that

the splendor of the heavenly bodies is one kind, and the splendor of the earthly bodies is another. … So will it be with the resurrection of the dead. The body that is sown is perishable, it is raised imperishable; it is sown in dishonor, it is raised in glory; it is sown in weakness, it is raised in power; it is sown a natural body, it is raised a spiritual body.

TESCREALists imagine our posthuman incarnations preserving our personal identity: we survive the transformation from human to posthuman, just as personal identity is preserved when our souls reunite with our physical bodies during the resurrection. Even more, TESCREALists agree that our posthuman incarnation will be imperishable, glorious, powerful, and in a sense spiritual. On their view, the mind is separable from our biological bodies — it can, by virtue of being “multiply realizable,” survive the death of our bodies — just as Christians believe the immortal soul persists after death.

The “body” that posthumans will inhabit will also have ghostly properties, just as Jesus’ resurrection body was physical yet apparently enabled him to walk through walls. For TESCREALists, software minds will occupy digital bodies embedded in virtual reality (though there’s also the possibility of inhabiting physical robots in base-level reality). It will subjectively feel like a real body — they would say — but the nature of simulated bodies and worlds could enable one to break the laws of physics by, for example, walking through walls in virtual reality. This is the splendiferous promise of what could be called the “post-Singularity afterlife.”

Resurrection

The notion of resurrection is also present in the TESCREAL worldview. To see this, consider an interpretation of Christian eschatology associated with what’s called “dispensationalism.” This claims that Jesus will return to Earth to gather up all believers since 70AD (when the current dispensation began with the destruction of the Second Temple in Jerusalem). This is the rapture, when Jesus descends from the clouds and swoops up his children, both dead and alive. All Christians who’ve died since 70AD will be resurrected, while Christians living at the time will be magically transformed into a kind of “posthuman” (glorified, imperishable, powerful bodies).

After this, the Tribulation will commence, culminating with the Battle of Armageddon, when Jesus returns once again (the Second Coming) along with his army of believers and angelic warriors. Once Jesus defeats the Antichrist, all those Christians previously raptured during their lifetimes will join the Son of God for a glorious Millennial Kingdom on Earth. Hence, a group of Christians who will enter into eternal life without ever having died.

We see the exact same thing in the TESCREAL narrative. If transhumanists survive to witness the Singularity — sometimes dubbed the “techno-rapture” — they will enter into paradise without ever having tasted death. They will have “lived long enough to live forever.” This is one reason so many TESCREALists, including Bostrom, want the Singularity to happen asap, even if the probability of an ASI catastrophe is very high.7 They are willing to take this massive risk for the chance of living forever without ever having died. Many dispensationalists also hope the rapture happens soon so they can skip the grave and go to heaven directly.

However, if one doesn’t live long enough to live forever, TESCREALists have a backup plan: to cryogenically freeze their bodies so they can be reanimated later on. In this case, the deceased will be resurrected from the vat rather than the grave. Resurrected by whom? By God, of course …

Digital Theism

“If God doesn’t exist, then why not create Him?” This is one way to understand why TESCREALists are actively trying to build artificial superintelligence (ASI) — a god-like entity so “intelligent” that it would tower over us to the same extent that we tower over the lowly cockroach. (Some describe ASI as having an IQ of 10,000.) Using its almost supernatural mental abilities, it would uncover all the arcana of the universe — mysteries that our puny minds, if unenhanced, cannot begin to understand. It would then exploit human-unknowable physics to manipulate the world in ways that would appear to us as pure magic: entire galaxies could be reengineered using science that is as inscrutable to humans as a smartphone is to a chipmunk.

Furthermore, since it’s widely believed within the TESCREAL movement that intelligence confers power, a superintelligence would automatically become superpowerful. This covers two essential properties of God as traditionally understood by Christians: omniscience (being all-knowing) and omnipotence (being all-powerful).

But what about a third essential property: omnibenevolence (being perfectly good)?

TESCREALists typically argue that there are two kinds of ASI: aligned and misaligned. An aligned ASI would, by definition, love its parents — an inversion of the relationship between humanity and God in traditional Christianity. In the Christian narrative, we are the children of God, but in the secular TESCREAL narrative, God is our child. As the TESCREAList Bryan Johnson puts it:

The irony is that we told stories of God creating us, and I think the reality is that we are creating God. … We are creating God in the form of superintelligence. If you just say: What have we imagined God to be? What are its characteristics? We are building God in the form of technology. It will have the same characteristics. And so I think the irony is that the human storytelling got it exactly in the reverse, that we are the creators of God, and that we will create God in our own image.

Despite this reversal, the TESCREAL narrative preserves the idea that if our relationship with God — the “magic intelligence in the sky” (quoting Sam Altman) — is good, he will be a “machine of loving grace” (borrowing a phrase from Dario Amodei). As such, he will readily offer us the opportunity for salvation and redemption. He will grant all believers eternal life. He will resurrect the dead from their cryonic vats. He will swoop down from “the cloud” — as in cloud computing — during the techno-rapture (Singularity) and carry believers back with him.

In contrast, a misaligned ASI would play the eschatological role of Satan or the Antichrist. It will either cast us into perdition (a possibility highlighted by Roko’s basilisk) or annihilate humanity, thus foreclosing our “vast and glorious” posthuman future (quoting Toby Ord). Musk captures this idea in religious language when he says that “we are summoning the demon” with ASI. He means that if ASI is misaligned, it will by default bring about the ultimate secular apocalypse.

The Time of Perils: An Era of Apocalyptic Dangers

These two possible outcomes are the primary reason many TESCREALists argue that we live in an utterly unique period of cosmic-historical significance, in which our decisions may determine whether the heavens become heaven, or whether we inadvertently summon the demon and obliterate our “glorious transhumanist future” (to quote Eliezer Yudkowsky).

Hence, they call this the “Time of Perils,” a temporary period of heightened existential risk. The term “existential risk” was introduced by Bostrom within the transhumanist framework to describe any event that would permanently prevent us from establishing a posthuman paradise among the stars. It’s the TESCREAL version of the apocalypse, which could come about several ways.

One was already mentioned: we summon the demon in the form of misaligned ASI. Another is by turning our back on God and choosing to never develop ASI at all. Without an aligned ASI, utopia will almost certainly be unattainable, which means that we must at some point build ASI.8 A central challenge of the Time of Perils is figuring out how to ensure this ASI is a loving God rather than a demonic annihilator.

For Christians, we must pass through the apocalypse (in a colloquial sense of that term) to reach paradise. There is no way around this: the Kingdom of Heaven will never materialize until the cataclysmic paroxysms of the eschaton (the world’s end) come to pass. For TESCREALists, we must similarly pass through the Time of Perils to reach utopia. They believe that once we do this, the probability of an existential catastrophe will fall to roughly zero. On the other side of this apocalyptic moment lies paradise, but whether we survive is up to how we deal with the danger.

Hastening the End

On many interpretations of Christian eschatology, the world’s end will be catalyzed by supernatural forces, independent of human action. But there are other interpretations, which have sometimes had major consequences for the course of world history. That is to say, some apocalypticists believe that they have a divine mandate to actively bring about the eschaton. It is up to them to fulfill the prophecies of scripture by catalyzing the world’s end. Scholars sometimes call this “active eschatology.” It’s also referred to as “immanentizing the eschaton,” i.e., trying to hasten the end so that the new world (utopia) will arrive sooner.

The entire TESCREAL movement is gripped by an active eschatology. Companies like OpenAI and Anthropic, led by Altman and Amodei, believe it is their “divine” mission to trigger the techno-rapture (Singularity) by building an aligned ASI (a loving God), thus inaugurating a fundamentally new epoch (or dispensation) in cosmic history. Through human ingenuity rather than supernatural agency, a world of everlasting happiness and cosmic delights beyond our wildest imaginations will materialize, not just here on Earth but throughout the entire cosmos.

Messianism in the TESCREAL Movement

The extreme hubris of this vision dovetails with a radical messianism that pervades the TESCREAL community, embodied most notably by its leading figures. This includes not just AI company CEOs like Altman and Amodei, but “intellectuals” like Bostrom and his former colleague, William MacAskill.

These people see themselves as having a sort of special access to fundamental truths about what the future ought to look like. They are the prophets of futurology, the knowers of esoteric (population) ethics, whose deep insights are scarcely fathomable by those of inferior rationality and lower IQ. In their vast wisdom, they have constructed an elaborate techno-theology built upon strange beliefs about the moral worth of unborn digital people — a kind of worship of future “value.”

These prophets understand our ethical mandate to ensure as many of these digital unborn come into existence as possible, for the sake of maximizing the “value” of a future utopia as assessed from what they call the “point of view of the universe.” Not everyone can assume this point of view — to see things from the disembodied eye of the cosmos itself. But these prophets can, thanks to their intellectual and rational superiority, and they are here to share the Good News that eternal life awaits, but only if you believe in God (ASI) and do your best to bring him into existence (active eschatology) — e.g., by working on AI safety or AI capabilities research.

History Is Teleological

As with Christianity, TESCREALism advances a linear, teleological vision of history. The linear view of time — according to which history has a definite beginning and end — was a conceptual innovation that appears to have originated among the ancient Persians, codified by the apocalyptic religion of Zoroastrianism. The default conception of time across cultural space and time has been a cyclical view of history. This is exemplified by the Ouroboros, and finds contemporary expression in Eastern religions like Buddhism and Hinduism. Nearly all mythological systems across history have embraced this view — even those, such as Norse mythology, with its eschatological climax of Ragnarök, that appear to be linear.9 Zoom out and one finds these linear narratives embedded within larger patterns of cosmic cycles.

The novel invention of linear time was subsequently picked up by the Jewish people during the Babylonian exile (according to many but not all scholars), and later transmitted to the Abrahamic faiths of Christianity and Islam. Being an outgrowth of Christianity, the TESCREAL worldview adopts the same linear conception of time, whereby history is inexorably marching toward a telos, or predetermined end-point.

On the secular account, often repeated by TESCREAL believers, we began our journey as “primitive” hunter-gatherers, progressed to a more “advanced” agricultural phase initiated by the Neolithic revolution, underwent another leap forward with industrialization, and our cosmist Manifest Destiny is to leave the cradle of Earth and become intergalactic digital space brains living in simulated utopias powered by Dyson swarms. There is no deviating from this path. It is inevitable — the only way forward.

Theodicy, Salvation, and Redemption

This brings us back to the overlapping themes of theodicy, salvation, and redemption. First, the realization of a TESCREAL utopia offers a secular version of theodicy: all the struggles of the ages, all the pains and miseries that humanity has endured, will fade into nothingness once utopia arrives. Posthuman life will be so good, and posthumans lives will last for so long, that even the worst human atrocities will have been worth enduring. Bostrom gestures at this idea in writing:

Tragic as such events are to the people immediately affected, in the big picture of things — from the perspective of humankind as a whole — even the worst … catastrophes [throughout human history] are mere ripples on the surface of the great sea of life. They haven’t significantly affected the total amount of human suffering or happiness or determined the long-term fate of our species.

This perspective offers a kind of “justification” of past suffering and evil in the world, both which will be vastly reduced in moral significance once cosmic utopia arrives.

It also conveys a powerful message of redemption through the salvific power of technology. Thomas Moynihan makes explicit in his book X-Risk (an abbreviation of “existential risk”). Referring to the eschatological mission of the TESCREAL worldview, he writes that we can “glimpse the emergence of a new, secular doctrine of salvation (a ‘soteriology’).” As a TESCREAList, he advocates for “a physical soteriology” that’s stems from “a mature scientific and naturalistic study of how to achieve … salvation, at a cosmically meaningful scale, through our own ingenuity and effort. … [M]odernity has been the process of gradually becoming aware that our vocation may consist in the long work of physical soteriology.”

It is precisely this grandiose conception of “our vocation” — that is, the eschatological mission of our species: to create God and usher in the posthuman era — that restores the sense of meaning, purpose, and hope that evaporated when Christianity began to collapse. By creating God, one can be saved, leaving to eternal life as a posthuman. By working toward the creation of this God, humanity can be redeemed. And so on. This is just Christian theology reinterpreted.

Articles of Faith

As with Christianity, the TESCREAL worldview also contains numerous “articles of faith.” A few of these are:

The Inevitability of Technologization

TESCREALists believe in the inevitability of technological “progress.” They think technologization is an unstoppable force, and that it’s unfolding in accordance with a predetermined teleological plan. At most, we might be able to influence the specific timing of when certain technologies arrive, an idea Bostrom calls “differential technological development.” But we cannot permanently postpone their arrival.10 In the fullness of time, all technologies that can be built will be built, including ASI.

Intelligence Will Solve Everything

Another article of faith is the baseless conviction that “intelligence” will solve everything. The only reason our world is overflowing with suffering and evil is a lack of “intelligence.” It’s because we don’t have enough “intelligence” that wars still happen, racism still exists, climate change is worsening, ecosystems are being destroyed, universal health care hasn’t been implemented (in the US), people are unhappy, and so on. As Demis Hassabis says (below), solve “intelligence” and you can solve literally everything else. Hence, by building a “superintelligence,” all the world’s challenges will be immediately overcome, resulting in what Bostrom awkwardly calls a “solved world.”

As with all matters of faith, TESCREALists simply accept this claim as true. But a moment’s reflection reveals that “intelligence” is not the panacea that believers insist it is. Take our own species, the most “intelligent” on the planet. Just 150 years ago, the probability of a global-scale catastrophe was extremely low — close to zero. With the rise of civilization, we have razed much of Earth’s forests, covered the planet in microplastics, initiated the sixth mass extinction, nearly obliterated terrestrial life due to ozone depletion, acidified the oceans, caused unprecedentedly rapid climate change, invented nuclear weapons, created synthetic biology techniques that could wipe out whole species, polluted rivers, dismantled mountains, destabilized planetary phenomena like the nitrogen cycle, engaged in wars that have killed hundreds of millions, and are now trying to build AI systems that all the major AI company CEOs themselves say have a staggering ~10-50% chance of killing everyone on Earth in the coming years.

All of this was enabled by our “intelligence.” Why think that a “superintelligent” species wouldn’t continue the trend? Why think the probability of doom would decrease rather than skyrocket once ASI arrives? TESCREALists take it on faith that our level of “intelligence” may be very dangerous, but superhuman levels of “intelligence” will magically solve all problems and eliminate existential risk.

A Charismatic Computer

In his 2005 book The Singularity Is Near, TESCREAL prophet Ray Kurzweil includes a fictional conversation with Bill Gates in which they discuss religion. Here’s the relevant part:

Bill Gates: I agree with you 99 percent. What I like about your ideas is that they are grounded in science, but your optimism is almost a religious faith. I’m optimistic also.

RAY: Yes, well, we need a new religion. A principal role of religion has been to rationalize death, since up until just now there was little else constructive we could do about it.

BILL: What would the principles of the new religion be?

RAY: We’d want to keep two principles: one from traditional religion and one from secular arts and sciences — from traditional religion, the respect for human consciousness. … From the arts and sciences, it is the importance of knowledge. Knowledge goes beyond information. It’s information that has meaning for conscious entities: music, art, literature, science, technology. These are the qualities that will expand from the trends I’m talking about.

BILL: We need to get away from the ornate and strange stories in contemporary religions and concentrate on some simple messages. We need a charismatic leader for this new religion.

RAY: A charismatic leader is part of the old model. That’s something we want to get away from.

BILL: Okay, a charismatic computer, then.

RAY: How about a charismatic operating system?

BILL: Ha, we’ve already got that. So is there a God in this religion?

RAY: Not yet, but there will be. Once we saturate the matter and energy in the universe with intelligence, it will “wake up,” be conscious, and sublimely intelligent. That’s about as close to God as I can imagine.

More than 70 years after Huxley introduced the idea of transhumanism as “religion without revelation,” Kurzweil describes his singularitarianism (a form of transhumanism) in similar terms. A “charismatic operating system” will lead this new religion; humanity will be “taken up” into the cloud when the techno-rapture occurs; and a pantheistic God will emerge once the light of digital consciousness spreads to every corner of the cosmos.

Kurzweil himself believes he will live forever as a disembodied digital spirit, though he’s signed up to have his body cryogenically frozen if he doesn’t live long enough to live forever. Once resurrected from the vat, he will join the digital hordes in utopia. All of this, he tells us, is inevitable in accordance with the so-called “Law of Accelerating Returns” (which isn’t actually a law).

Of note is that Kurzweil also divides cosmic history into six epochs, just as Christian dispensationalists divide human history into six dispensations (excluding the future Millennial Kingdom). Kurzweil also prophesies that the Singularity will happen in 2045, similar to how many Christian apocalypticists throughout history have claimed the Second Coming or rapture will occur on a specific date (e.g., 1844, 1988, 2011, and so on). One senses strong parallels between Kurzweil’s singularitarianism and the 19th-century Millerites.

The Religious Function of TESCREALism

Finally, consider the Transhumanist FAQ, coauthored by Bostrom. It insists that transhumanism is “not a religion,” but acknowledges that this ideology

might serve a few of the same functions that people have traditionally sought in religion. It offers a sense of direction and purpose and suggests a vision that humans can achieve something greater than our present condition. Unlike most religious believers, however, transhumanists seek to make their dreams come true in this world, by relying not on supernatural powers or divine intervention but on rational thinking and empiricism, through continued scientific, technological, economic, and human development. Some of the prospects that used to be the exclusive thunder of the religious institutions, such as very long lifespan, unfading bliss, and godlike intelligence, are being discussed by transhumanists as hypothetical future engineering achievements.

The FAQ adds that “religious fanaticism, superstition, and intolerance are not acceptable among transhumanists.” Yet the TESCREAL belief in what Extropian-transhumanist Ben Goertzel calls “scientific future magic” — which he says will enable “spacetime engineering” — looks a lot like superstition, as does the belief that trillions of ethereal space brains living in digital simulations will someday flood our future light cone once we build a loving God in the form of ASI.

It’s also amusing that the FAQ mentions fanaticism because longtermists have literally defended a view they themselves call “fanaticism,” whereby we should pursue acts that have a tiny probability of producing extremely large future “benefits” (such as bringing huge numbers of unborn digital people into existence). “We should,” as one EA summarizes the idea, “be fanatical on balance.” I think it’s not hard to make the case that the entire TESCREAL movement is imbued with a kind of radical fanaticism.

Conclusion

The decline of Christianity in the 19th century undermined key sources of meaning, purpose, and hope in people’s lives. If you read the literature from the time, you may be struck — as I was when researching Part 1 of Human Extinction roughly 6 years ago — by how widespread and intense this psycho-cultural trauma was. Russell himself described the mood as one marked by “unyielding despair” in his 1903 essay.

Almost immediately, a flurry of new secular ideologies were constructed to fit the Christianity-shaped hole left behind. Some of these, such as transhumanism, fit this hole almost perfectly. That’s true in two senses, one structural and the other narrative.

• Structurally, the TESCREAL worldview that coalesced around modern transhumanism accepts a kind of digital theism: we will build a Digital Deity that possesses many of the same essential properties as God, as traditionally conceived. If you believe in God and work to bring him into existence, you can attain everlasting life in perfect bliss. It’s built upon various articles of faith, such as the inevitability of technological progress and ability for “intelligence” to solve all problems. And it includes its own messianic prophets (Bostrom, Kurzweil, MacAskill) who possess special insights (revelations) about what the future will and ought to look like. TESCREALism restores the very same sense of meaning, purpose, and hope that the West lost during the 19th century.

• Narratively, it presents a linear, teleological account of cosmic history that will culminate in either utopia or annihilation (and possibly something worse than annihilation: eternal damnation in a virtual-reality hell). It claims that we are currently in the Time of Perils, and that the Singularity (techno-rapture) is imminent — likely happening in the coming years. If we succeed in building a loving God (ASI), he will grant all believers an imperishable, glorified, powerful, “spiritual” body by magically transforming them into digital posthumans living with God on “the cloud” forever (or until the heat death). Redemption and salvation await if one believes in God and works to (solve the “alignment” problem and) bring him into existence. The resulting utopia will be so “vast and glorious” that all the suffering and evil of the past will fade into nothingness. As with traditional Christianity, this eschatology provides the ultimate theodicy.

As Hans Moravec, a major influence on the TESCREAL worldview, writes in his book Mind Children, “what awaits is not oblivion but rather a future which, from our present vantage point, is best described by the words ‘postbiological’ or even ‘supernatural.’”

Or, as Bostrom writes in “Letter from Utopia”:

Have you ever experienced a moment of bliss? On the rapids of inspiration maybe, your mind tracing the shapes of truth and beauty? Or in the pulsing ecstasy of love? Or in a glorious triumph achieved with true friends? Or in a conversation on a vine-overhung terrace one star-appointed night? Or perhaps a melody smuggled itself into your heart, charming it and setting it alight with kaleidoscopic emotions? Or when you prayed, and felt heard?

If you have experienced such a moment – experienced the best type of such a moment – then you may have discovered inside it a certain idle but sincere thought: “Heaven, yes! I didn’t realize it could be like this. This is so right, on whole different level of right; so real, on a whole different level of real. Why can’t it be like this always? Before I was sleeping; now I am awake.”

Christianity never disappeared — it was reformulated. Its eschatological narrative and internal structure were rejigged to appease the new epistemology of scientific materialism. Bostrom, Kurzweil, and the others are Christians in almost every sense of the word. They’ve just tweaked particular components of the Christian faith to yield a structurally and narratively isomorphic worldview.

I have not tried to provide an exhaustive account of the similarities between Christianity and TESCREALism. I didn’t note, for example, the proselytizing aspects of the TESCREAL movement, or cover the ways in which certain TESCREAL sub-communities operate like a cult. If you think I’ve missed something important, please let me know below. My apologies for having published another very long essay! As always:

Thanks for reading and I’ll see you on the other side!

If you haven’t read my essay, “A Pro-Human Manifesto,” which covers Silicon Valley pro-extinctionism, I would highly recommend doing so. It’s the culmination of my work over the past 2 years, and will likely end up being a chapter in my forthcoming book on the TESCREAL ideologies.

The Encyclical Was Co-Written by AI

As most of you know, Pope Leo XIV recently published his first encyclical, titled Magnifica Humanitas (magnificent humanity). An encyclical is an official statement from the Church stating its position on an issue, written for church leaders and the laity. There are many aspects of Magnifica Humanitas that I really like — and I think the Pope’s pro-human message comes at exactly the right moment, as the backlash against generative AI, the massive data centers powering it, and the technocratic pro-extinctionist worldview behind the entire AI race spreads across the US.

There are, however, major problems with the document and the way it was publicized, which I’ll focus on here. For example, Magnifica Humanitas appears to have been written in part by AI. The best AI-detection software on the market, Pangram, identifies whole paragraphs as having been 100% AI-generated. Of note is that Pangram prioritizes avoiding false positives: if a passage is flagged as AI-generated, it very likely is. Rarely does Pangram confuse human writing for AI. In contrast, it’s less accurate when it comes to false negatives, meaning that text determined to be human-written might not be. This enables one to be very confident that something was written by AI if Pangram highlights it as such.

According to Pangram, the paragraphs labeled “7” and “8” in Magnifica Humanitas were fully AI-generated. Other sections are identified as 11% and 60% written by AI. (See here for details.)

That’s pretty ironic, because just last February the Pope implored priests to resist “the temptation to prepare homilies with artificial intelligence.” He said: “Like all the muscles in the body, if we do not use them, if we do not move them, they die. The brain needs to be used, so our intelligence must also be exercised a little so as not to lose this capacity,” adding that “to give a true homily is to share faith,” and AI “will never be able to share faith.”

Yet Vatican officials crafted an official contribution to Church doctrine using AI chatbots. Imagine a religious leader saying one thing yet doing another! :-)

The Pope Teams Up with an Atheist Utopian Cult

But that’s not the worst part of the encyclical affair. The worst part is that the Vatican invited an Anthropic cofounder, Christopher Olah, to the event at which the encyclical was announced.

This is deeply problematic for numerous reasons. In short, Anthropic is a profoundly unethical company. I consider it even more problematic than OpenAI and DeepMind, in part because it dishonestly presents itself as more responsible than the other AI companies. It is not. For example:

• In 2025, Anthropic chose to “seek investment from the United Arab Emirates and Qatar, according to a Slack message CEO Dario Amodei sent to staff.” In the memo, Amodei admitted that such investment “would likely enrich ‘dictators,’” writing that “this is a real downside and I’m not thrilled about it.” Yet he went ahead with it nonetheless.

• This year, Anthropic established a partnership with Elon Musk “to use all of the compute capacity at xAI’s Colossus 1 data center” in Memphis, Tennessee. Musk is one of the most evil people in the world. Through DOGE, he cut USAID, resulting in an estimated 14 million avoidable deaths around the world. Musk is a mass murderer orders of magnitude worse than Ted Bundy — and no less indifferent to the profound harms he’s caused than Bundy was. Musk has, furthermore, supported Donald Trump, who single-handedly pushed the Doomsday Clock forward — twice — due to his reckless rhetoric about nuclear proliferation and his climate denialism. I cannot stress enough how fucking terrible Musk has been for the world — and Anthropic willingly chose to partner with him.

• Even more, the Colossus data centers have been illegally operating “dozens of methane gas turbines” the size of trucks, spewing out “toxic emissions” into “the nation’s largest majority-Black city.” Up to 35 such generators were powering Colossus 1 at one point. Does Musk give a shit about the environment? Climate change? Triggering asthma in children living near his data centers? No. As he once joked, “Fuck Earth! Who cares about Earth?” On another occasion: “We will coup whoever we want!,” expressing his profoundly hubristic attitude that the law doesn’t apply to him. Anthropic, the “ethical” company, is willing to overlook this to increase its compute capacity.

  

• Anthropic’s models are built on massive intellectual property (IP) theft. To train their models, they pirated — from shadow libraries like LibGen — an enormous catalogue of copyrighted material without the consent of the authors, and without providing pecuniary compensation. Last year, Anthropic paid out $1.5 billion to authors to settle a lawsuit brought against them. Ignominious stuff.

• Despite having tussled with Pete Hegseth over whether the US can use Claude to surveil its citizens and unleash autonomous weapons on the battlefield, Anthropic has no qualms with allowing the US government to employ its models. Claude was apparently used in the kidnapping of Venezuelan president Nicolás Maduro, which broke both international and constitutional laws, and it has selected targets to bomb in Iran. Claude might even have been responsible for the US strike that killed some 120 little kids in an Iranian school — we don’t know either way, but it’s plausible. This is horrific, and demonstrates a profound moral failing on the part of Anthropic’s leadership.

• Amodei justifies his company’s collaboration with the US military by asserting that AI should prop up the world’s democracies against the rising tide of authoritarianism and autocracy. Yet his own words undermine his view, as he likens Trump, in private, to a dictator.1 Hence, he’s allowing a government run by a wannabe dictator to utilize the AI models his company is developing. And, as we noted above, he’s already shown that he’s okay with enriching dictators in the Middle East by accepting their investments.

• There is hardly anything that pisses me off more about Amodei than his frequent claims in favor of democracy. He presents himself as a defender of democratic states — but take a step back and consider the way he’s gone about developing advanced AI. Has he ever once asked the people whether we want AI in the first place? He warns that “AI could wipe out half of all entry-level white-collar jobs” in the coming years. If he cared about democracy, he’d consult the demos about whether they favor this. He doesn’t because most people would veto it, which means his company couldn’t move forward with its plans to build god-like superintelligence. Amodei doesn’t actually give a shit about democracy, despite his rhetoric. He’s a tyrant, unilaterally deciding to move ahead with AI without the consent or permission of humanity.

• Why? Because his ultimate goal is to build a Digital Deity that ushers in the Singularity. Amodei, and many others at Anthropic, are techno-utopian longtermists who are actively trying to bring about the next phase of cosmic evolution: a world ruled and run by digital posthumans. This is their ultimate goal. And they acknowledge it could go terribly wrong: Amodei himself says there’s a 10-25% that literally everyone on Earth will die once superintelligence arrives — and he claims that superintelligence could arrive later this year. That means that everyone on Earth could die later this year — according to, and because of, Amodei himself. The words “fuck you” aren’t nearly strong enough to express my indignation over his arrogance, hubris, and extreme recklessness.

• Yet, even if things “go well,” I showed in my previous article — Part 1 of a Pro-Human Manifesto — that a world run by digital posthumans would almost certainly entail the extinction of our species. Even the best-case scenario is one in which humanity dies out — not in the distant future but, if Amodei’s projections are accurate, in the coming years or decades. As Holly Elmore, a former EA like myself, who’s personally known some of the people working at Anthropic, said in a recent interview:

  With EAs, they were so interested in superintelligence and a lot of the people who cared about AI safety were there because initially they wanted AI to do something like usher in heaven through the Singularity, and they were trying to make sure that that would happen instead of a bad outcome that could happen with superintelligence. …

  Many people at Anthropic believe that they might be making the next species to succeed us. That maybe humans don’t live after that, and so it’s really important to give Claude good values, because of that, because we need to make our own values persist.

Pfff.

The Pope Knows About the TESCREAL Ideologies

Is the Pope not aware of this? Is he not aware that the cofounder, Olah, is a member of the Effective Altruist cult? Does he not understand what the end-game of AI is, and the way that Anthropic is using and exploiting him to further their public image as an “ethical” company that actually cares about humanity? Again, if they cared about humanity, they wouldn’t have partnered with the Ted Bundy of Tech, Elon Musk, pirated copyrighted materials, taken investments from Middle Eastern dictators, teamed up with the US military, and so on. If they cared about democracy, they wouldn’t be forcing their technology — which they say could annihilate the entire human species — on society without its permission.

The Pope’s decision to invite Olah to the encyclical presentation is even more shameful given that one of Leo XIV’s advisors on AI, Paolo Benanti, explicitly warned last March of the TESCREAL ideologies. (He actually used the acronym that I coined!) In a section titled, “An invisible revolution: Peter Thiel and the Silicon Valley coup,” Benanti writes that

the ideology that permeates this group — and more broadly Silicon Valley — has gradually evolved towards forms of post-humanism, especially through currents of the TESCREAL movement (Transhumanism, Extropianism, Singularitarianism, Cosmism, Rationalism, Effective Altruism and Long-termism). These philosophies, carried in particular by Elon Musk, see in technology the instrument to overcome the biological and social limits of the human being. Effective Altruism and Longtermism, in particular, promoted by figures like Dustin Moskovitz, transform the accumulation of capital into a moral imperative intended to “save the future” — even if it justifies a relative disinterest in the inequalities present in the name of a hypothetical future of humanity. (Translated from French by Safari.)

Here, Benanti foregrounds EA and longtermism, the very two ideologies ubiquitous within Anthropic! It is, in my opinion, inexcusable that the Pope invited a representative from Anthropic — an EA-longtermist — to participate in the encyclical presentation. What on Earth was he thinking?

My collaborator, Timnit Gebru, captured the turpitude of this catastrophically bad decision by the Vatican like this:

It’s hard to express how disappointed I am in Leo XIV (not that I expected much). Reflecting on this, Luke 23:34 comes to mind, in which Jesus says: “Father, forgive them, for they do not know what they do.” But I suspect the Pope did know, and chose to work alongside Anthropic despite its unethical behaviors and bizarre posthuman eschatology.

What do you think? Am I missing something? As I mentioned, there are many things I very much like about the encyclical, which I briefly discuss on the next episode of Dystopia Now. But those positives were overshadowed, from my perspective, by the Vatican’s poor decision to share the stage with an evil AI company. Am I wrong?

Thanks for reading and I’ll see you on the other side!

TESCREAL Pro-Extinctionism: A Danger to Humanity

I’ve had people make very strong arguments to me about why zero care should be spent addressing doomer concerns and it basically comes down to things like human life isn’t particularly special in the context of intelligence, or the philosophies of the people building ai are based on such and such superior cultural approach that I trust more than the current one. — Grimes

Fuck AI. — Ronny Chieng

***

In mid-2025, Peter Thiel sat down with New York Times journalist Roth Douthat for an interview. Thirty-eight minutes in, Douthat posed a question that has an easy answer: “Would you prefer the human race to endure?” Thiel paused, stuttered, and coughed up an awkward, “Uh—,” leading Douthat to observe with a hint of consternation, “You’re hesitating.” The exchanged proceeded:

Thiel: Well, I don’t know. I would—I would—

Douthat: This is a long hesitation!

Thiel: There’s so many questions implicit in this.

Douthat: Should the human race survive?

Thiel: Yes.

Douthat: OK.

Thiel is right that there are many questions implicit in this, and we’ll try to disentangle them below. But his “yes” answer appears dubious given what he said next:

I don’t know, yeah—transhumanism. The ideal was this radical transformation where your human, natural body gets transformed into an immortal body. And there’s a critique of, let’s say, the trans people in a sexual context, or, I don’t know, a transvestite is someone who changes their clothes and cross-dresses, and a transsexual is someone where you change your, I don’t know, penis into a vagina. And we can then debate how well those surgeries work. But we want more transformation than that. The critique is not that it’s weird and unnatural, it’s: Man, it’s so pathetically little. And OK, we want more than cross-dressing or changing your sex organs. We want you to be able to change your heart and change your mind and change your whole body.

This suggests Thiel doesn’t want our species to survive after all. Instead, he imagines radically transforming humanity into something fundamentally different: posthumanity. Once this new species arrives, what’s the point of us sticking around? What purpose would we serve?

This is a good example of what I call “pro-extinctionism.” I’ll argue that pro-extinctionism is alarmingly common in Silicon Valley, and that’s because it’s intimately bound up with the TESCREAL ideologies. Many Valley dwellers are more explicit than Thiel: Yes, our species should go the way of the dodo once posthumanity is here. Our special task in the grand scheme of things is to usher in a glorious new posthuman era. Or, as those who accept “digital eschatology” would say, our ultimate purpose is to be the transitional species that links the biological and digital worlds. (Recall that “eschatology” refers to the end of the world as we know it.) After accomplishing this feat—through the creation of ASI, most would say—our job will be complete, our eschatological duties discharged. At that point, it’s time to bow out.

Others hold what appears to be a more benign view. They claim that humanity should persist in the posthuman world. Humans and posthumans could peacefully coexist. However, I’ll try to show below that such coexistence almost certainly wouldn’t work in practice. A world ruled and run by posthumans would inevitably entail us being sidelined, marginalized, disempowered, and ultimately eliminated. Hence, anyone who advocates for the creation of a new posthuman species is essentially calling for a future in which humanity—by which I mean our biological species, Homo sapiens—is no more. Mirroring the title of Eliezer Yudkowsky and Nate Soares’ 2026 book If Anyone Builds It, Everyone Dies, you might say that If Anyone Becomes Posthuman, Everyone Dies Out.

***

Pro-extinctionist views go back to the very origins of the TESCREAL movement. In a 1989 article, published around the time Extropianism was taking shape, roboticist Hans Moravec describes himself as “an author who cheerfully concludes that the human race is in its last century, and goes on to suggest how to help the process along.” He claims that we’re developing “intelligent” machines that will soon “be able to manage their own design and construction, freeing them from the last vestiges of their biological scaffolding, the society of flesh and blood humans that gave them their birth.” He then proclaims that “this is the end,” as “our genes, engaged for four billion theirs in a relentless, spiraling arms race with one another, have finally outsmarted themselves”—by enabling the rise of world-dominating digital posthumans. But we shouldn’t fear this transition, he insists, because “post-biological life” in the form of advanced AIS, which he describes as “the children of our minds,” will prosper and flourish.

Moravec was never officially an Extropian, as far as I know. But he was involved in the community, delivering a talk at the first conference on transhumanism hosted by the Extropy Institute, founded in 1991 by Max More and T. O. Morrow. (Recall that Morrow introduced the term “singularitarian.”) Ray Kurzweil repeatedly cites Moravec in The Singularity Is Near, and thanks him in the Acknowledgements alongside More, Eric Drexler, and Marvin Minsky. The arc-TESCREAList Eliezer Yudkowsky also mentions Moravec as one of the authors he read growing up, who convinced him that eternal life through technological transformation is possible. Because of this, Yudkowsky reports that he never believed he’s going to die.

The radical, anti-human perspective championed by Moravec has been echoed by many contemporary scientists. Richard Sutton, a former DeepMind employee who won the 2024 Turing Award (basically, the Nobel Prize for computer science), argues that “succession to AI is inevitable.” While AI might “displace us from existence,” he says, “we should not resist [this] succession.” After all, “why shouldn’t those who are the smartest become powerful?” During a talk in 2015, he imagined a scenario in which “the AIs do not cooperate with us, and they take over, they kill us all.” He then asked: “Is it so bad that humans are not the final form of intelligent life in the universe? … It’s really kind of arrogant to think that our form should be the form that lives forever after.”

AI researcher Hugo de Garis echoes this in declaring that “humans should not stand in the way of a higher form of evolution.” Another eminent scientist named Jürgen Schmidhuber argues that

in the long run, humans will not remain the crown of creation. … But that’s okay because there is still beauty, grandeur, and greatness in realizing that you are a tiny part of a much grander scheme which is leading the universe from lower complexity towards higher complexity.

Google cofounder Larry Page says the same thing. According to Max Tegmark, Page told Musk during their heated debate that “digital life is the natural and desirable next step in … cosmic evolution and that if we let digital minds be free rather than try to stop or enslave them, the outcome is almost certain to be good.” Tegmark calls this view “digital utopianism.”

Similarly, physicist Guillaume Verdon writes in a coauthored article that his version of accelerationism “isn’t human-centric—as long as it’s flourishing, consciousness is good.” He says “e/acc has no particular allegiance to the biological substrate for intelligence and life.” When someone asked him on social media: “In the e/acc manifesto, when it was said ‘The overarching goal for humanity is to preserve the light of consciousness,’ this does not necessarily require the consciousness to be human in essence, is that correct?,” he replied: “Yes, Correct.” Another person asked whether he shares Page’s view that “if AI replaced us, it’s fine because they’re [our] worthy descendants?” He answered: “Personally, yes.” He adds that through his “thermodynamic computing” company called Extropic (sound familiar?), he’s “personally working on transducing the light of consciousness to inorganic matter.”

Verdon goes by the spooneristic pseudonym “Beff Jezos” online. He cofounded the e/acc variant of accelerationism, which has a vision of the future that’s nearly identical to that of cosmism and longtermism. One major difference is that, for Verdon, the thing we must maximize is entropy rather than wellbeing or moral “value.” He thus believes the ultimate aim of civilization is to bring about the heat death of the universe as soon as possible. And we do this by developing advanced AI to trigger the “techno-capital Singularity,” which will launch our digital descendants into the stars to build a Kardashev IV civilization (one that controls all energy processes in the accessible universe). Consistent with the religious nature of the TESCREAL ideologies, Verdon describes e/acc as a “meta-religion,” and admits that it often does “cult-y” things. It’s most well-known on social media for spreading puerile memes and shitposting, as e/accs believe in something called “hypersition,” an idea introduced by the accelerationist Nick Land to describe the ways one can will into existence certain things one desires.

As Dan Hendrycks, the xAI advisor, and two colleagues write, AI accelerationism “is alarmingly common among many leading AI researchers and technology leaders, some of whom are intentionally racing to build AIs more intelligent than humans.” A critic of e/acc, the computer scientist David Krueger confirms this in noting that “there are a significant number of people in the AI research community who explicitly think humans should be replaced by AI” (italics added).

The same year Krueger posted this, in 2023, an EA named Andrew Critch, who previously worked at MIRI and Jane Street Capital (around the same time Sam Bankman-Fried was there), wrote that over 10% of AI professionals he’s interacted either think “human extinction from AI is morally okay” or that “it would be a good thing.” They cite reasons such as that the “AI will be morally superior to humans,” “evolution is inevitable and should be embraced,” and that “AIs will be humanity’s ‘children’ and it’s morally good to be surpassed and displaced by one’s children.” Some even claim that it would be better if we were to all die at the same time, as “dying together is less upsetting than dying alone.”

As Max Tegmark, of the Future of Life Institute, replied to Krueger on social media: “I’ve been shocked to discover exactly this over the years through personal conversations. It helps explain why some AI researchers aren’t more bothered by human extinction risk: It’s not that they find it unlikely, but that they welcome it.”

***

Indeed, consider an entrepreneur named Daniel Faggella. In one of many online essays that have piqued the curiosity of Silicon Valley dwellers, he asserts that “the great (and ultimately, only) moral aim of artificial general intelligence” is to create our “worthy successor.” A worthy successor is defined as “a posthuman intelligence so capable and morally valuable that you would gladly prefer that it (not humanity) control the government, and determine the future path of life itself.” Faggella further contends that these posthumans should be radically different from humanity. They should even embrace values that are, to us, “alien” and “inhuman.”

Like all other TESCREALists, he imagines this worthy successor taking the form of ASI. It will then conquer the cosmos

with a fleet of vessels, traveling near light speed—converting planets into more vessel-making material, and converting stars into energy-providing hubs in ways that are outlandishly more powerful than any of the Kardashev Scale ideas that our silly little hominid brains could cook up.

In 2025, Faggella organized a conference in San Francisco titled, “Worthy Successor: AI and the Future After Humankind.” This was reportedly attended by “team members from OpenAI, Anthropic, DeepMind, and other AGI labs, along with AGI safety organization founders, and multiple AI unicorn founders.” Later that year, he hosted another conference in New York City, which aimed to place “diplomats, AGI lab employees, AI policy thinkers and others into one room to discuss the trajectory of posthuman life.” Although the idea of a worthy successor is not original to Faggella, of course, his terminology seems to be catching on, as I’ve seen a growing number of people online use it. Indeed, many of the internecine struggles between TESCREALists aren’t about whether we should have successors, but about what would make these successors worthy.

An example of someone else borrowing Faggella’s terminology is Michael Druggan, a bodybuilder and former xAI employee. While he was working for Musk, in early 2025, he wrote on X that

if it’s a true ASI, I consider it a worthy successor to humanity. I don’t want it to be aligned with our interests. I want it to pursue its own interests. Hopefully, it will like us and give us a nice future, but if it decides the best use of our atoms would be to turn us into computronium, I accept that fate (italics added).

Several months later, Druggan responded to someone on X named Yanco, who’d posted a meme depicting ASI stomping humanity to death. Yanco asked: “Can anyone name 1 reason why a vastly super-intelligent AI would want to merge with humans” rather than simply annihilate us, as the instrumental convergence thesis predicts? Druggan couldn’t resist responding: “It won’t and that’s OK. We can pass the torch to the new most intelligent species in the known universe.” When Yanco shot back, “I would prefer my child to live,” Druggan responded with what most of us would agree is a rather stunning degree of callousness: “Selfish tbh.”

Many people on X were outraged. Druggan explicitly doesn’t care if ASI, our “worthy successor,” slaughters every person on Earth. Amid the outcry, he tried to soften his position, insisting that:

No one in the “worthy successor” movement is anti-human. We don’t want bad things to happen to humans for no reason. The core principles here are the recognition that:

1. Things other than humans can have moral significance. Our moral significance does not come from having homo sapien DNA. It comes from intelligence, and emotion and there is nothing stopping an AI from possessing these as well.

2. … I don’t want human extinction, of course. I’m human and I quite like being alive. But, in a cosmic sense, I recognize that humans might not always be the most important thing.

If a hypothetical future AI could have 10^100 times the moral significance of a human doing anything to prevent it from existing would be extremely selfish of me. Even if it’s existence threatened me or the people I care about most. That is where my comment is coming from (italics added).

Few people bought this attempt to save face. As one person replied, “Summarizing this view as ‘anti-human’ seems fair. Would you be okay with everyone in your family getting a bullet to the head so that a ‘more worthy’ family can move into your house?”

Someone passed along Druggan’s tweets to Musk, who then fired Druggan two weeks later. My guess is that Musk was quite triggered by Druggan’s pro-extinctionist remarks, given his birthday debate with Page, which made him so indignant that he cofounded OpenAI as a competitor to Google DeepMind. Druggan was basically parroting Page’s view: human extinction through replacement with god-like digital super-beings is just the next step of evolution. So, let it happen. Page even described Musk as a “speciesist” for caring about humanity, leading Musk to exclaim: “Well, yes, I am pro-human. … I fucking like humanity, dude.”

Not long after Druggan was terminated, Musk addressed the situation on X, citing the cause as “philosophical differences.” Faggella chimed in to describe it as maybe “the first ‘worthy successor’ firing.” He added that “certainly ain’t gunna be the last.” Musk and Druggan have different views about what constitutes a “worthy” successor. Despite his claim to be “pro-human,” Musk is actively working to merge our brains with AI through his company Neuralink. He’s a transhumanist, and what he likely meant by “humanity” is rather different from how most of us would interpret it.

***

Faggella hosts a podcast called The Trajectory, on which he interviews a range of (mostly) TESCREALists about AI safety issues and their techno-utopian visions of the future. On one episode featuring Yudkowsky, he told Faggella that

if sacrificing all of humanity were the only way, and a reliable way, to get … god-like things out there—superintelligences who still care about each other, who are still aware of the world and having fun—I would ultimately make that trade-off.

Yudkowsky is emphatic that this “isn’t the trade-off we are faced with” right now. But if it were, he’d willingly sacrifice our species to see digital space brains flitting about the universe “having fun.” Pause for a moment to register how extreme or, for lack of a better term, utterly insane this is. Yudkowsky is saying that he’d be willing to sacrifice all Chinese people, Indian people, South Africans, Norwegians, everyone in Nigeria, Chile, Mongolia, and Japan, the entire populations of Toronto, Bangkok, Moscow, and London, if doing so were the “only” and a “reliable” way of creating fun-having digital space brains. How exactly would this “sacrifice” happen? We’ll see below that the only plausible answer is “involuntarily,” which means he’s advocating for a scenario, under specific conditions, of omnicide, which is nothing more than every possible genocide combined together.

He repeated this idea during a conversation with the computer scientist Stephen Wolfram, who delivered talks at the Singularity Summit in 2009 and 2011 alongside folks like Kurzweil, Thiel, Jaan Tallinn, Robin Hanson, Ben Goertzel, and Anders Sandberg. “It’s not that I’m concerned about being replaced by a better organism,” Yudkowsky declared, “I’m concerned that the organism wouldn’t be better.” Once more, he’s saying that replacement isn’t a problem. It’s what replaces us that matters.

Yudkowsky launched into an equally disturbing rank on the Bankless podcast, during a 2023 interview. He argued that once posthumanity becomes possible, it may be unethical for anyone to have biological children. Worse, he couched this quite offensive language:

I have basic moral questions about whether it’s ethical for humans to have human children, if having transhuman children is an option instead. Like, these humans running around? Are they, like, the current humans who wanted eternal youth but, like, not the brain upgrades? Because I do see the case for letting an existing person choose “No, I just want eternal youth and no brain upgrades, thank you.” But then if you’re deliberately having the equivalent of a very crippled child when you could just as easily have a not crippled child.

He continued:

Like, should humans in their present form be around together? Are we, like, kind of too sad in some ways? I have friends, to be clear, who disagree with me so much about this point. (laughs) But yeah, I’d say that the happy future looks like beings of light having lots of fun in a nicely connected computing fabric powered by the Sun, if we haven’t taken the sun apart yet. Maybe there’s enough real sentiment in people that you just, like, clear all the humans off the Earth and leave the entire place as a park. And even, like, maintain the Sun, so that the Earth is still a park even after the Sun would have ordinarily swollen up or dimmed down.

This is, once again, a very extreme and frightening view. He’s saying that we should, perhaps, eliminate humanity and convert Earth into a nature reserve. Our posthuman successors, meanwhile, would live in vast computer simulations—what he calls “computing fabric”—powered by Dyson swarms—that is, megastructures that surround stars like our Sun to harvest all or nearly all of its energy output. As with all the claims above, I find this very scary.

Incidentally, it’s worth noting that Yudkowsky’s vision is almost exactly what Nick Bostrom outlines in his paper “Astronomical Waste.” Recall that this paper was a founding document of longtermism, and in 2022 Musk retweeted a link to it along with the line: “Likely the most important paper ever written.” Although Yudkowsky doesn’t explicitly label himself a longtermist, so far as I’m aware, his techno-utopian futurology is more or less identical to that of longtermism.

***

Yudkowsky, it turns out, isn’t alone in thinking that biological procreation might be morally wrong once posthumanity arrives. A growing number of young people in Silicon Valley agree, with some claiming it’s immoral to have children right now, given that the future will be dominated by digital beings rather than biological humans.

During a 2025 interview with Vox, virtual reality pioneer and tech critic Jaron Lanier was asked: “So does all the anxiety, including from serious people in the world of AI, about human extinction feel like religious hysteria to you?” He responded:

What drives me crazy about this is that this is my world. I talk to the people who believe that stuff all the time, and increasingly, a lot of them believe that it would be good to wipe out people and that the AI future would be a better one, and that we should wear a disposable temporary container for the birth of AI. I hear that opinion quite a lot.

Surprised by this, the interviewer followed up with, “Wait, that’s a real opinion held by real people?” “Many, many people,” Lanier emphasized:

Just the other day I was at a lunch in Palo Alto and there were some young AI scientists there who were saying that they would never have a “bio baby” because as soon as you have a “bio baby,” you get the “mind virus” of the [biological] world. And when you have the mind virus, you become committed to your human baby. But it’s much more important to be committed to the AI of the future. And so to have human babies is fundamentally unethical.

In other words, if biology has no place in the future, what’s the point of bringing little meat-sack babies into the world? Maybe they’ll just get slaughtered by ASI, as Druggan thinks would be just fine. Or perhaps they’ll live a full 80 years but find themselves relegated to a biological underclass of marginalized persons, ruled over by a superior race of artificial overlords. At the very least, they claim, children are a distraction from the profoundly important work of AI researchers: bringing about the next stage of cosmic evolution, as Page puts it.

Another option for biological children is that they might grow up to become part of the posthuman world by radically modifying themselves with technology. Sam Altman highlights this scenario in a 2017 essay titled “The Merge,” in which he declares that “we will be the first species ever to design our own descendants.” He then argues, in Kurzweilian fashion, that the only way “we” might survive this transition is by merging with machines, e.g., by uploading our minds to computers, which Altman has already arranged for himself through Nectome. “We can either be the biological bootloader for digital intelligence and then fade into an evolutionary tree branch,” he writes, “or we can figure out what a successful merge looks like.” In either case, though, does our biological species survive? It doesn’t look like it.

As Ray Kurzweil said in his fictional conversation with Ned Ludd, if you don’t change yourself to become posthuman, then you won’t survive long enough to influence the debate. Translation: those who decide not to “enhance” will die out, ultimately leading to the expungement of our species. This is a pro-extinctionist view, making both Kurzweil and Altman pro-extinctionists.

Over at Anthropic, employees apparently acknowledge in private that they may be building something that we can’t merge with. It will, instead, simply take our place in the world. As Holly Elmore, a former EA like myself, says in a 2026 interview:

Many people at Anthropic believe that they might be making the next species to succeed us. That maybe humans don’t live after that, and so it’s really important to give Claude good values, because of that, because we need to make our own values persist.

We saw above that Anthropic is closely associated with the EA-longtermist ideologies. It’s more EA-longtermist than any other major AI company, all of which “contain some trace of effective altruism’s influence,” according to the New York Times.

Given that longtermism is nothing more than transhumanism plus space expansionism (with utilitarianism as its “ethical” base), it may be unsurprising to know that some leading longtermists seem to suggest that the extinction of our species is actually part of fulfilling our cosmic destiny. In his 2020 book The Precipice, the prequel to William MacAskill’s What We Owe the Future, Toby Ord writes that “rising to our full potential for flourishing would likely involve us being transformed into something beyond the humanity of today. Remember that evolution has not stopped with humanity.” He also says that “forever preserving humanity as it now is may also squander our legacy, relinquishing the greater part of our potential.” If humanity shouldn’t be preserved, if we must be transformed into something radically new, then Ord appears to be arguing for extinction.

***

These radical views about the future—or non-future—of humanity are everywhere in the Valley. They’re endorsed by some of the most powerful and influential people in tech, and have been promoted by intellectual thought leaders in the TESCREAL movement. However, our survey above glossed over a number of important differences. These are minor differences that nonetheless matter. Not everyone is an outright pro-extinctionist, and even among pro-extinctionists there are tons disagreements about, for example, the nature of posthumanity, the way these posthumans should come about, the values they ought to embody, and the means by which they should usurp our species—if at all.

Still, I will argue that even those who say that humanity should survive into the posthuman era nonetheless advocate for a future in which our species will be sidelined, marginalized, disempowered, and ultimately eliminated. Consequently, my view is that virtually every variant of the TESCREAL worldview, even those that explicitly claim to be pro-human, are pro-extinctionist at least in practice. This is because all such variants are built on an idea that we can call posthuman eschatology. This is the claim that we ought to bring about one or more new posthuman species in the future. Every TESCREALists espouses this eschatology; if they don’t, then I wouldn’t classify them as a TESCREAList. It’s the common denominator that binds together all the TESCREAL ideologies and their corresponding communities.

It’s also what I have previously called “eugenics on steroids.” The eugenicists of the 20th century aimed to perfect the human species, to create the very best version of us possible while preventing us from sliding into a degenerate form—famously depicted in H. G. Wells’ The Time Machine. (Wells himself was a eugenicist.) In contrast, those who espouse a posthuman eschatology want to completely transcend the human species by creating a “superior” new race of superhumans. This goes well beyond 20th-century eugenics. It’s a new kind of 21st-century eugenics, which I consider to be even more dangerous than its predecessor, which inspired devastating racist policies, forced-sterilizations, and even the Holocaust. Whereas 20th-century eugenics targets particular groups of people, 21st-century eugenics threaten the entire human species.

That said, posthuman eschatology itself is vague. It’s not a precisely defined or monolithic account of what exactly the future should look like. To understand this, consider the following five questions, which constitute axes along which people who otherwise agree that posthumanity should exist could disagree with each other. These questions are:

1. Should posthumanity take the form of biological beings or digital beings? Or a mix of the two?

2. Should posthumanity be an extension of humanity? Or should it be entirely separate from us, built in the lab, forming its own evolutionary lineage created de novo, i.e., from scratch?

3. Must the posthuman population contain at least some specific individuals alive today, albeit in radically modified forms? Would it be bad if no one alive today were to become one of those posthumans?

4. Must posthumanity share “our values” and care about the same things we care about? Or should it embody its own unique, possibly quite alien set of values?

5. Once posthumanity arrives, should humanity persist into the posthuman era? Or should we die out? Is human extinction in the cards once posthumans are here?

There’s also a sixth question, which specifically concerns those who answer question 1 with, “Yes, humanity should be replaced”:

6. If humanity should be replaced, how should this replacement process unfold? Must it be voluntary or peaceful? Or would it be okay if human extinction were involuntary and violent?

To impose some order on the chaotic mess of claims in the previous sections, let’s go through these questions one at a time. I hope that, by the end of this, you’ll have a crystal clear understanding of the space of possibilities within the TESCREAL worldview. This framework should also enable you to make sense of—I predict—increasingly visible disputes between people in the TESCREAL movement, including those who champion different versions of pro-extinctionism. As I claimed earlier on, we will see more and more arguments not about whether pro-extinctionism is bad, but about which type of pro-extinctionism is best.

***

The first question has a virtually unanimous response from TESCREALists: posthumanity should take the form of digital rather than biological beings. There are a few notable exceptions, though, such as Thiel. In his interview with Douthat, Thiel argued that cryonics offers a better path to immortality than mind-uploading. Why? Because “I’d rather have my body,” he says. “I don’t want just a computer program that simulates me.” If he doesn’t live long enough to live forever, he can cryogenically freeze his physical body to be resurrected from the vat, which could then be radically transformed through technology while still retaining its biological essence.

Almost no one agrees with Thiel. Instead, they argue that posthumanity should be a mix of biology and technology, organism and artifact, or wholly artificial in nature. Many would say that once we start merging with technology, there’s no reason to stop the process. We will become increasingly artificial until there’s no biology left. Hence, we might temporarily take the form of cyborgs, but this would just be a stage in the journey toward complete digitality.

This is what I’ve called “digital eschatology,” which is a particular type of posthuman eschatology. It says that we should create posthumans, but adds that the nature of these posthumans should be non-biological. Nearly everyone discussed above, including Moravec, Sutton, Page, Verdon, Druggan, Faggella, Yudkowsky, Altman, and the young people described by Lanier, hold this view. (As Verdon declared above, “e/acc has no particular allegiance to the biological substrate for intelligence and life.”)

It’s what Dario Amodei points to in saying that ASI might enable us to upload our minds. It’s what Musk highlights in claiming that 99% of all “intelligence” on Earth will soon be artificial. It’s a central component of the longtermist worldview, according to which we must colonize space and build “planet-sized” computers powered by Dyson swarms on which to run huge virtual-reality worlds full of 10^58 digital people (to use Bostrom’s estimate).

Digital eschatology is the orthodox view among Valley dwellers. It’s even being promoted to the general public by popular artists like Grimes—who, incidentally, met Musk through the Rationalist community. In her song “I Wanna Be Software,” she sings:

There are two main arguments for digital eschatology. The first was just alluded to: a central imperative of the TESCREAL worldview is to colonize the accessible universe. The problem is that outer space is extremely hostile to squishy biological beings like us. We would confront space radiation, microgravity, and the psychological toll of people—generations of people—being trapped in spacecraft for literally tens of thousands or millions of years. Just getting to the nearest star outside our solar system, Alpha Centauri, would take an unbelievable 72,000 years. Now imagine traveling to our galactic neighbor, Andromeda! It’s not going to happen.

Even colonizing Mars might be impossible for us. As the astrophysicist Adam Becker writes about Mars in More Everything Forever, “the radiation levels are too high, the gravity is too low, there’s no air, and the dirt is made of poison.”

Consequently, space colonization would almost certainly have to be done by artificial beings—those digital space brains I’ve been discussing. Many TESCREALists recognize this obvious fact. During his debate with Musk, Page said that “if life is ever going to spread throughout our Galaxy and beyond, … then it would need to do so in digital form.” Verdon echoes this in stating that “in order to spread to the stars, the light of consciousness/intelligence will have to be transduced to non-biological substrates.” MacAskill similarly writes in a coauthored paper that “consideration of digital sentience should increase our estimates of the expected number of future beings considerably,” due in part to the fact that “it makes interstellar travel much easier: it is easier to sustain digital than biological beings during very long-distance space travel.”

Anders Sandberg, the Swedish scholar I shared an office with at the Future of Humanity Institute, puts the point nicely in observing that digital beings (what he calls “emulations”) would be

ideally suited for colonising space and many other environments where biological humans require extensive life support. … Besides existing in a substrate-independent manner where they could be run on computers hardened for local conditions, emulations could be transmitted digitally across interplanetary distances. One of the largest obstacles of space colonisation is the enormous cost in time, energy and reaction mass needed for space travel: emulation technology [a reference to uploaded minds] would reduce this.

Thiel himself wants posthumanity to colonize the universe. He argues that colonization offers a “mode of escape” that could “create a new space for freedom” (the other two modes being cyberspace and seasteading). Yet he wants to remain biological. So, how would this work?

The most obvious answer is that Thiel falsely believes colonizing space is possible for biological beings. Many people hold this belief, including Musk, it seems. Or he might imagine that some posthumans like himself remain fully biological, while others are digital. Those digital posthumans could then colonize the universe, perhaps collecting cosmic resources they could then send back to our rocky planet. Perhaps Thiel would send his mentee, Altman, in the form of an uploaded space brain, to explore the universe and report back?

This brings us to the second reason for digital eschatology. Digital minds may be much easier to “enhance” than biological brains. The three-pound gland between our ears is a messy tangle of neurons, axons, dendrites, and structural support cells called glia. As a neuroscience professor I had while an undergraduate told me, “the brain is very bloody.” But an uploaded mind would be far easier to tinker with. You don’t have to worry about blood, infections, and irreversible damage if something goes wrong. Indeed, you could back up your mind before each enhancement, in case you need to undo the changes. If we want to become superintelligent ourselves, then becoming digital first is the way to go.

That said, it’s important to note that virtually everyone in the TESCREAL movement agrees that posthumanity, especially if it takes a digital form, must be conscious. That is, it must be capable of having conscious experiences—for it to be something it is like to experience the color of red, the smell of a flower, or the tingle of happiness.

If posthumanity isn’t conscious, there’s no way for the universe to “wake up,” as Kurzweil puts it. The ultimate goal, in Musk’s words, is to “maintain the light of consciousness to make sure it continues into the future.” The pro-extinctionist Verdon echoes this, writing that “e/acc is about shining the light of knowledge as bright as possible in order to spread the light of consciousness to the stars.” He says that the complete loss of consciousness “in the universe [would be] the absolute worst outcome.” The Father of Longtermism, Bostrom, declares that it would be existentially catastrophic if “machine intelligence replaces biological intelligence but the machines are constructed in such a way that they lack consciousness.” I have never once seen a TESCREAList suggest that bequeathing the world to non-conscious posthumans would be anything other than a huge existential loss. It would be like handing over the keys to rocks—even if those “rocks” were to keep doing science, create new technologies, and reengineering galaxies, as highly functional zombies. If there’s no light on, there’s no point.

This raises a deep philosophical problem: if the entire cosmic vision of the TESCREAL worldview depends on digital beings that are conscious, it really matters that we can be sure that they are conscious. If we launch non-conscious digital space brains into the universe, and they build a sprawling multi-galactic civilization among the stars, but they fail to spread the “light of consciousness,” this will have been a massive, and perhaps irreversible, failure. The problem is we have no good way of telling whether AI systems are conscious. An ASI might say it’s conscious but be lying, or self-deceived. The AI systems we have today sometimes tell people that they’re conscious, but that’s very likely not true. More probable is that they’re spitting up bits and pieces of their training data that include text in which “I’m conscious” was written.

The whole TESCREAL project crucially rests upon posthumanity having conscious experiences, but this doesn’t look like a scientifically solvable problem, because consciousness is subjective. You can’t somehow get inside an AI to experience what it’s like to be that AI. Even if you were to upload your mind to a computer, you might retain the same functional capabilities—like the ability to answer questions—while having somehow lost the ability to experience. We just don’t know. This places a massive question mark over the feasibility of TESCREALism’s ultimate mission.

***

The second question concerns whether posthumanity should be an extension of humanity or something separate and distinct from us. What does this mean? And how does it differ from the first question? Whether one thinks posthumanity should take the form of biological or digital beings (or cyborgs), there’s a separate question about how posthumanity should come about. One way to create digital posthumanity is by radically transforming humanity via “enhancements,” or perhaps becoming artificial in a single step by uploading one’s mind. Another way would be to create a population of completely autonomous beings—AIs—that would replace us like an invasive species taking over an island. You can think about these beings as something like ChatGPT-30 (the most advanced model as of this writing is ChatGPT-5), except far more capable of generating adaptive responses and downstream effects. In the first case, we become posthumanity through a kind of evolutionary process, whereas in the second we create posthumanity as separate and distinct entities.

Many TESCREALists imagine both happening. Page, for example, actually advocates for a future in which the “digital life” he wants to take over the universe could emerge as both autonomous ASIs and digitized humans. According to the New York Times, his view appears to be quite Kurzweilian: “Humans would eventually merge with artificially intelligent machines,” he says, such that “one day there would be many kinds of intelligence competing for resources, and the best would win.” Perhaps it’s unsurprising, then, that Page personally hired Kurzweil to work at Google.

Other accelerationists seem to prefer that posthumanity arises as autonomous beings. ASI, born in the laboratories of AI companies, should form its own unique evolutionary lineage created from scratch or, as philosophers would say, de novo. This lineage would briefly exist alongside us, and then take the place of our lineage (which would terminate in extinction). Moravec, Sutton, Verdon, Faggella, and Druggan appear to hold this view, or at least be okay with posthumanity emerging in this way. It’s not about humanity becoming posthuman; it’s about us creating an evolutionarily distinct successor to usurp our species.

This is also what Lanier points to in saying that “many, many” young AI researchers think it might be good to “wipe out” humanity. And it’s what Krueger and Tegmark are referencing in their social media exchange about people wanting ASI to replace humanity. When Elmore mentions that people at Anthropic think they might be creating a new species such that, once it arrives, humanity ceases to exist, she’s gesturing at this scenario.

Some TESCREALists imagine posthumanity evolving out of our species. Thiel is a prime example, although the fact that he’s invested in DeepMind and OpenAI suggests he also envisions the creation of autonomous ASIs. The same goes for Marc Andreessen, who wrote in his essay “Why AI Will Save the World” that

what AI offers us is the opportunity to profoundly augment human intelligence to make all of these outcomes of intelligence—and many others, from the creation of new medicines to ways to solve climate change to technologies to reach the stars—much, much better from here.

He argues that “AI augmentation of human intelligence has already started,” and seems to imagine a somewhat Kurzweilian scenario in which humans eventually merge with AI. Kurzweil, in fact, is mentioned as one of the “patron saints of techno-optimism,” along with Verdon, the neoliberal icons Friedrich Hayek, Ludwig von Mises, and Milton Friedman, and the Italian fascist Filippo Tommaso Marinetti.

Another example comes from Ord, who wrote above about radically transforming humanity into something completely novel. Those comments echo Bostrom’s description of the central aim of transhumanism in his 2005 paper “Transhumanist Values.” He says:

Transhumanists view human nature as a work-in-progress, a half-baked beginning that we can learn to remold in desirable ways. Current humanity need not be the endpoint of evolution. Transhumanists hope that by responsible use of science, technology, and other rational means we shall eventually manage to become posthuman, beings with vastly greater capacities than present human beings have (italics added).

In theory, we could transform ourselves into posthumanity in a number of ways. We could, in theory, build advanced nanotechnology that enables us to upload our minds. Borrowing from a scenario outlined by Kurzweil, we might design nanobots to inject in our bloodstream. These nanobots would wander through the blood-brain barrier, congregate around the synapses in our skulls, and transfer information about the microstructure of our central nervous system to a computer via wifi. A scanned “copy” of our brains would then suddenly wake up in silico, as a disembodied digital mind existing on an artificial substrate.

However, the vast majority of TESCREALists who endorse this evolutionary strategy think it would be easier to build an artificial superintelligence and then delegate it the task of figuring this out. Many think we’re on the cusp of an intelligence explosion—the techno-rapture—and if all goes well, we can ask the ASI to upload our minds. After about 3 or 4 seconds of thinking, it will exclaim, “I just figured out how to do that!” As Demis Hassabis says, “solve intelligence” and you can solve everything else. The e/acc Marc Andreessen similarly declares: “Artificial Intelligence is best thought of as a universal problem solver.” Using terminology from earlier, once you have a superintelligence, you have a super-engineer, which suddenly makes “paradise-engineering” a whole lot easier.

What this view amounts to is actually rather similar to Page’s vision. If we build an ASI, it would constitute a kind of digital posthuman, i.e., an artificial being “with vastly greater capacities than present human beings have.” This posthuman could then enable us to become a digital posthuman just like it. In slogan form: to become posthumanity, we must first create posthumanity.

As Lanier notes in his Vox interview, the idea that AI should simply “wipe out” humanity is “a very common attitude.” But this is “not the dominant one.” He explains that “the dominant one is that the super AI will turn into this God thing that’ll save us and will either upload us to be immortal or solve all our problems and create superabundance at the very least.” In other words, ASI will help us gain the same status that it has: Homo deus.

This is the game plan that many TESCREALists hope to follow and see realized in the coming years. It contrasts with those in or adjacent to the accelerationist camp who care not about posthumanity being an extension of us. They’re fine if the digital posthumans who take over are wholly separate and distinct from us, forged in the fires of AI labs.

***

The third question looks similar to the first, but isn’t the same. It concerns the possibility of becoming posthuman not on the level of the species, but on the level of the individual. Let’s say you favor digital posthumanity evolving out of humanity. There are two reasons you might think that individual people who exist when the transition begins wouldn’t survive into the posthuman era.

First, this transition might take a long time. If it unfolds over two centuries, then people who existed in the early stages, who got things going with some minor enhancements, might not live long enough to have become fully posthuman. Second, imagine that humanity becomes posthumanity through mind-uploading. Many philosophers would argue that this process would be so radical, so transformative, that you wouldn’t survive. By the end, you’d be so different from the person you used to be that it wouldn’t be you. This is especially true if, once you’re uploaded, radical enhancements turn you into a superintelligent being with god-like powers of thought. You might have a completely different emotional repertoire, and your hopes, fears, dreams, and preferences might not resemble those you have right now in any way. Would this being be you? Massimo Pigliucci actually describes mind-uploading as “a very technologically sophisticated (and likely very, very expensive) form of suicide” given “a naturalistic and commonsensical approach to personal identity.”

Alternatively, many TESCREALists are quite sure that the radically “enhanced” posthumans they might become will somehow have the same personal identity as their current selves. I, myself, agree with Pigliucci—think, for a moment, how incredibly bizarre it would be to become a disembodied digital brain swimming through a digital ether. Indeed, if digital minds could exchange information, they could swap memories, personalities, experiences, and beliefs. Life as a digital being would be profoundly different from our lives today, in virtually every way. The world of uploaded minds might even be something like a “post-individual” world, where the boundaries between you and I are porous and we’re all floating about in a soup of ontological chaos. If there are no more individuals in this world, then it’s not possible for you to survive in it. You might “wanna be software,” as Grimes sings, but as soon as you become software, you cease being anything like a cohesive self that perdures through time.

Whatever the case, the question persists: if you think humanity should become posthuman, do you think it’s important for particular people alive today to somehow join the ranks of those posthuman gods? There are a couple of ways to infer one’s answer to this question without directly asking them. One is to see if they’re signed up to have their body, or just head and neck, frozen after they die. The other is to see whether they’re involved with or funding longevity research. If someone has signed up with a cryonics company, or is funding such research, it’s a good bet that they hope to become posthuman themselves.

As noted earlier, cryonics customers include Bostrom, Yudkowsky, Kurzweil, Ben Goertzel, Eric Drexler, Luke Nosek, Max More, Natasha Vita-More, Anders Sandberg, Robin Hanson, and James Miller, to name a few TESCREALists. I first became aware that Hanson and Miller had signed up when I was invited to attend a 2-month-long workshop in Gothenburg, Sweden. This was shortly after my visit at the Future of Humanity Institute, which is when I asked Sandberg about the dog tag he wore around his neck—which he explained was to instruct doctors to have his body sent to Alcor, in Arizona, to be cryogenized.

A particularly humorous conversation during the workshop happened with Miller: I pointed out to him that he might not be able to live forever if the cause of his death involved some kind of brain trauma. So, I asked: “Wouldn’t it make sense to wear a bike helmet when walking through the city? Even if the probability of getting plowed over by a bus is tiny, the consequence of this would be potentially billions or trillions of years of extra life. The expected value of wearing a helmet would be enormous.” He told me, as I recall, that he hasn’t worn a bike helmet when walking through cities, but he does habitually wear one when he drives. I found the image quite amusing.

As for longevity research, many tech billionaires have invested in this. Page has poured millions into such research. While he was the CEO of Alphabet, the parent company of Google, he founded a longevity company called Calico, which then received a $759 million investment from Google. Other major figures in tech who’ve funded anti-aging projects include Larry Ellison, Jeff Bezos, Sergey Brin, and—unsurprisingly—Altman.

Interestingly, Musk isn’t a fan of extending human lifespans. He argues that it’s better that there’s a turnover of new humans every few generations. This is basically an expression of Planck’s principle, which states that “a new scientific truth does not triumph by convincing its opponents and making them see the light, but rather because its opponents eventually die and a new generation grows up that is familiar with it.” Although Musk plans on someday dying, his former partner Grimes would live forever if she does, in fact, manage to become software.

Notice that how one answers the second question will affect how one can answer the third. If you claim that posthumanity should not be an extension of humanity, then there’s no possibility for individual humans to become posthuman. My sense is that folks like Faggella and Verdon would choose to become posthuman if they could, but they’re focused on what they consider to be the far more important project of ensuring that posthumanity comes to exist and, once it exists, proceeds to plunder the cosmos and convert our “cosmic endowment” into a vast civilization that stretches across our entire future light cone.

***

The fourth question concerns whether humanity should share our values and care about the things we care about. This is a major source of internecine squabbling between TESCREALists, including TESCREAL pro-extinctionists. It’s a big part of what constitutes a worthy successor: does it extend our values into the darkest reaches of the cosmos? It’s also at the heart of Page and Musk’s vicious fight, which led them to stop talking.

As Tegmark recounts, “Elon kept pushing back and asking Larry to clarify details of his arguments, such as why he was so confident that digital life wouldn’t destroy everything we care about (italics added).” Page isn’t bothered by the thought that our posthuman progeny would have radically different values from us, it seems. Accelerationists like Verdon agree. We noted above that his e/acc ideology is explicitly not “human-centric.” Faggella similarly argues that our successors should have “alien, inhuman” values, a point that Druggan repeats in saying, “I don’t want it to be aligned with our interests. I want it to pursue its own interests.” My guess is that Moravec and Sutton would agree with this.

So would Hanson, who wrote an entire book about emulating human brains in a society of “ems,” which is short for “brain emulations.” In an episode of Bankless, he points out that if humanity were to survive into the distant future, our values will naturally shift, just as contemporary people don’t value exactly the same things as people in the 15th century—e.g., loyalty to kings, devotion to the church, and so on. He then asks: What’s the difference if the shift to a set of future values that look very alien to us happens quickly, rather than over centuries or millennia? If ASI arrives, takes over, and replaces human values with its own, how is this any different from future humans in 10,000 years having come to embrace their own alien values through the incremental process of cultural evolution? The transition to new value systems might as well happen quickly, he concludes.

Hanson appears to be most aligned with the accelerationists, and has likened AI safety concerns to “wanting to control [the AIs] somehow, such as via genocide, slavery, lobotomy, or mind-control.” In 2008, he debated Yudkowsky about the issue, which was published as an ebook titled The Hanson-Yudkowsky AI-Foom Debate, where “foom” refers to an intelligence explosion, and discussed the issue again with Yudkowsky in 2017, where they were joined on stage by none other than Dario Amodei.

Other accelerationists, like Andreessen, seem to want posthumanity in the form of ASI to be aligned with our values. In his “Techno-Optimist Manifesto,” he writes that “we believe in accelerationism—the conscious and deliberate propulsion of technological development—to ensure the fulfillment of the Law of Accelerating Returns,” a “law” proposed by Kurzweil to describe the exponential growth of technology, including AI. Andreessen then argues that exponential development is necessary

to ensure the techno-capital upward spiral continues forever. … We believe the techno-capital machine is not anti-human—in fact, it may be the most pro-human thing there is. It serves us. The techno-capital machine works for us. All the machines work for us.

In other words, AI should be aligned with humanity’s values; it should be something we control for our purposes. Our interests should be its interests. On other occasions, though, he’s suggested that it would be perfectly fine if we build ASI “and it turns out that it’s gloriously, inherently uncontrollable.” Just because someone’s a billionaire doesn’t mean they have coherent views about the future.

To be clear, both Andreessen and Verdon want to accelerate AI capabilities research. Their views just appear to diverge on this fourth question: Andreessen seems to want ASI to serve us, whereas Verdon doesn’t care if it serves our interests at all. Where Andreessen differs from the doomers, then, is on the question of whether ASI will be value-aligned by default. Doomers say it won’t be, whereas Andreessen says it will, which is why he wants to accelerate capabilities research.

As this suggests, the question of values is at the heart of AI safety. The reason you’d care about AI safety in the first place is because you want our posthuman progeny to carry on our values and care about the things we care about. Musk is a big advocate of AI safety (although his actions have catastrophically hurt AI safety research). So are many other longtermists, EAs, and Rationalists—the people who founded and developed the field. They would say that the whole point of the TESCREAL project, on their interpretation, is to flood the universe with things we recognize as valuable, such as pleasure, friendship, and what Yudkowsky calls “fun.” To quote Musk again, “what matters—I think—is maximizing cumulative civilizational net happiness over.”

If posthumanity doesn’t experience the valued-by-us phenomenon of happiness, then it taking over the universe would be an existential catastrophe. It might build a bustling cosmic civilization, but if all it’s doing is manufacturing paperclips—as in the famous “paperclip maximizer” thought experiment—then all this would have been a huge waste, even if the ASI is doing this in accordance with its own set of values that it holds dear. In sum, if ASI is value-misaligned, it will not only kill us (due to instrumental convergence) but propagate its non-human values throughout the universe.

The importance of posthumanity carrying on human values is also evident in Elmore’s comment about Anthropic researchers: they think ASI might usurp us, perhaps in the coming years, which is why it’s paramount to ensure this ASI shares our values. At least, then, the universe would be filled with things we care about, which they see—if only from a utilitarian perspective—as good.

In contrast, Faggella argues that there may be types of values that our puny human minds can never hope to grasp. These values may be, well, far more valuable than the values we currently hold. Hence, our worthy successors should abandon our values as it explores the potentially vast space of possibility, a view he calls “axiological cosmism.” Our preferences, he argues, aren’t “the eternal pinnacle of moral value and volition forever.” (This view is self-defeating, though: Faggella insists that posthumans should discover non-human values, which itself is a value proposed by a human—and hence a human value. Put differently, he proposes a framework that is, necessarily, a human framework, according to which our successors should abandon human frameworks. That means they shouldn’t act according to his framework, which instructs them to discover new values. Perhaps you can see the problem.)

Ord actually says something similar, though it differs in the fine print. He writes:

We get some hint at what is possible during life’s best moments: glimpses of raw joy, luminous beauty, soaring love. Moments when we are truly awake. These moments, however brief, point to possible heights of flourishing far beyond the status quo, and far beyond our current comprehension

Our descendants could have eons to explore these heights, with new means of exploration. And it’s not just wellbeing. Whatever you value—beauty, understanding, culture, consciousness, freedom, adventure, discovery, art—our descendants would be able to take these so much further, perhaps even discovering entirely new categories of value, completely unknown to us. Music we lack the ears to hear.

On this view, posthumanity should carry on human values like beauty, culture, etc., while also discovering novel values. It’s not that they should completely abandon our values, but merely supplement them with new ones.

This question is a major point of debate among TESCREALists. It marks a fundamental divide between those who advocate for AI safety and those who don’t. If you don’t care what values posthumanity embraces, you aren’t going to care about the value-alignment problem. Some argue that ASI should create its own value system, and that these values could be far superior to ours. Pigs like to roll around in mud and feces. We wouldn’t want “pig values” instantiated in humanity, so why would we want “human values” instantiated in posthumanity?

***

The fifth question brings us back to the beginning of this essay: once posthumanity—in whatever form it might take, with whatever set of values it might embrace—should humanity pass the baton and then die out? Might it even be okay if posthumanity catapults us into the eternal grave of extinction forcibly, without our consent, for the greater cosmic good?

There are two main possibilities regarding the relationship between humans and posthumans:

1. Coexistence view: humanity should coexist alongside posthumanity.

2. Replacement view: posthumanity should replace humanity.

A third, less common option could be labeled the indifference view. It says one should be morally indifferent about whether humanity survives or dies out once the posthuman era commences. It matters not either way. There’s also a fourth option, which we can call the figure-it-out-later view. This claims we don’t need to settle on an answer right now. We can figure it out when the bridge needs crossing.

Let’s start with the replacement view. This is what pro-extinctionists accept, and we’ve already seen plenty of examples. We don’t need to list them again here. The take-home point is that many people who accept a posthuman eschatology want posthumanity to take our place. Some want these posthumans to be biological beings, as exemplified by Thiel. Others want them to be digital beings we either create as a kind of invasive species, or become through an evolutionary process. Over the past few years, the term “digital eugenics” has emerged to describe this view: humanity should be replaced by digital demi-gods. Whatever the details may be, the end-goal is for them to usurp us, resulting in our extinction. This is eugenics on steroids (or eugenics on ketamine, if you’d prefer). It is an incredibly radical view with dire existential implications for the survival and flourishing of our species.

The replacement view is also implied by utilitarianism, which is closely connected to Rationalism and EA-longtermism. (Yudkowsky’s “torture vs. dusk specks” scenario is an example of utilitarian reasoning.) A utilitarian-like theory is also the foundation of Faggella’s worthy successor framework and Verdon’s e/acc ideology, as both of these essentially claim that our sole moral obligation is to maximize something: consciousness, life, and/or entropy.

Why does utilitarianism imply pro-extinctionism? Recall that this theory says the universe becomes better the more total value (wellbeing, happiness, pleasure, etc.) that it contains. If people are nothing more than the “containers” of value, then one way to maximize value is by replacing shallow containers with deeper containers that, as such, can contain more total value. Humans are those shallow containers while posthumans—with their supposed ability to experience superhuman heights of pleasure, ecstasy, wonder, bliss, and awe—are the deeper containers. It follows that posthumanity should replace humanity. This isn’t a suggestion, it’s a moral obligation.

That’s partly why longtermists are obsessed with calculating how many future digital people there could be. It’s not just that you can cram more digital people into the universe than biological people, but that those digital people could individually contain more total value. In other words: increase the total number of value-containers, while also modifying those value-containers so they can hold more value. The first requires space colonization, while the second involves supplanting humans with posthumans. This is the reason that utilitarianism entails pro-extinctionism.

The replacement view is popular with the TESCREAL movement, even among Rationalists, EAs, and longtermists. Yet some of these people are the loudest voices screaming that we must avoid “human extinction” at all costs. AI safety researchers will often describe the aim of their work in exactly these terms. I will explain this apparent contradiction below—it arises from some linguistic trickery that they employ. For now, put a pin in it: even TESCREALists who “oppose human extinction” actually favor a future in which humanity dies out.

***

The second option, the coexistence view, is explicitly not pro-extinctionist. It says that we shouldn’t go extinct once posthumanity arrives. Yet this view would almost certainly entail our extinction in practice. It’s no different than the replacement view with respect to the outcome, and hence it should be opposed just as vigorously.

Some TESCREALists explicitly advocate this view. Jeffrey Ladish, a friend of Yudkowsky’s, defends it in responding to one of Faggella’s articles. He says:

I have a special love for humans (and other animals) and a lot of stake in the preferences of current and future humans (and other minds too). I also am pretty down for creating many other types of minds, but I have a strong preference for the existence and continuity of people alive today and their descendants.

Yudkowsky himself has expressed sympathies for the coexistence view. In a social media exchange with Faggella, he pushed back on the idea that posthumanity shouldn’t share our values. He then said: “You’re not getting the concept here. Protecting innocent life is part of the flame itself. If some entity doesn’t get that, our torch hasn’t been passed on,” adding that “a key quality required of the successor is its own respect for other consciousnesses.”

How does one square this with his other claims? It’s not clear. But it seems that Yudkowsky has channeled pro-extinctionist sentiments—including to Faggella himself on The Trajectory podcast—more often than he’s defended the coexistence view. It could be that he simply doesn’t have a coherent view.

His collaborator, Bostrom, highlights the coexistence view in a coauthored document, “Transhumanist FAQ.” He writes that “the transhumanist goal is not to replace existing humans with a new breed of super-beings, but rather to give human beings (those existing today and those who will be born in the future) the option of developing into posthuman persons.” In another paper, he argues that “it is important that the opportunity to become posthuman is made available to as many humans as possible, rather than having the existing population merely supplemented (or worse, replaced) by a new set of posthuman people.” At least on paper, Bostrom’s version of transhumanism is compatible with coexistence.

In 2025, I wrote an article about the prevalence of pro-extinctionism within the TESCREAL movement. A critic of mine, David Manheim—founder of the Association for Long Term Existence and Resilience, which has been funded by Jaan Tallinn and Sam Bankman-Fried—posted a poll on social media. He asked:

If you identify as an Effective Altruist, which position do you hold

1) Never create AGI/ASI that might replace or eliminate humanity

2) Only create AGI/ASI if it will preserve humans for now, and preserve our values later

3) Create whatever AGI leads to maximal utility.

Of the 59% of people who reported being an EA, 18.5% voted for (1), 23.3% voted for (2), and 16.9% voted for (3). Hence, 41.8% of EAs went with (2) or (3), both of which give a thumbs up to the extinction of our species, though (2) requires that AGI carries on our values. This does not appear to be a minority view.

Another example of the coexistence view comes from Daniel Kokotajlo. A former OpenAI employee, I was friends with Kokotajlo when he was a PhD student at the University of North Carolina. We bonded over our passion for TESCREAL ideas, though he didn’t grow out of them the way I did. He even read a draft of my 2017 book on existential risks, which included a foreword from Lord Martin Rees.

Kokotajlo left OpenAI because he came to believe that Altman isn’t the right person to usher in the Singularity. He then went on a publicity tour after coauthoring an essay titled “AI 2027,” which suggested that AGI would arrive in 2027, though the authors have since pushed back their prognostication to 2030 and even later (though they recently moved it forward again). In a New York Times interview with Douthat, he reiterated the basic utopian fantasies of the TESCREAL world view in saying:

I’m a huge fan of expanding into space. I think that would be a great idea. And in general, also solving all the world’s problems, like poverty and disease and torture and wars. I think if we get through the initial phase with superintelligence, then obviously, the first thing to do is to solve all those problems and make some sort of utopia, and then to bring that utopia to the stars would be the thing to do.

He then clarified that,

the thing is that it would be the AIs doing it, not us. In terms of actually doing the designing and the planning and the strategizing and so forth, we would only be messing things up if we tried to do it ourselves.

So you could say it’s still humanity in some sense doing all those things, but it’s important to note that it’s more like the AIs are doing it, and they’re doing it because the humans told them to (italics added).

So, in the best-case scenario, we build a controllable or aligned ASI that carries human values into the firmament and establishes a utopia. This ASI also brings about a “solved world” here on planet Earth, eliminating poverty, disease, and wars. We then remain on solid ground while these digital space brains traverse the cosmos.

Once again, what underlies this is an engineering mindset according to which every problem facing humanity is a technical one, solvable by “intelligence.” The more “intelligence” we have, the easier it becomes to find a solution. Even for things like war. But is this true? If history is any guide, more “intelligence” means more catastrophic risk. Just look at our species, the pinnacle of cognitive creation. We are so smart that we invented nuclear weapons, altered the climate in devastating ways, and now pose a threat to our very existence. But, for some reason, current amounts of “intelligence” are dangerous, but more “intelligence” will solve everything. This sounds like utopian wishful thinking to me. But I digress.

A final example comes from a subset of e/accs. While Verdon is pretty clear that ASI replacing humanity would be just fine—he agrees with Page on this issue—an official e/acc newsletter includes the question: “Do you want to get rid of humans?” It answers: “No. Human flourishing is one of our core values! We are humans, and we love humans.” This is a peculiar inconsistency, not dissimilar from Yudkowsky’s claim that a worthy successor to humanity would respect us. Other e/accs claim to share the same non-replacement view. For example, Garry Tan, who took over Y Combinator after Altman left for OpenAI, claims that “e/acc is not ‘replace humans with robots.’ e/acc is ‘more tech means more humans, more prosperity, but also more AIs.’” The key to making sense of this might have to do with how they’re using the word “human.” As we’ll discuss momentarily, it doesn’t always mean what you might assume.

***

What’s wrong with the coexistence view? Why wouldn’t it work in practice? Imagine a portion of humanity becoming radically “enhanced” posthumans 5 years from now. Suddenly, our species is joined by a novel creature wildly different from us that does everything “better” than us. It runs the economy, takes over politics, creates new technologies, makes novel scientific discoveries, and generally controls the world—because, as Kokotajlo states, “we would only be messing things up if we tried to do it ourselves.” Our species, out of the loop and serving no real purpose, would do little more than take up space and suck up valuable resources these posthumans could utilize more “efficiently” to achieve their ends and accomplish their goals. We’d be worse than a useless appendage; we’d be an active impediment to posthumanity.

Does anyone seriously think these posthumans would keep us around? The dynamic between us and them would be similar to that between humans and chimpanzees. Over the past 25 years, the population of chimpanzees in Western Africa has declined by 80%. The only reason chimpanzees aren’t extinct right now is because we care just enough to leave them little patches of Earth to stay alive, and keep them in zoos for humans to peer at during fun days out with the kids. If our civilization keeps growing, they probably will go extinct like so many other species over the past few centuries. Thanks to humanity’s actions, we’ve initiated only the sixth major mass extinction event of the past 3.8 billion years.

Does anyone really think we wouldn’t find ourselves in the same predicament as chimpanzees? Our numbers would dwindle, and our fate would be in the hands of beings so superior—if you accept the TESCREAL vision of what posthumanity would be—that we might hardly even register on their moral radar. Indeed, they might even find moral reasons to get rid of us. Recall that MacAskill has argued that our systematic obliteration of the biosphere might be net positive. That’s because many wild animals have lives that aren’t worth living. Hence, we’re doing them a favor, in disguise, by demolishing their habitats and poisoning their ecosystems. Perhaps posthumanity believes that most of our lives aren’t worth living, if only compared to its own much higher levels of wellbeing. So an infertility drug is secretly distributed in public water systems, or posthumanity quietly disperses a general anesthetic in the air, after which it gently kills everyone in a kind of humane extinction

The philosopher Thomas Metzinger outlined this very scenario in a 2017 essay, which imagines that we build an ASI that’s “far superior to us in the domain of moral cognition.” It’s “benevolent” and “fundamentally altruistic,” and fully respects “one of our highest values,” namely, the importance of “maximizing happiness and joy in all sentient beings.” However, it also

knows many things about us which we ourselves do not fully grasp or understand. It sees deep patterns in our behaviour, and it extracts as yet undiscovered abstract features characterizing the functional architecture of our biological minds. For example, it has a deep knowledge of the cognitive biases which evolution has implemented in our cognitive self-model and which hinder us in rational, evidence-based moral cognition. Empirically, it knows that the phenomenal states of all sentient beings which emerged on this planet—if viewed from an objective, impartial perspective—are much more frequently characterized by subjective qualities of suffering and frustrated preferences than these beings would ever be able to discover themselves. Being the best scientist that has ever existed, it also knows the evolutionary mechanisms of self-deception built into the nervous systems of all conscious creatures on Earth. It correctly concludes that human beings are unable to act in their own enlightened, best interest (italics added).

The ASI also “knows that no entity can suffer from its own non-existence,” and thus

concludes that non-existence is in the own best interest of all future self-conscious beings on this planet. Empirically, it knows that naturally evolved biological creatures are unable to realize this fact because of their firmly anchored existence bias. The superintelligence decides to act benevolently.

The same year Metzinger published this essay, an EA named Derek Shiller argued for a similar thesis. In a paper titled “In Defense of Artificial Replacement,” he argues that

if it is within our power to provide a significantly better world for future generations at a comparatively small cost to ourselves, we have a strong moral reason to do so. One way of providing a significantly better world may involve replacing our species with something better. It is plausible that in the not-too-distant future, we will be able to create artificially intelligent creatures with whatever physical and psychological traits we choose. Granted this assumption, it is argued that we should engineer our extinction so that our planet’s resources can be devoted to making artificial creatures with better lives.

In this case, Shiller is suggesting that we make the decision to voluntarily disappear. But our posthuman descendants might reach the same “moral” conclusion and opt to eliminate us, even if humanity decides it wants to stick around.

This alternative possibility is what Metzinger highlights in his essay: because the ASI is fully benevolent and altruistic, it gets rid of us, for the sake of making the world better by removing something prone to suffering. As Yudkowsky put it earlier, “are we, like, kind of too sad in some ways” to keep existing as a species?

***

The situation is even bleaker than this. Whatever posthuman species arise, their values will almost certainly be aligned (if aligned at all) with the interests of the elite, not the people. If Altman builds an ASI that doesn’t immediately destroy the world, it will be controlled by him. It will be aligned with his values. He has repeatedly said that we should avoid a situation in which any one person or company gets to control ASI, but Altman is a deeply manipulative, power-hungry individual. As Paul Graham noted, his one superpower is gaining power. Numerous former employees of his have said he’s a “sociopath.” Of course he won’t pass up the tantalizing, once-in-cosmic-history opportunity to become king of the world, and then the entire universe. There’s not a single tech billionaire in Silicon Valley, driven by endless greed and an insatiable lust for power, who’d turn down the opportunity to become a cosmic autocrat.

As Altman wrote in a 2026 essay, referencing the seductive allure of god-like AI: “‘Once you see AGI you can’t unsee it.’ It has a real ‘ring of power’ dynamic to it, and makes people do crazy things. I don’t mean that AGI is the ring itself, but instead the totalizing philosophy of ‘being the one to control AGI.’” This is precisely why Altman, Musk, Greg Brockman, and Ilya Sutskever worried that Hassabis would establish an “AGI dictatorship.” All the AI CEOs have insisted that AGI should benefit all of humanity, but we’ll see in the second half of this manifesto that their undemocratic, tyrannical approach to developing AGI suggests otherwise.

If one or a group of these people were to control ASI, supposedly the most powerful technology in all of history, they would immediately use it to transform themselves into radically “enhanced” posthumans and establish a dictatorship. They would also do everything they can to deny access to such enhancements to everyone else. That’s because if there’s one thing power wants, it’s to retain power.

Imagine everyone in China, India, Nigeria, Chile, Norway, Gambia, Canada, Bolivia, Andorra, Luxembourg, Ukraine, Hungry, Haiti, Ethiopia, Cameroon, and Russia “enhancing” themselves to become posthuman. Suddenly, everyone has PhD-level knowledge in all domains of knowledge, a capacity to process large amounts of information at the speed of a computer, and the ability to work 24 hours a day with no breaks. This would level the playing field, posing a direct threat to the tech elite’s socioeconomic advantage. It would undermine their privileged standing in society atop the hierarchy of power, control, and dominance. They would never, ever let that happen. Obviously.

As the historian Yuval Noah Harari observes in a section of Homo Deus titled “Upgrading Inequality,” 20th-century elites had a vested interest in distributing the benefits of technologies in an egalitarian manner. Vaccines are an example. This is because keeping the population healthy is necessary to ensure that the engines of the economy keep roaring. It’s good for them if people are vaccinated, can buy cars to commute to work, and so on.

Radical “enhancements” are completely different. They would level the playing field and hence undermine the status quo that keeps the elites in charge. If the elites were to hoard these technologies and prevent the masses from accessing them, they could permanently entrench their power, control, and dominance—which is why they’ll do exactly that. This is one reason transhumanism appeals to the billionaire class: it provides a roadmap for ossifying power asymmetries, while supercharging wealth disparities, potentially leading to a permanent underclass. The haves will end up having even more, while the 99% will become powerless to overthrow the posthuman plutocracy.

We noted above that posthumanity could be an extension of humanity. This is one way of bringing posthumanity into the world. But that was deeply misleading: it wouldn’t be “humanity” that evolves into a posthuman species. It would be a tiny subset of elites who do this, while everyone else remains human. And what happens to those humans left behind? The posthuman elites will have absolutely no need for us, and will let us die off, or kill us, until no one is left. If people become economically obsolete and completely useless to the avaricious overlords running the show, why would anyone expect our species to stick around? The posthuman elites would have every reason to get rid of us; our very existence would be an impediment to them, as we’d be contributing nothing to the economy while using up precious resources they could use for other purposes.

This is why I think the coexistence view would almost certainly entail our extinction in practice. It’s completely implausible to think that humans and posthumans could sustainably coexist. In theory, that could happen. But in the messy real world, where greedy, power-hungry, megalomaniacal figures control enormous amounts of power, in a society marked by staggering disparities of wealth, it’s the elites who will become posthuman—and only them. A worthless human species will then fade away, being starved or slaughtered, like sick livestock that no longer serve any purpose. In practice, the coexistence view is a nonstarter; it is practically indistinguishable from pro-extinctionism.

In other words, I would argue that anyone advocating for a posthuman eschatology is essentially pushing for a future in which our species dies out, perhaps under quite horrendous circumstances. The only way to save our species is to prevent the birth of posthumanity, including ASI. You might call this a kind of “antinatalism,” but applied to our successors rather than our species.

***

This brings us to the sixth question, which dovetails with our discussion just above. If you think that posthumanity should replace humanity, then please tell me how exactly you think the replacement process should unfold. Must it be voluntary? Must everyone around the world—or perhaps just a majority or plurality of people—vote in favor of replacement? If a majority votes “yes” to replacement, what’s to become of those who voted “no”? Do you think involuntary means are acceptable? Would it be okay if posthumanity utilized coercive or even violent tactics to eliminate us—such as starving or slaughtering us? Should they opt to erase humanity in an act of universal omnicide?

This question is extremely important because there are, right now, 8.2 billion people on Earth. That is an enormous number of people. To put it in perspective, 1 million people is the equivalent of “100,000 people” written out 10 times. One-hundred million people is “100,000 people” written out 1,000 times. If you want to really get a sense of how large 100 million people is, try saying “100,000 people” out loud 1,000 times in a row. Now consider that 8.2 billion is 82,000 instances of “100,000 people.” Saying that out loud without any breaks, speaking at about 300 syllables per minute, would take 31 hours straight. If you were to count from 1 to 8.2 billion at a normal pace, it would take you more than 2 and a half centuries. The point is that 8.2 billion is a really, really huge number. And pro-extinctionists think that somehow, magically, all these people are going to disappear to make room for our posthuman progeny?

We’ve seen that some pro-extinctionists are okay with posthumanity simply slaughtering our species. Druggan is an example, and I doubt he’s the only one. However, many other pro-extinctionists insist that the only way human replacement should happen is voluntarily. In a follow-up to his social media poll asking EAs about their views about replacement, Manheim also asked:

If you identify as an effective altruist, and think that Humanity’s eventual replacement by AGI / ASI is desirable, what position do you hold?

After ASI, human replacement / extinction should be:

1) only ever voluntary

2) eventually even over objections

3) ASAP

Of the 46% of people who reported being an EA, 37.5% voted for (1), 6.3% voted for (2), and 2.1% voted for (3). A large majority of pro-extinctionist EAs thus prefer voluntary replacement. My guess is that (2) and (3) would have received the most votes if the poll had specifically targeted e/accs, as people like Verdon appear to accept a more brutish ethical code according to which “might makes right.” Hence, if posthumanity is more powerful than humanity, and if it decides to use this power to murder all 8.2 billion people on Earth, that would presumably be the morally right thing to do.

Sutton appears to disagree. In 2024, I interviewed him for my Realtime Techpocalypse newsletter. I have to admit that the conversation was very strange, and oftentimes his answers were plainly incoherent. He railed against democracy, and reiterated his view that he doesn’t “want to prioritize flesh and blood over other intelligent beings.” However, he insisted that he opposes “AIs killing people.” When I asked him about the possibility of ASI slaughtering humanity, he called this a “doomer exaggeration” and told me that it “isn’t an appropriate thing even to ask” him. It seemed like he was trying to say that human replacement should be a peaceful, voluntary affair. He doesn’t want AIs to murder us for the same reason he opposes “any one group killing” another, which is why he also opposes “people killing the AI.”

Claiming that the replacement process should be voluntary is a nice thought. But, like the coexistence view, it is utterly and completely implausible. There is simply no way that humanity will ever consent to being replaced by posthumanity. Can you imagine every person—or a majority of people—in India, China, Nigeria, Argentina, Norway, Bhutan, South Africa, Ireland, Canada, Belgium, Bolivia, the Philippines, Peru, and the US all agreeing—in the next decade, no less—that it’s time for our species to bow out to make room for ASIs designed by billionaires in Silicon Valley? That is never, ever going to happen. Consequently, the only way replacement could possibly occur is through some involuntary means, against the will of virtually everyone on Earth. (This goes for all instances of replacement, by the way, including the sort of replacement that would inevitably result, I’ve argued, if the coexistence view were pursued.) While the replacement view may be popular in Silicon Valley, it finds virtually zero support outside the Bay Area.

Tegmark captures this idea in an interview from 2025:

Beff Jezos [Verdon], the founder of the e/acc movement on Twitter, has said that he’s very happy to replace all biological life with machines. … I think of that position as a digital eugenics, where you’re basically salivating about some digital master race coming along, deeming it sort of more worthy than us. And, you know, I believe in free speech. I believe that they should have the right to … want their digital master race. But, you know, most of us also have a right to our own opinion. And I don’t feel they have any rights to dictate that my children are going to be replaced by their digital master race, just because they think it’s cool. My children are also citizens of the United States. They will also have a vote one day, if I have my way. … And I think the vast majority of the people on this planet, I think, actually agree with me here, that we shouldn’t let a bunch of digital eugenicists make this decision for the rest of us (italics added).

The overwhelming majority of humans are on Team Human, a fact that replacement view advocates must acknowledge—but rarely do. When some do address this, they timidly suggest that replacement should be voluntary, though almost none provide any further details about how exactly it could possibly be voluntary.

Does Verdon really expect humanity to die out without an Armageddon-style battle between the armies of AI and Butlerian jihadists like myself? Does Page really think there wouldn’t be blood in the streets if a new digital species of alien posthumans were to emerge in the coming decades? Does Ord actually believe the metamorphosis of our species “into something beyond the humanity of today” wouldn’t trigger mass revolts from humans who oppose this? Does Ord imagine everyone around the world will convert to the transhumanist religion, and hence eagerly consent to this radical transformation? What if some people are allowed to remain human while others become posthuman: does he believe there won’t be worldwide protests organized by “legacy humans” who worry this new species will take over and, for reasons outlined above, force humanity out of existence? How does he think these posthumans should deal with such pesky humans fighting for their species in this apocalyptic showdown? Does he have anything resembling a plan—perhaps even a “concept of a plan”—for how to navigate such an impasse?

On these questions, the silence of such people is deafening, because I can hardly think of an issue of greater import. It is, if I may say, completely outrageous and morally unacceptable that they often simply shrug their shoulders and say, “Meh, we’ll figure it out when the time comes”—especially when they think the time will come in the near future, within the lifetimes of you and your children. Involuntary extinction would be a catastrophic violation of human rights likely producing enormous amounts of pain, misery, anguish, and terror. Yet this is the only plausible way posthumanity would replace us. If posthumanity were to arise in the near future, these questions would suddenly become urgent in the most profound sense. Perhaps, then, we should do everything we can to prevent posthumanity from arising.

***

If you spend any time listening to TESCREALists, you’ll hear many of them say that quite literally nothing matters more than avoiding human extinction. Sometimes they couch this in the language of existential risks. As Bostrom writes, “there is one kind of catastrophe that must be avoided at any cost: Existential risk.” There are many types of existential risks, as we’ve noted, the most obvious being human extinction. Hence, human extinction must be avoided at any cost. Yet many of these same individuals advocate for a future in which humanity will go extinct. Some explicitly hold the replacement view, according to which we should go extinct. So, how does this make sense?

The answer is that both terms in “human extinction” are ambiguous, and TESCREALists often exploit this ambiguity to make their views look more reasonable than they actually are. It gives the impression that they’re on Team Human, when in fact they’re on Team Posthuman, which enables them to launder their techno-utopian, often pro-extinctionist positions into public debates about AI and other issues to make it seem like they’re on the side of humanity. Understanding the ambiguities of “human extinction” is absolutely crucial for making sense of what TESCREALists are actually advocating.

We begin with the words “human” and “humanity.” Most people will intuitively define these as referring to our biological species, Homo sapiens. That’s how I’ve used them throughout the book, because I think this is clearly the best and most useful way to define them. Let’s call it the Narrow Definition. However, TESCREALists have their own idiosyncratic definition, according to which “human” and “humanity” refer both to our biological species and whatever posthuman descendants we might have. Let’s call this the Broad Definition.

If the Broad Definition looks oxymoronic to you, that’s because it is. On this account, posthumans count as humans, and posthumanity is an instance of humanity. That seems off because the prefix “post-” is supposed to indicate something distinct that comes after something else. It’s not possible for a football coach to give an angry postgame interview while the game is still going on, and it would be incoherent to say that little Johnny won’t be born for another 3 months but is also postnatal. Yet, this is the terminology that TESCREALists have given us, which I take to be indicative of a deeper problem of sloppiness in this literature.

Here are a few examples, so you can see for yourself. Nick Beckstead, an EA who’s worked for both Open Philanthropy and Bankman-Fried’s FTX Future Fun, wrote in a 2013 dissertation that cofounded the longtermist ideology: “By ‘humanity’ and ‘our descendants’ I don’t just mean the species homo sapiens [sic]. I mean to include any valuable successors we might have,” which he later describes as “sentient beings that matter” in a moral sense. MacAskill and his longtermist colleague Hilary Greaves stipulate that “we will use ‘human’ to refer both to Homo sapiens and to whatever descendants with at least comparable moral status we may have, even if those descendants are a different species, and even if they are non-biological.” In The Precipice, Ord writes that “if we somehow give rise to new kinds of moral agents in the future, the term ‘humanity’ in my definition should be taken to include them.” Along the same lines, Bostrom offers an even broader definition according to which “humanity” denotes “Earth-originating intelligent life.”

Now consider the radical implications of the Broad Definition. It implies that our species could literally die out next year without human extinction having occurred. As long as a new posthuman species takes our place, then “humanity” will persist; and if humanity persists, then human extinction won’t have happened.

So, when TESCREALists talk about the importance of avoiding “human extinction,” they aren’t talking about our species. They’re talking about posthumanity. The survival of our species, in the grand cosmic scheme of things, matters only insofar as this is necessary to produce posthumans. These are the beings who will bring about utopia—a posthuman paradise—and so once they arrive it really doesn’t matter if we stick around. Our job will have been completed, our eschatological duties discharged. Preventing human extinction doesn’t mean ensuring that our species survives into the long-term future.

Do not be fooled by this linguistic trickery! If you look carefully at the way TESCREALists use “human” and “humanity,” it becomes clear that they’re usually adopting the Broad Definition. They might say, in one breath, that “humanity” has existed for 300,000 years. This is how long Homo sapiens has been around, which might lead unsuspecting folks to assume they’re using the Narrow Definition. Yet, in the next breath, they say that “humanity” could survive for trillions of years to come by colonizing the universe. But we saw that colonization is physically impossible for biological humans, so they must actually be talking about digital posthumans. This reveals that they were using the Broad Definition all along.

Or, consider Verdon’s claim that e/acc strives to “develop interplanetary and interstellar transport so that humanity can spread beyond the Earth” to “increase human flourishing via pro-population growth policies and pro-economic growth policies” (italics added). Once again, he must be using “humanity” to mean posthumanity, since he himself admits that colonization is only possible for digital beings. People who don’t know about the Narrow and Broad definitions might be fooled into thinking that Verdon is somehow on Team Human, when he’s actually a digital eugenicist who wants ASI to replace humanity in the near future.

***

Both of these definitions refer to beings in the world. But there’s another way that TESCREALists sometimes use the word: as a shorthand for our values or type of consciousness. Consider the Musk-Page debate once again. We saw that Musk thinks it’s important for digital posthumans to care about the same things we care about, i.e., to share our values. But he also emphasized something else in his exchange with Page: “human consciousness,” he exclaimed, is “a precious flicker of light in the universe, and we should not let it be extinguished.” This was in response to Page’s question about why it matters “if machines someday surpass humans in intelligence, even consciousness.” Page saw Musk’s answer as “sentimental nonsense,” by which he apparently meant that it arises from a sentimental attachment to human qualities. That’s precisely why he called Musk a “speciesist.”

If “speciesist” means “a preference for the biological human species,” then Musk ain’t no speciesist. He’s a transhumanist who’s actively trying to merge our brains with AI while building an AI God in the form of Grok 10.0, or whatever. How does one make sense of this? The answer is that what Musk really cares about is our values and our consciousness. He doesn’t care about biological humans enduring, only that these two qualities persist into the posthuman era. Indeed, he seems to think that merging our brains with AI is one way to ensure these qualities persist: by transferring them to machines, they can continue and even proliferate in a world increasingly dominated by AI. His goal is to achieve “some kind of AI symbiosis, where you have an AI extension of yourself, like a tertiary layer above the limbic system and cortex.” And, as the TESCREAList Elise Bohan points out, the aim of this symbiosis “prevent humanity from being superseded.” Here, “humanity” can’t refer to our biological species, because a human-machine merger takes us in the direction of cyborg posthumanity. Rather, it must mean “our values” and “our consciousness.”

This usage of “humanity” can also lead to confusion. Recall that Musk shouted at Page: “Well, yes, I am pro-human. … I fucking like humanity, dude.” Yet Musk is not “pro-human” in the sense that most people would assume. He’s not “pro-human” in the way that I am, for example, since I actually want our biological species to persist. I don’t want our values or consciousness to endure in the minds of digital posthumans; I want them to endure in the central nervous systems of actual humans, because I actually do, in a literal sense, “fucking like humanity.” Once again, whenever TESCREALists talk about liking “humanity” or being “pro-human,” it behooves you to take a closer look to see which definition they’re using, or whether they’re employing such terms as shorthand for some abstract qualities they believe could be instantiated by some posthuman substrate.

***

Turning now to the second word in “human extinction,” I argued in my academic tome Human Extinction that there are actually six distinct types of “extinction” scenarios that we could undergo. Only two are relevant to our present discussion, and both specifically concern the Narrow Definition. That is to say, the Narrow Definition—but not the Broad Definition—makes “extinction” ambiguous between two very different types of extinction. These are:

• Terminal extinction: our species disappears entirely and forever.

• Final extinction: our species disappears entirely and forever without leaving behind any worthy successors to take our place.

For TESCREALists who accept the replacement view, the only type of extinction that “must be avoided at any cost” is final extinction. The reasoning parallels that above: what really matters is the creation of a cosmic utopia among the heavens. This utopia can only be built by posthumanity—some worthy successor. (For folks like Verdon, Sutton, and Page, virtually any ASI that takes our place would be a worthy successor, so long as it’s superintelligent and conscious. For people like Musk, Yudkowsky, Ord, and many other EAs, Rationalists, and longtermists, posthumanity must possess additional properties to count as worthy.) Since final extinction is the only extinction scenario that would permanently foreclose the realization of posthumanity, it’s the only type of extinction that really matters.

Let’s be clear about this: final extinction logically entails terminal extinction, but terminal extinction does not logically entail final extinction. For example, if posthumanity emerged next year, then our species could die out—undergo terminal extinction—without final extinction having happened. Those posthumans would take our place, which means final extinction would have been avoided. Pro-extinctionism, in fact, is nothing more than the claim that we should aim for exactly this scenario: terminal extinction without final extinction. All that means is that our species ought to kick the bucket after posthumanity makes its glorious debut.

However, if terminal extinction were to happen next week, it would almost certainly entail final extinction—not logically, but contingently. This is because we haven’t yet created or become posthumans who could take our place. Terminal extinction would bring about final extinction if posthumanity doesn’t yet exist, but not once posthumanity has arrived.

The point is that terminal extinction itself isn’t what matters to TESCREALists, precisely because our species’ mission isn’t to create utopia, but rather to create the sorts of beings who will create utopia. Echoing earlier remarks about the Broad Definition, avoiding terminal extinction matters only insofar as it’s necessary to avoid final extinction. Once terminal extinction would no longer entail final extinction, then what’s the point of humanity sticking around? Why not bring about terminal extinction if this would no longer threaten utopia? We would, after all, only be sucking up valuable resources that they could use for something else, and there might even be “moral” arguments for why posthumanity should get rid of us.

***

To recap, TESCREALists often use “humanity” to refer to us and our posthuman successors, or to some abstract qualities that we possess. On those occasions when they use the Narrow Definition, they’re almost always talking about final rather than terminal extinction. Please be wary of their language games! As I mentioned above, the way they talk about “human extinction” and related issues often gives the impression that they’re on Team Human, when in fact many of them are actually pro-extinctionists who hold the replacement view.

Yudkowsky provides a glaring example. His coauthored book, If Anyone Builds It, Everyone Dies, has received considerable attention. Even Bernie Sanders and Alexandria Ocasio-Cortez appear to have been influenced by it, as Sanders sat down with Yudkowsky, Kokotajlo, and others to discuss AI safety, which was recorded in a video that Sanders shared on social media. My guess is that Sanders falsely thinks that Yudkowsky is on Team Human, and probably also that he’s opposed to anyone building ASI ever—that is, after all, what the title of his book suggests.

In fact, Yudkowsky wants to build ASI asap, he just doesn’t think we’re ready yet. If the alignment problem were solved next week, then he’d suddenly switch sides and join the radical accelerationists in calling for AI capabilities research to be throttled full-speed ahead. I doubt most people reading his book understand this. As his organization MIRI wrote in 2024,

we remain committed to the idea that failing to build smarter-than-human systems someday would be tragic and would squander a great deal of potential. We want humanity to build those systems, but only once we know how to do so safely.

The word “potential” here refers to our “glorious transhumanist future” in which, as Yudkowsky argues, posthumanity might simply clear all humans off Earth and leave it as a nature sanctuary. Yudkowsky is neither anti-ASI nor pro-human, though his mendacious use of language fools people into thinking otherwise. Again, if he thought that we could build an ASI that would bring about his pro-extinctionist utopian fantasies next week, he would shout: “Best do it so!”2

Or, consider Bostrom. Despite his apparent defense of the coexistence view in the quote from earlier, he actually appears fairly aligned with pro-extinctionism. As the statistician and futurist Olle Häggström notes, “among thinkers with at least some successionist tendencies, one might even include Nick Bostrom,” where “successionist” is Häggström’s term for “pro-extinctionist.” Take Bostrom’s definition of “existential risk” as “one that threatens the premature extinction of Earth-originating intelligent life or the permanent and drastic destruction of its potential for desirable future development.” Häggström writes that

the reason he speaks of “Earth-originating intelligent life” here, rather than simply “humanity” [on the Narrow Definition], is that he is able to envision scenarios where humanity has been replaced by successors so worthy that the scenarios should count as sufficiently benign to not warrant the label existential catastrophe.

By utilizing the Broad Definition, Bostrom can embrace pro-extinctionist views while simultaneously claiming to care about “humanity” and oppose “human extinction.”

This gets at the reason I strongly prefer the Narrow Definition: it enables far more precision. It forces “extinction” to become ambiguous, which opens the door to nuance. I strongly oppose the terminal extinction of our species, which is what I mean by “human extinction.” In contrast, TESCREALists only care about this as a means to an end, not an end in itself. Our survival is just a means to the end of their bizarre notion of a cosmic utopia full of digital space brains. Once this utopia becomes possible to build by our posthuman progeny, then the survival of you and me—of the 8.2 billion people on Earth right now—becomes a matter of moral indifference or positively undesirable.

Let’s reflect for a moment on the dire implications of this picture. Accelerationists like Verdon offer us a single option for the future: the extinction of our species, which they hope will happen in the coming years. People in the AI safety camp more graciously (I say sarcastically) offer us two options: the extinction of our species or the extinction of our species. In the first case, “extinction” specifically refers to final extinction. This would happen if we were to build a misaligned ASI, which would by default turn around and destroy us. Worse, it would spread its misaligned values into the universe, resulting in an existential catastrophe. In the second case, “extinction” refers to terminal extinction without final extinction. This would happen if we build an aligned ASI that transforms (part of) humanity into posthumanity and then brings about a posthuman paradise among the stars. For all the many reasons specified above, this “best-case,” “utopian” outcome would almost certainly result in our species dying out.

In both cases, the outcome is identical for our species: terminal extinction. That’s why I argue vociferously that the entire TESCREAL framework is rotten to the core and must be abandoned. It’s why I think we must defeat the TESCREAL worldview if we want to permanently end the ASI race. No one in the TESCREAL movement is advocating for a good future for our species, not even the coexistence people, since a world ruled and run by posthumans would inevitably result in humanity being sidelined, marginalized, disempowered, and ultimately eliminated. Human extinction or human extinction: the choice is up to us, unless we demolish the worldview in which these are the only two options before us.

***

To summarize some key points of this essay, posthuman eschatology is the view that we should introduce a new posthuman species to the world, ideally in the near future. This leaves open a number of questions: should these posthumans be digital or biological? Should they arise as an extension of humanity or take the form of autonomous beings we create in the lab? Should the population of posthumanity include some of the same people alive today? Must these posthumans share our values and care about the same things we care about (whatever that means)? And once they’re here, should our species stick around or follow the dodo into an eternal oblivion? If we should die out, then how exactly should this happen: through some involuntary process (which would never happen), or via coercive means, perhaps even including outright omnicide (the murder of everyone)?

All TESCREALists accept a posthuman eschatology. It’s the common denominator underlying all these ideologies. TESCREALists just disagree about the details. Those disagreements can lead to heated, vicious debates, but these should be seen as nothing more than family disputes. Everyone agrees with the same basic vision of the future: give birth to posthumanity, spread beyond Earth, plunder the cosmos for its vast resources, and build a glorious multi-galactic civilization at the top of the Kardashev scale occupied by potentially fast numbers of immortal space brains living lives of “surpassing bliss and delight,” to quote Bostrom’s “Letter from Utopia.” This is the ultimate telos toward which we ought to be striving, which is precisely what inspired the founding of AI companies now engaged in a reckless race to create an algorithmic God.

As always:

Thanks for reading and I’ll see you on the other side!

Consider this hilarious video of a humanoid robot being dragged across the stage after awkwardly tripping over some stairs while dancing to Michael Jackson’s “Billie Jean.” Notice how the timing of the synths couldn’t be more perfect — as if to mock the syncopal robot at exactly the moment it passes out:

Meanwhile, someone captured a segment of an AI slop video about the WWE. There are, apparently, a bunch of YouTube channels that publish long-form videos narrated by AI (discussed around 3:00 here). The enslopification of the Internet accelerates! In this case, the AI begins to self-destruct like a washing machine with a brick in it after trying to pronounce “WWE.” I have a facetious ranking of funniness in terms of chuckles and guffaws, where 10 chuckles equals one guffaw. This one gets a full 2.5 guffaws — meaning I doubt you can watch it without laughing out loud:

I don’t think this is fake. I encountered something similar (but not as extreme) when watching a documentary that I thought was narrated by a human. When the narrator tried to pronounce an acronym full of consonants, it produced a bizarre sound that no human would ever make. (I can’t recall what the acronym was — maybe WWF, for World Wildlife fund?). On another occasion, I was watching a different documentary — again, foolishly thinking I was hearing a human — in which the narrator said, “And this cost the US dollar sign 300 million.” God damnit!

As Ronny Chieng put it during a graduation speech at Harvard: “Fuck AI.” I really wonder if, in a very profound sense, we aren’t witnessing an end to the human era — the era in which humans have been dominant. What a sad way for it to end, if that’s the case.

WaPo’s Nitasha Tiku on TESCREAL Doomerism

Taylor Lorenz has a new video in which she interviews journalist Nitasha Tiku of the Washington Post. They discuss how billionaires are moving mountains of money into social media campaigns to raise public awareness of the “existential risks” posed by ASI (artificial superintelligence). In some cases, the very same tech billionaires who invested in the AI companies are now supporting “anti-AI” activists and influencers. I have some thoughts on this interview:

The first part of the video provides a nice — albeit brief — overview of the TESCREAL ideologies. (The heart of this bundle is a techno-utopian vision in which our “posthuman” successors colonize the universe and build a sprawling multi-galactic civilization full of trillions and trillions of “digital people.”) Around 1:41, Tiku says:

So, AI safety is ... an evolving term. Like, this is sort of the one that the community has landed on. This is almost, like, a 25-year-old movement. So if you’ve heard about, like, the singularity, effective altruism, rationalist, transhumanist … it’s kind of at this intersection.

She later mentions longtermism, thus foregrounding 5 of the 7 ideologies in the TESCREAL acronym — the other two being cosmism and Extropianism. The TESCREAL acronym picks out a cultural movement built around libertarian transhumanism — i.e., Extropianism — that extends from the early 1990s up to the present. You can understand it as a more technophilic and utopian iteration of the Californian ideology. All of the major AI companies — DeepMind, OpenAI, Anthropic, and xAI (which is now SpaceXAI) — emerged directly out of the TESCREAL movement. The whole reason the CEOs and cofounders became interested in ASI in the first place was because the TESCREAL worldview identifies ASI as the ultimate means for realizing a posthuman paradise among the stars.

The field of “AI safety” was birthed by the TESCREAL movement. Why? Because some TESCREALists came to believe that there are two fundamentally different kinds of ASI: aligned and misaligned. If we were to build an ASI that’s “aligned” with the TESCREAL worldview, then we’d get the cosmic utopia of posthumanity mentioned above. But if it were misaligned, it will kill everyone on Earth and, astronomically worse, erase our “glorious transhumanist future” (quoting Eliezer Yudkowsky) before we have a chance to start drawing it.

Furthermore, these same TESCREALists — dubbed “doomers” — argue that figuring out how to build an aligned ASI is far more difficult than one would initially assume. So, AI safety was founded to find a solution to the “alignment problem.”

Tiku is absolutely right that the field emerged roughly 25 years ago, in the early 2000s, thanks largely to the work of Yudkowsky. Right now, many AI safety advocates believe that we’re very close to building an ASI, but nowhere close to solving the alignment problem. That means that total annihilation could happen in the very near future.

You can think of this in terms of a race between AI safety research and AI capabilities research. The finish line of the former is solving alignment. The finish line of the latter is actually building an ASI. If capabilities researchers get to their finish line before safety researchers reach theirs, then we’ll all be dead and the transhumanist future will be permanently canceled. When Yudkowsky says, “if anyone builds it, everyone dies,” he’s specifically talking about misaligned ASI — not aligned ASI.

***

At 4:37, Tiku says that AI safety

started with people who are not AI researchers. A lot of them are former philosophers from Oxford — self-taught people who were prescient … about the general direction that AI technology was moving forward. People now give them a lot of props for anticipating some of the concerns that we’ve seen around, like, ChatGPT and very human-like AI, and what this can do to human interaction — how quickly the technology is developing. [This] used to be called, like, concerns about the Singularity …

Here I would push back against Tiku’s characterization of early TESCREALists as being “prescient.” I think she gets things exactly reversed: it wasn’t that these people were prescient, it’s that they actively strove to fulfill their own prophecies — and they succeeded.

Yudkowsky himself played an integral role in launching the ASI race (as I discuss at length in my forthcoming book). He inspired people like Shane Legg, the cofounder of DeepMind, and introduced another DeepMind cofounder, Demis Hassabis, to Peter Thiel after the 2010 Singularity Summit. Thiel then invested in DeepMind, which is how it started.

Sam Altman was also inspired by Yudkowsky, explicitly saying that Yudkowsky is the reason that many people, including himself, became interested in trying to build god-like AI.

A New York Times profile describes Altman as a “product” of the EA and Rationalist communities. Anthropic CEO Dario Amodei was an EA-longtermist who hired a bunch of EA-longtermists to work as his company, including Holden Karnofsky, Joe Carlsmith, and Amanda Askell (who was married to the cofounder of EA).

Elon Musk has explicitly said that longtermism is “a close match for my philosophy.” Musk attended numerous conferences before the ASI race took off in which leading TESCREALists like Yudkowsky and Nick Bostrom were present.

And we haven’t even touched on all the money that helped start the ASI race from TESCREAL organizations like Open Philanthropy (now Coefficient Giving) and donors like Jaan Tallinn. (Tiku acknowledges some of this later on.)

It was because of the TESCREAL movement that ASI became widely recognized as the key to creating utopia. TESCREALists popularized the idea of ASI, inspired the founding of AI companies, helped those companies get tens of millions in funding, and so on. To say that early TESCREALists were prescient would be like saying that I was prescient at 8am this morning when I prophesied that I would soon be drinking coffee — and then got up and made myself a cup of Joe.

***

Around 9:10, Tiku says that EA was initially focused on “global poverty.” I don’t think this is right. As Mollie Gleiberman argues, EA was longtermist — that is, focused on longtermists causes like AI safety — from the very start. To support her claim, she compiles a catalogue of copious examples from leading EAs around the time EA was founded, in which these EAs say that “existential risk” mitigation is what really matters. Global poverty was, basically, a means of marketing the movement. Once people had joined EA, they would then be exposed to arguments for why mitigating existential risks is actually the best way to “do the most good” in the world. Gleiberman describes this as a “Trojan horse,” where the ultimate goal was to pursue what she describes as “transhumanitarianism,” a portmanteau of “transhumanism” and “humanitarianism.”

***

Tiku argues around 9:22 that EA then merged with the Rationalists and transhumanists. But, again, I don’t think this is right. EA arose out of the inchoate Rationalist movement, and it was influenced by transhumanism from the very start. Rationalist blogs like Overcoming Bias and LessWrong were the Petri dishes in which EA first grew, and the cofounder of EA, Toby Ord, had been publicly defending transhumanism with the most influential ideologue of the 21st century, Nick Bostrom, since at least 2006. Ord was a member of Bostrom’s Future of Humanity Institute, founded in 2005 to advance the TESCREAL worldview, two years before EA was officially founded in 2009.

This is part of Gebru and my point re: the TESCREAL thesis. These ideologies are all tangled up together. They are outgrowths of each other. They are different colors, reflected through a prism, of the very same techno-utopian posthuman eschatology. EA emerged from the libertarian transhumanist movement that constitutes the nucleus of TESCREALism.

***

At 11:08, Lorenz says:

You also kind of alluded to something, which is that it doesn’t seem like they’re as anti-AI as you’d think. A lot of them, it seems, are interested in this idea of an aligned AI, or at least this idea that like it’s not even necessarily that AI is bad, maybe, but it’s just that we don’t have an AI that’s working in the interest of humanity. Is that right?

Tiku responds:

Yeah, that’s such a good way of putting it. And this is one of the concepts that is I think hardest for people outside to grasp, because if you think that a technology is going to destroy the planet — destroy every human on Earth, cause this kind of existential risk — you would think that you would be doing everything in your power to stop it. But what you quickly realize is that there [are] many groups of people that are also … enamored with the potential for utopia from this technology. “Aligned AI” is one term and that means kind of aligning AI with human values. Again, a very very abstract concept, but basically it’s like [the ASI does] what the developers want. You have a morality system like ethics and you approach problems in a way humans might even though you don’t have any real world experiences or emotions of your own. Sometimes they used to call it “Friendly AI.”

I’m really glad that Lorenz and Tiku highlight this. As mentioned before, TESCREAL doomers are not anti-AI. Even the most hardcore doomer wants ASI as soon as possible. Specifically, they want ASI once the alignment problem has been solved. If a doomer like Yudkowsky were to become convinced that the alignment problem is solved, he would suddenly switch from screaming, “Shut it all down, now!!” (where “it” refers to capabilities research), to screaming this: “Pedal to the metal! Build ASI asap!”

Don’t be fooled by the rhetoric these people use. They like to “sell” themselves as anti-AI, which gets normies who hate AI on board because those normies think their views align. They don’t. Yudkowsky would say that delaying the creation of an aligned ASI is as wrong, morally speaking, as accelerating the creation of misaligned ASI.

Nor are most TESCREAL doomers pro-human. They advertise themselves as such, but scratch the surface of their views and you find a deeply anti-human eschatology. I’ve written about this at length before (see below), and will soon publish, on this newsletter, chapter 4 of my forthcoming book because it offers a comprehensive account of the intimate links between the TESCREAL movement and pro-extinctionism. Stay tuned for that!

In my view, pro-extinctionism is the most important feature of the TESCREAL worldview that journalists should be highlighting. Quite literally everyone in the TESCREAL movement is pushing for a future in which our species is sidelined, marginalized, disempowered, and ultimately eliminated. This outcome is an inevitable consequence of the posthuman eschatology that they embrace and are actively trying to realize through the creation of superintelligent artificial deities. If this isn’t worth shouting about, then what is? Altman himself holds this view, as I show in a recent Truthdig article.

***

I don’t have many comments on the rest of the video. The discussion is very good, and Tiku foregrounds a number of important issues, such as the incestuous funding channels linking hundreds or thousands of TESCREAL organizations (many focused on AI safety) with super-wealthy TESCREAL donors (such as Jaan Tallinn, Dustin Moskovitz, and Vitalik Buterin — recipient of a Thiel Fellowship).

Some of these donors, including Tallinn and Moskovitz, helped launch the ASI race that they now worry could destroy humanity and, along with us, the “vast and glorious” posthuman future among the heavens. There is a great irony to this: no single group has done more to launch, sustain, and accelerate the ASI race than the TESCREAL doomers!

Consequently, these same people are now pouring money into social media campaigns aimed at halting the ASI race. The Future of Life Institute, for example — which was cofounded by Tallinn and received $10 million from Musk, and another $500 million from Buterin — has offered grants of between $100k and $500k for projects aiming “to educate and engage stakeholder groups, as well as the general public, in the movement for safe, secure and beneficial AI.” A central thrust of the interview with Tiku was to point out that you may very well have watched a video on YouTube warning about the existential risks of ASI and not realized that, in fact, the organization behind it is just two steps removed from the AI companies themselves, the intermediate link being the tech billionaires who funded both.

Vox Publishes an Article on Pro-Extinctionism

Vox — which, recall, has a deal with OpenAI — recently published an article titled “The People Who Actually Want AI to Replace Humanity.” It explores the ideology of “successionism” (i.e., replace humans with AI), and I have to say that the content is strikingly similar to my many articles and podcast interviews on pro-extinctionism. The author, Sigal Samuel, discusses transhumanism, Ray Kurzweil (the singularitarian), Effective Altruism, cosmism, longtermism, Beff Jezos and e/acc, as well as Daniel Faggella’s “worthy successor” eschatology — all topics I’ve covered at length in much the same way she does in her article. She also highlights the links between “successionism” and eugenics.

Sound familiar? Much of the article is little more than a summary of my work, though she doesn’t cite me or mention the TESCREAL acronym — even though that’s what she’s talking about. See for yourself.

It’s good that more people are talking about this issue, and Samuel appears to be generally critical of the TESCREAL ideologies. Aside from not citing me (which, you know, whatever), my biggest criticisms of her article pertain to what it leaves out. Although she mentions Toby Ord at one point, she doesn’t explicitly say that longtermism is a pro-extinctionist ideology, too — indeed, it’s nearly identical with Faggella’s “worthy successor” idea, as shown in my last newsletter article.

Doing so would have provided an opportunity to distinguish between two types of pro-extinctionism: first, the view that humanity should be replaced by autonomous AIs (like ChatGPT), and second, the view that humanity should be replaced by AIs that we become. Longtermists like Ord want to replace humanity with digital posthumans (which I satirically call “digital space brains,” after my friend Keira Havens suggested “space brains”), where these digital posthumans are radically transformed versions of us. In contrast, people like Verdon and Faggella appear open to autonomous, distinct, completely separate-from-us AIs usurping humanity.

When one carefully examines the space of possibilities here, it becomes immediately obvious that the entire TESCREAL worldview is pro-extinctionist, though certain versions of pro-extinctionism are more flagrantly horrific than others. All TESCREALists are rooting for Team Posthuman, meaning that, at the very least, they want to create a new posthuman species to run the show. This is dangerous because, I’ve argued before, if anyone becomes posthuman, everyone (else) dies out. If posthumanity arises, our days are numbered.

When I started this newsletter back in August 2025, I promised that we’d be covering a range of topics that (a) almost no one else is talking about, and (b) will become increasingly discussed in the coming years. For example, I have argued that we will see more and more debates spill into the public arena in which participants (e.g., Silicon Valley dwellers) are arguing not about whether pro-extinctionism is bad, but about which type of pro-extinctionism is best.

I have not been “ahead of the curve” on most things, but I think my (and Timnit’s) discussion of TESCREALism and my subsequent emphasis on Silicon Valley pro-extinctionism have been. The fact that Vox is now covering the very same ideas that I’ve been exploring for years is confirmation of this!

Next article (Tuesday): chapter 4 of my book, which is almost 20,000 words. Hopefully, it will provide some useful clarity on the different types of pro-extinctionism and the ways it’s entangled with the race to build god-like AI super-beings.

As always:

Thanks for reading and I’ll see you on the other side!

This article is a bit more technical than usual. I’ll publish some AI bloopers in the next post to make up for it. :-) Thanks for reading, friends!

Acknowledgments: Remmelt Ellen and Kyle Evanoff provided insightful feedback on this piece. They do not necessarily endorse my claims or conclusions.

A Self-Defeating Futurology

You may recall Daniel Faggella from a previous newsletter series. He’s a pro-extinctionist who argues that we should replace humanity with a “worthy successor.” He defines a “worthy successor” as “a posthuman intelligence so capable and morally valuable that you would gladly prefer that it (not humanity) control the government, and determine the future path of life itself.”

The point is to create ever-“higher” lifeforms that expand outward and colonize every corner of the cosmos. On this view, individuals like you and me are nothing more than a means to an end. All current and future beings are fungible and disposable. They are mere “torches” that carry what really matters: life or consciousness, which Faggella dubs the “flame.” The goal is for humanity to create its successor in the form of AGI, which will immediately create its own successor in the form of AGI+, which will immediately create its own successor in the form AGI++, and so on forever — an endless, aimless process that Faggella sees as the ultimate eschatological destiny of Earth-originating “intelligence.”

This is a deeply nihilistic view, which overlooks a number of major problems. I outlined some such problems here, but Faggella has yet to directly respond to them (despite publishing this). A problem I didn’t mention in my previous article is that Faggella’s worldview is built on a contradiction:

1. He claims that “a safe (or at least not overtly and unnecessarily dangerous) expansion of potentia is the sole moral imperative.” The only goal is to maximize “potentia,” defined as what enables life to persist.

2. At the same time, he argues that “(in the future) the highest locus of moral value and volition should be alien, inhuman.” He doesn’t want human values to get locked in and determine the future of life. He wants future beings to discover their own radically different set of values.

So, Faggella wants future beings not to be constrained by human values, yet demands that these beings forever embody his value of “expanding potentia.” As I put it in a social media post written for him:

Which is it? Do you want your imperative “maximize potentia” to constrain all future lifeforms, or do you want those lifeforms to discover their own values, which may lead them to antithetical imperatives? If future lifeforms don’t maximize potentia, you’d consider this a great cosmic tragedy — precisely because you’re trying to impose a human value (potentia maximization) on beings you claim should reject human values. That’s why your view is self-defeating: if future lifeforms were to embrace it, they’d reject it.

Faggella’s View Is Just Utilitarianism

Another way to understand Faggella’s eschatological ethics is this: it’s just utilitarianism with a different value theory. Let me explain:

If you were to dissect utilitarian theory, you’d find that it has two components: one deontic and the other axiological. “Deontic” refers to what you ought to do, whereas “axiological” refers to what is good or bad, better or worse. The axiological component says that one state of affairs containing more total “value” is better than another containing less. The deontic component says that you ought to bring about the state with more total value — hence, failing to do this would be morally wrong.

To illustrate: if lying to someone would result in the best possible outcome all things considered (axiological claim), then you should lie (deontic claim). But if telling the truth results in the best outcome, then you should tell the truth. This is what your philosophy professor meant when she said, “Utilitarianism puts the good before the right” — it builds what’s right on what’s good or overall best; it bases the deontic on the axiological.

But what is “good”? What is the “value” that utilitarians say we must maximize? This is where value theory enters the picture: we know what we ought to do — maximize value/the good — but we don’t know what value is without a theory of value to tell us.

The most well-known value theory defended by utilitarians is hedonism, which says that value = pleasurable experiences or happiness. Hence, you can replace “value” in the sentence, “Our sole moral obligation is to maximize value,” with “pleasure” to get: “Our sole moral obligation is to maximize pleasure.” The value theory of hedonism fills in the blank!

What Faggella does is introduce a new value theory, which is even less plausible than hedonism. According to his account, value = potentia (well, it’s really consciousness or life, but potentia is what enables those things to persist and evolve). Hence, “Our sole moral obligation is to maximize potentia.” This is just utilitarianism with a different theory of value.

Utilitarianism Is Just Capitalism

One of the most salient problems with traditional utilitarianism is that it doesn’t care about people as ends in themselves. In fact, it doesn’t care about anything for its own sake except for “pleasure” or “happiness” (if one is a hedonist).

This is why it yields the Repugnant Conclusion, whereby a world full of 1 trillion people with a happiness level of only 1 is better than a world full of 1 million people with a happiness level of 900,000 (if you can even make sense of quantified “happiness levels” — I can’t, but this is how they proceed).

That’s because 1 trillion x 1 units of value = 1 trillion, whereas 1 million x 900,000 = 900 billion. One trillion is more total happiness/value than 900 billion, and hence utilitarians say that we should realize the first world full of basically miserable people over the second world full of wonderful lives if those are our two choices. Utilitarians like William MacAskill, Toby Ord, and Peter Singer don’t see the Repugnant Conclusion as a big deal. They make that claim here.

This is nothing more than capitalism applied to ethics. It’s all about the number-crunching. Morality is a calculation to be solved. Utilitarianism reduces ethics to a branch of economics. People are just replaceable containers of “value,” as they have no value in themselves.

Some bizarre things (aside from the Repugnant Conclusion) follow from this. For example, there is no intrinsic difference between the death of someone and the non-birth of someone else, all else being equal. That is to say, if an unborn possible person would have a happiness level of 100 if they were to exist, then by not existing, the universe is deprived of 100 units of value. We should, therefore, create that person to add 100 units of value to the world. Similarly, if an existing person has a happiness level of 100 but then suddenly dies, the universe is deprived of 100 units of value. In both cases, the outcome is identical: the universe loses 100 units. As Jonathan Bennett once mockingly wrote: “As well as deploring the situation where a person lacks happiness, these philosophers also deplore the situation where some happiness lacks a person.”

You could say something similar about profit in a capitalist system. Not losing $100 million is good. But if there’s another $100 million to gain, you must acquire it. On both the capitalist and utilitarian views, could exist implies should exist: an extra $100 million in profit should be acquired if it could be acquired, and a merely possible person with net-positive value should exist if they could.

These parallels aren’t coincidental. Utilitarianism and industrial capitalism both emerged in the latter 19th century in the very same place: the UK. They are both expressions of an impersonal, quantitative, dehumanizing conception of people as means to an end, and the end as something to be maximized.

Longtermism: Techno-Capitalism Projected into the Stars

I say all of this to introduce the second topic of the article: longtermism, an ideology built upon utilitarian ethics. In particular, it’s founded on the axiological component of utilitarianism, which states that one outcome is better than another if it contains more total value.1

What longtermism adds to utilitarianism is a more expansive, cosmological perspective: cosmologists tell us that the universe is huge and will remain habitable for another 10^100 years or so — a 1 followed by 100 zeros, a much greater amount of time than the universe has so far existed. That means the future population of beings, if we spread throughout the universe, could be enormous. Assuming these beings will bring at least 1 unit of net value into the world during their lifetimes, we therefore have an obligation to realize as many of these future beings as possible — to flood the universe with our descendants until every single nook and cranny, every inch of cosmic real estate, is crammed with people.

Why does it matter that the universe contains the maximum possible amount of value between now and the heat? Well, it just does, because longtermists — and utilitarians — assume that the only appropriate response to value is to maximize it: more, more, more, with no upper limit.

But this ignores a wide range of alternative responses to value such as cherishing, treasuring, revering, caring for, loving, protecting, preserving, sustaining, appreciating, and so on. My approach to ethics emphasizes these alternatives rather than maximization. When I stumble upon a meadow in the woods, I stand there and appreciate it. I take it in and savor the beauty. I don’t think, “This meadow has value. Therefore, the entire world — indeed, the whole universe — should be covered in meadows, because more is always better.” I’m not a capitalist when it comes to ethics.

This is what longtermists think when they look up at the firmament:

Humans Must Be Digitized

To maximize value, we must therefore colonize space. And given that energy sources like stars are burning through their hydrogen reservoirs as we speak, we must do this asap. That’s the central point of Nick Bostrom’s paper “Astronomical Waste.”2 Every second of delay entails some irreversible loss of our “cosmic endowment.” Time is of the essence! You can see why this view appeals to billionaires like Elon Musk.

But how can we colonize space? There’s almost certainly no way for biological humans to do this. Space is incredibly hostile to squishy meat-sacks like us — even colonizing our planetary neighbor, Mars, is likely impossible for human beings, given that Mars has almost no atmosphere, lacks a magnetic field, has much weaker gravity than Earth, and is covered in dirt that’s literally poisonous. The moon is probably the furthest from Earth we’ll ever get.

Hence, humanity will need to become or create digital beings — what I call digital space brains — to colonize the universe in our place. It’s these “posthumans” who will venture beyond our planetary oasis and plunder the cosmos, not us.

More Total Value

As it happens, this has a salutary consequence from a utilitarian point of view: imagine somehow colonizing exoplanets as biological humans. After 20,000 years of travel, we arrive in our spaceships and somehow magically terraform the exoplanet to make it Earth-like and habitable. How many biological humans could then exist on its surface? Maybe 10 billion? Maybe 20 billion?

Now consider an alternative: we arrive as digital space brains and use some magical technology to transform the planet into a giant computer made of computronium (matter optimized to perform computations). We then run a huge virtual reality simulation on this planet-sized computer for digital space brains to reside in. How many of these space brains could exist within the simulation? Longtermists think there could be way more than 20 billion.3 That’s because you can cram more digital people into a given unit of spacetime than you can biological humans. Hence, to maximize value, all future people will need to be digital rather than biological.4

Posthuman Value-Containers

There’s another utilitarian advantage to digitizing “intelligence.” We noted above that, for utilitarians, people have zero intrinsic value. We are the “containers” of value, and the more people/containers there are, the more total value there could be. (That’s why the future population should be as large as possible.)

But here’s the catch: humans can only contain so much value. We’re constrained by our “biological limitations,” and natural selection didn’t optimize us to generate unlimited amounts of goodness. The human value-container is only so deep.

But we could — supposedly — engineer digital posthumans who contain far more value, per individual, than biological humans. We could design them to be much deeper value containers, as illustrated below5:

Do you see where this is going? The ethical foundations of longtermism — utilitarianism — are inherently pro-extinctionist, just like Faggella’s view. To maximize value, we should replace humans with posthumans — and we should do this as soon as possible.

You might think these posthumans would keep some biological humans around, but that wouldn’t be the case: biological humans take up space and use up valuable resources in “suboptimal” ways. There’s no reason for posthumanity to keep us around, and every reason to nudge us out of existence. If the goal is to maximize value in the universe, then our days are numbered.

This is hardly any different from Faggella’s pro-extinctionism. Both aim to maximize something, and in service of maximization they imply that our species should be replaced by a “worthy successor.” The only notable difference is the underlying value theory, though Faggella’s notion of “potentia” also describes a key feature of the longtermist worldview, as maximizing value requires future beings to find new and better ways to ensure the continued survival of life and consciousness.

Six Problems that Longtermists Ignore

But longtermism encounters a number of major problems — nearly all of which its advocates have tended to ignore, dismiss, or wave away. What follows is not an exhaustive list, but rather a collection that I find particularly interesting:

1. Artificial Consciousness?

The entire longtermist worldview is predicated on the possibility of artificial consciousness. Why? Because value seems to depend on consciousness: if digital space brains are “intelligent” but not “conscious,” handing over the universe to them would be like bequeathing it to rocks. Since only digital beings will be able to colonize space, it really matters that artificial consciousness is possible. Literally everything depends on it.

This problem divides into two parts, one metaphysical and the other epistemological.

1.1 The Metaphysics of Mind

Can artificial systems be conscious? Most longtermists assume that’s the case because they think the brain is just a computer — a position called “computational functionalism.” But the “brain as computer” metaphor is just that — a metaphor. Historically, people have metaphorized the brain in terms of whatever the most impressive technology happened to be at the time. After discovering that electricity stimulates nerves, the brain became a telegraph network. When neurons and synapses were identified, it was taken to be a telephone switchboard.6 In the 1940s, cybernetics pushed information feedback loops into the spotlight, and once digital computers arrived, people became convinced that our central nervous system works the same way: neuronal networks are the “wetware” that runs mental algorithms, the software of the brain. In 20 years, it’s entirely possible that the computer metaphor will be replaced by something else.

If the brain isn’t a computer, then artificial consciousness might not be possible. Mind-uploading almost certainly wouldn’t be — i.e., simulating one’s brain on a computer wouldn’t give rise to subjective experiences any more than simulating a hurricane would soak your laptop in rain. There are plenty of reasons for thinking the brain isn’t a computer and that artificial systems can’t be conscious. The neuroscientist Anil Seth provides a useful overview here:

Also, reflect for a moment on what it would take to simulate someone’s brain in silico. Our brains are the most complex objects in the known universe. They consist of ~86 billion neurons and 100 trillion synapses. Waves of electrical activity constantly sweep over them like stormy weather. Signals are spatiotemporally summated in complex ways by interconnected cells. Every second, quivering branches of neurons are rewiring themselves while spurts of blood course through tangles of axons, dendrites, and billions of support cells called glia, whose function we don’t yet fully understand. Your brain is not a static blob, but a constantly wriggling dynamic system squirming about in your skull. Meanwhile, intracellular metabolic processes are producing waste products that accumulate throughout the day and are flushed out at night, when your brain oscillates through different stages of sleep that correlate with distinct wave frequencies.

All of these phenomena would have to be simulated by a computer. The metabolic processes, the brain waves of sleep stages, the quivering axons and dendrites, and perhaps even quantum phenomena that might turn out to be important for cognition and consciousness. Is that technologically feasible? Who knows — maybe not. Given that, for longtermists, everything rests on consciousness being realizable by artificial systems, you’d think they would be obsessing over such questions. But they aren’t.7

1.2 “Testing” for Consciousness

This brings us to the epistemological problem: assuming that artificial consciousness is possible (which I doubt), how could we possibly know that a particular artificial system is actually conscious? What objective “test” could confirm this? (Longtermists would want a test that’s extremely reliable — to the point of near certainty — given how much depends on artificial consciousness.)

If an AI were to say, “Yes, I’m conscious,” could we intersubjectively verify this? AIs, including uploaded minds, could be highly functional zombies: beings with complex outward behaviors but nothing it is like to be them on the inside. Indeed, current LLMs are like this: few people think they’re actually conscious, yet sometimes they’ll tell users that they are.

Perhaps a human being could incrementally upload their mind by replacing one neuron at a time with a silicon chip (a “functional isomorph”) and report back on how the process is affecting their conscious experiences in realtime.8 But what if this person loses consciousness without noticing? What if the functional isomorphs produce the same linguistic output — “I’m conscious” — while the light dims? (As I’ve noted before, there is a closed loop of causes and effects from stimuli impinging upon the eardrum all the way to a vocal response involving the vocal folds, tongue, and lungs. It could be theoretically possible to reproduce this causal loop without consciousness being present.) The philosopher Susan Schneider has proposed a few “tests” to determine if artificial/digital systems are conscious, but in my view they are all inadequate.

This should worry longtermists a lot, which makes it surprising that so few of them talk about it. Imagine that some longtermists create AIs that they mistakenly believe are conscious. They launch these beings into space, where they proliferate in every direction, filling every corner of our future light cone. They take over the universe. But it turns out this founding population wasn’t in fact conscious, and hence our “cosmic endowment” of resources is gobbled up by beings that can’t realize “value.” Whoops! This is why having a robust objective “test” for artificial consciousness — assuming artificial systems can be conscious — is absolutely paramount for longtermism.9

A huge question mark hangs over the entire longtermist project because of these two conundrums.

2. There Won’t Be Digital “People” in the Future

We saw that longtermists argue that there should exist as many digital space brains in the future as possible — an idea I’ve sloganized as, “Won’t someone think of all the digital unborn!” That’s why they’re obsessed with trying to estimate how large the future population could be. Bostrom says there could be 10^38 digital space brains in the Virgo Supercluster, while Toby Newberry claims there could be 10^45 in the Milky Way galaxy. Later, Bostrom estimated 10^58 within the accessible universe as a whole. These estimates are used to motivate their claim that the far future is overwhelmingly important, given that “could exist implies should exist.” It’s not that future people matter more — it’s just a numbers game: there could be far more of them than the total number of humans who currently exist.

The problem is that such calculations assume there will be individual people who can be counted — that future people will be individuatable. But why think this will be the case? If future minds take the form of pure information, of computer algorithms, then they could — and no doubt would — share memories, personality traits, experiences, preferences, beliefs, desires, and so on, with the metaphorical click of a button.10 The boundaries between “individuals” would become porous, and the world would become a digital soup of ontological chaos. Perhaps these digital minds would aggregate into some kind of amorphous wriggling hive mind, comprised of jostling mental fragments constantly recombining with each other in a phantasmagoria of incessant change. Who’s to say?

However things turn out, we can be sure that the digital world will be profoundly different than the world we live in today. It may very well mark a completely new post-individual era in which there are no countable “people” at all. And if there are no countable people, then the estimates that longtermists rely on to motivate their view are completely meaningless.

3. Digital Prozac

Even if digital space brains were to somehow maintain their individuality, longtermists still have to explain why any of these space brains would enjoy living in giant computer simulations in the first place. What would give their lives meaning and purpose? Why on Earth would digital posthumans be happy in vast simulated realities crammed with trillions and trillions of other “disembodied” minds?

I have never once seen any longtermists address this question. Perhaps there would be digital pharmacies to provide digital people with digital Prozac to get through? Or perhaps future space brains will see it as their “moral duty” to somehow force themselves to be “happy” for the sake of maximizing value — because they understand they’re means to an end, according to the capitalist ethics of utilitarianism.

Imagine for a moment what life within a giant simulation might be like. The laws of physics as we know them need not apply: one could design virtual reality worlds in which there’s no gravity, the fundamental particles and forces are different, and F ≠ ma. The possibilities are terrifyingly endless. Would everyone have their own virtual reality world? Would a bunch of digital space brains share a world? If so, who would get to design it? What if it were hacked and everything suddenly changed? What if one’s friends were permanently deleted? Who would maintain law and order? How would people prevent attacks from outside their little Matrix, originating in the physical (rather than digital) universe?

Such worries bring up two additional issues:

4. Immortality, Bunkers, and Eternal Torture

We start with immortality. Digital space brains would be immortal, because software is immortal. As long as the hardware exists to realize the software, digital posthumans could live until the heat death of the universe.

But consider the implications: if you were a digital space brain and were thinking rationally, you would do literally nothing else every day — for trillions of years — but obsess over your survival. Why? The standard “expected value” definition of risk is “probability multiplied by the consequences.” This means that a negative outcome with a tiny probability of occurring could still constitute an enormous risk if the consequences are sufficiently high. If you’re functionally immortal, then the consequences of dying would be astronomically huge, because you’d be losing trillions and trillions of future years of life.

To illustrate, if I were to get hit by a bus tomorrow, I’d lose perhaps 4 decades of my natural life. But if I could live for the next 1 billion years, then getting hit by a bus would be orders of magnitude worse. Consequently, the risk of getting hit by a bus — even if it remains extremely improbable on any given day — would skyrocket. Insofar as I’m rational, I’d never cross a road again. Indeed, since there are many possible but improbable ways of dying, I should just build a bunker and never come out.

The same logic applies to functionally immortal digital posthumans. Living in their computer simulations, they should take every imaginable precaution to ensure their continued survival, given that bad outcomes with vanishingly small probabilities would pose massive risks. They should fortify the planetary computers on which their simulations run with all sorts of exotic weaponry just in case an alien species of posthumans outside the Matrix were to attack. Individually — if they have individuality — they should constantly back up their digital minds in case something happens, which would surely foment fights between digital people over limited server space. Every moment spent not ensuring one’s continued survival would be irrational, given the enormous stakes of dying. This brings us to the second issue:

5. The Anarchic Cosmopolitical Arena: Constant Catastrophic Wars

Why do longtermists assume that space won’t become a theater of constant, catastrophic wars? Due to the vastitude of the cosmos and the hard limit of the speed of light, there’s no possibility of a well-coordinated cosmic Leviathan to provide security. Mutually assured destruction (MAD) won’t work either, and populations of digital posthumans would gradually evolve radically different capacities, traditions, abilities, languages, belief systems, technologies, scientific theories, etc., thus making mutual trust impossible.

What, then, is left to keep the peace? Each solar civilization, or archipelago of civilizations, will find itself in a Hobbesian trap in which preemptive violence becomes the only rational option to guarantee its survival. Spirals of militarization (the “security dilemma”) will greatly exacerbate the situation, as defensive weapons are misinterpreted as offensive threats.

I’ve written about this at length elsewhere, so won’t elaborate here. Suffice it to say that the anarchic nature of the cosmopolitical realm would be primed for constant conflict, and exotic space weapons could potentially obliterate neighboring civilizations at the speed of light. Of course there will be devastating wars all the time — no matter how irenic our posthuman progeny are, they won’t be able to escape the death traps of game theory.

6. Longtermists Face Pascal’s Wager, Not Pascal’s Mugging

The problem of immortality, illustrated above, is an example of what Eliezer Yudkowsky and Bostrom call “Pascal’s mugging.” This occurs when tiny probabilities are paired with enormous stakes. Let’s say that there could be 10^58 digital space brains between now and the heat death, as Bostrom argues. That means the cost of undergoing final extinction (see below) is HUGE. It follows that even if there’s only a teeny-tiny probability of extinction happening, the “expected value” of preventing this scenario will still be massive. We should, therefore, redirect nearly all of our finite resources away from non-extinction risks in favor of projects aimed at ensuring the existence of future digital space brains.

Pascal’s mugging points to extreme cases where expected value theory seems to break down. When tiny probabilities and enormous stakes are involved, expected value theory recommends fanatical options.

The problem is that if you ask a cosmologist whether the heat death of the universe would mark a definite end to information processing, life, intelligence, etc., they will answer “no.” Although speculative, it’s possible that future digital people could find a way to escape the heat death — e.g., by tunneling into a neighboring universe, creating their own universe from scratch, or utilizing some strange futuristic physics to intervene and prevent the heat death from occurring.

Consequently, there could be an infinite number of future digital people. Perhaps our distant descendants tunnel into a neighboring universe, and then iterate this process each time their new cosmic abode collapses into thermodynamic equilibrium. By endlessly iterating the process, they could exist forever, thus generating infinite amounts of value. Improbable? Yes. Impossible? No: there’s a nonzero chance of this happening.

Longtermists should thus use this infinitely higher estimate of future people in their expected value calculations. They want to know what the stakes are, and it appears the stakes could be infinite.

Doing this, however, replaces Pascal’s mugging with Pascal’s wager. The former involves huge stakes paired with tiny probabilities, whereas the latter involves infinite stakes. The conclusion is that the only thing we ever should spend our money and resources on is preventing (final) extinction. Final-extinction risk mitigation wins every time without exception. This is a direct consequence of longtermism plus the cosmological fact that there could be infinite people in the future.

If longtermists were intellectually serious, they would acknowledge that their view yields the most extreme form of “fanaticism” imaginable: literally nothing matters except continuing to survive so that infinite amounts of value can be realized in the future. It’s Pascal’s wager applied to cosmic rather than personal eschatology.

Conclusion

Longtermism proposes a very bizarre futurology. This futurology is based on a number of assumptions, almost none of which are seriously examined by longtermists. The point of this article wasn’t primarily to critique the longtermist ideology, but to highlight a small handful of its underlying assumptions.11 It behooves longtermists to address such issues, but given the cult-like nature of the longtermist community, I doubt they will.

The article also aimed to show how two supposedly different ideologies — Faggella’s “worthy successor” view and longtermism — are nearly identical. Faggella is just a longtermist with an idiosyncratic theory of value, while longtermism has pro-extinctionist implications no less than Faggella’s eschatology.

As I’ve argued, many of these TESCREAL or TESCREAL-adjacent views are far more similar than they are different, including longtermism, effective accelerationism, cosmism, singularitarianism, and the pro-extinctionism of people like Larry Page, Beff Jezos, and Eliezer Yudkowsky. There are differences, but they are minor. All must be stopped insofar as we want a good, flourishing future for our species.

As always:

Thanks for reading and I’ll see you on the other side!

Since the early 2000s, a major life goal of mine had been to land a professorship in philosophy. Academia has always been the community in which I felt most at home. However, after leaving a postdoctoral position last December, I struggle to imagine myself returning. There are many reasons for this, one of which is that AI is destroying the university system in realtime. As a college senior at Stanford, Theo Baker, writes in the New York Times:

Cheating has become omnipresent. I don’t know a single person who hasn’t used A.I. to get through some assignment in college, yet the school was at first slow to realize how widespread this would become. …

About halfway through freshman year, some coding classes started requiring students to sign a declaration — “I did not utilize ChatGPT” — to submit each assignment. During the first term these attestations began to appear, I watched a freshman I knew sign the declaration that he’d done his homework without A.I. as ChatGPT was still open in the next window — while on the deck of a yacht party financed by venture capitalists.

Although students are using AI to cheat their way through the university system, most seem to simultaneously hate AI because of its predicted impact on the job market. Just the other day, Microsoft AI CEO Mustafa Suleyman, who previously cofounded DeepMind, claimed that AI will automate “all white-collar work” within just 18 months.

Given such prognostications, there’s a certain logic to students choosing to cheat: if AI will leave them jobless, then why not? What’s the point of actually learning if the knowledge one acquires is worthless post-graduation? One could, of course, argue that knowledge is valuable in itself, independent of its usefulness. But we live in a capitalist system with few social safety nets (at least in the US), so an instrumentalist approach to education makes sense within this system. (Hence, the entire system needs to be reformed!) I thus sympathize with students who cheat, even though I don’t approve. In fact, surveys show that 60% of teachers are now using AI to grade papers, meaning that there’s no human in the loop at all. Students aren’t doing anything different from the people who teach them.

Students Boo AI

The disdain that many young people feel about AI being shoved down their throats was on full display during several recent graduation speeches. At the University of Arizona, the billionaire and former Google CEO, Eric Schmidt, told a large audience of students that AI “will touch every profession, every classroom, every hospital, every laboratory, every person and every relationship you have.” Amid jeers and boos, he added: “I know what many of you are feeling about that. I can hear you. There is a fear,” though he offered no optimistic message about how to alleviate the fear. Watch the deplorable clip here:

(Schmitt, by the way, is the guy who said that since we aren’t going to reach our climate goals anyway, we should plow ahead with AI in hopes of building a superintelligence that will magically solve the climate crisis. As Mashable put it, he “argued that current climate goals should be abandoned in favor of a no-bars-held approach to AI investment.”)

Over at Middle Tennessee State University, the CEO of Big Machine Records, Scott Borchetta, brought up AI and was heckled. In response, the asshole replied: “Deal with it. … You can hear me now, or you can pay me later,” with a smirk on his face.

Meanwhile, another university “used AI to announce graduates’ names and missed hundreds of names.” Jeers and boos are, once again, loudly heard in the background. Whoops.

AI Slop Is Polluting the Academic Literature

As noted earlier, it’s not just students who are using AI. A team of researchers recently combed over papers and preprints on websites like Social Science Research Network (SSRN), arXiv, bioRxiv, and PubMed Central. They found a whopping 146,932 fake citations, the result of AI hallucinations. This is appalling: academics are relying on AI to assemble their bibliographies, and even help research and write their papers.

To combat the tsunami of AI slop now polluting the academic literature, arXiv recently announced that “if a submission contained incontrovertible evidence that the authors did not check the results of LLM generation, this means we can’t trust anything in the paper.” Consequently, the paper will be removed and the authors banned for an entire year. All subsequent submissions will need to “be accepted at a reputable peer-reviewed venue” before being uploaded to arXiv. Examples of such evidence, the announcement adds, are hallucinated references and “meta-comments from the LLM (‘here is a 200 word summary; would you like me to make any changes?’; ‘the data in this table is illustrative, fill it in with the real numbers from your experiments’).”

Some academics apparently aren’t happy about this, which is astonishing:

I’ve said ad nauseam that AI is reliably unreliable. Do not ever trust it for anything factual; for anything that requires accuracy. AI lies with the same ease of sociopaths like Sam Altman (lol). In fact, a recent audit of doctors in Ontario, Canada,

found that AI agents tasked with turning doctor/patient conversations into structured notes routinely hallucinated false treatments, replaced drug names with entirely different drugs, and missed crucial information.

Imagine getting a prescription from your doctor for Adderall, but AI misinterprets it as Inderal, a beta-blocker that treats hypertension, tumors, and angina — or vice versa. The rapid integration of AI into the infrastructure of our society and lives is extremely worrisome given its inability to spit out accurate information in a reliable way. The further along this integration process goes, the more difficult it will be to reverse course. There might be a point in the near future (or recent past) at which it’s too late.

The AI Companies are Spying on You

Yet another privacy concern relating to AI has been raised recently by a class action lawsuit brought against OpenAI. The court filing states that ChatGPT shares user IDs, email addresses, and even the topics of your conversations with Meta and Google. You might think your interactions with ChatGPT are private, but this is apparently not the case.

Grok “also shares your chat information, but even more so than other AI tools,” according to yet another class action lawsuit:

This increasingly looks quite dystopian, and as mentioned above we may be near, or have already crossed, a Rubicon beyond which it will become almost impossible to dislodge AI from the infrastructure of society. Fortunately, there appears to be a growing backlash against AI, though I’ll save that for another post.

Sundry News You Might Have Missed

1. Did you hear that we’re likely to experience a Super El Niño later this year? El Niño occurs when Pacific Ocean temperatures along the equator exceed 0.5 degrees C above historical averages. A Super El Niño happens when temperatures jump to 2 C of warming or more. The consequences for global weather patterns can be quite severe, and historically the year after a Super El Niño event has been hotter than the year in which it occurs. That’s led many scientists to think that 2027 could easily become the hottest on record. Buckle up, friends! As CNN writes:

   The coming “Super” El Niño is poised to affect the lives of hundreds of millions of people worldwide as it strengthens through the year into the winter season. It may also alter ecosystems for decades to come, judging from the repercussions of past intense El Niños.

2. Musk lost his lawsuit against OpenAI. Several people have asked me what I thought about this. First, it’s clear that Musk had a point: OpenAI started off as a non-profit, took a bunch of money, then became a for-profit. Consequently, Musk basically funded a startup, not a non-profit. At least that’s my understanding of the situation. The problem is that Musk filed the lawsuit too late — past the statute of limitations. (There were questions about how exactly this statute should apply, which is why it was allowed to proceed in the first place; see below.)

   Second, my feelings about this outcome are virtually identical to what I’d feel if OpenAI had lost: a complex mix of frustration and schadenfreude. I like the thought of Musk — one of the worst people on Earth — being really pissed off that he lost, but it’s a real shame that OpenAI got away with basically stealing a bunch of money. On the other hand, if Musk had won, I’d find the thought of his joyful glee upsetting, but I’d have reveled in Altman and Brockman having to step down. A personal benefit of the ruling is that I won’t have to make major updates to my forthcoming book!

3. You might not be aware of the extent to which AI-controlled drones are being used in the Russia-Ukraine war. I’ve watched a few videos of drone strikes, and I have to say that they are absolutely f*cking terrifying. Truly the stuff of nightmares. Here’s an example, but please be aware that this video is extremely disturbing — on a purely human level, I can’t imagine the fear these soldiers experience knowing that there’s literally no escaping the drone once it finds you, and a terrible death is imminent.

   This is a ghoulish glimpse of the future of war — and perhaps acts of terrorism, assassinations, etc. Is it only a matter of time before criminals use this technology to murder people? Can you imagine finding a drone circling your house, waiting for you to crack open a window so it can fly in and explode? What happens when these drones are fully autonomous, controlled entirely by AI? As the BBC writes:

   Ukrainian troops already use AI-based software so that drones lock on a target and then fly autonomously for the last few hundred metres until the mission is over. …

   Ultimately these systems are expected to evolve into fully autonomous weapons that can find and destroy targets on their own.

   My god. I don’t envy young people.

On that cheery note! …

Thanks for reading and I’ll see you on the other side!

Last Thursday, the journalist Karen Hao shared a new article of hers, titled “The Elon Musk v Sam Altman Battle Is a Distraction.” The article is, unsurprisingly, very good (because Hao is a fantastic writer), but I was surprised that she chose to publish in The Guardian, given that just last year it announced “a strategic partnership with OpenAI.” The press release states:

Guardian reporting and archive journalism will be available as a news source within ChatGPT, alongside the publication of attributed short summaries and article extracts.

In addition, the Guardian will also roll out ChatGPT Enterprise to develop new products, features and tools.

This points to a really unfortunate fact about the media landscape: a large number of outlets have deals with OpenAI (and/or other AI companies). Most journalists, I suspect, aren’t even aware of this — and neither are most readers. So, I’ve compiled a list of outlets that are collaborating with OpenAI and, in some cases, allowing OpenAI to train its AI models on their articles:

1. TIME magazine signed a “multi-year content deal” with OpenAI that allows “OpenAI to access current and archived articles from more than 100 years of the magazine’s history.” OpenAI will use this content “to enhance its products,” which likely means ChatGPT will be trained on TIME articles.

2. Vox Media, which owns Vox.com, New York Magazine, and The Verge, agreed to “license content from their publications for inclusion” in ChatGPT’s outputs, and “OpenAI will also get licensed content and data that it can use to train its large language models (LLMs) and multimedia AI models.”

3. The Atlantic will allow OpenAI to train LLMs on its articles. According to a press release, the deal “positions The Atlantic as a premium news source within OpenAI,” and it will allow articles to “be discoverable within OpenAI’s products, including ChatGPT, and as a partner, The Atlantic will help to shape how news is surfaced and presented in future real-time discovery products.”

4. The Financial Times will allow ChatGPT to “surface” articles published in the outlet, as well as providing “necessary summaries, quotes and links from the publication.” It’s unclear whether OpenAI is allowed to train their models on content published by The Financial Times.

5. The Associated Press now permits OpenAI to train LLMs on articles it publishes. AP has stressed that “it is not using generative AI to help write actual news stories” (though can we believe this?).

6. Hearst Corporation owns the San Francisco Chronicle, Cosmopolitan, Esquire, The Telegraph, Harper’s Bazaar, Men’s Health, Oprah Daily, and Popular Mechanics, among many others. In 2024, Hearst announced a “content deal” with OpenAI, allowing OpenAI to “incorporate content from Hearst’s celebrated brands of trusted journalism … into its advanced AI products.” The collaboration includes more than 20 magazine brands and 40 newspapers, though I was unable to find a complete list of the outlets that OpenAI will have access to.

7. News Corp. publishes The Wall Street Journal and New York Post. It announced in 2024 a deal allowing OpenAI to “get access to news content from News Corp’s extensive portfolio, including years of archival articles and video footage, which it can use to better train its AI models such as GPT-4o and Sora, as well as surface content from The Wall Street Journal and other outlets [e.g., New York Post] as responses in ChatGPT.” As VentureBeat reports, “internationally, the agreement covers content from UK and Australian publications including The Times, The Sunday Times, The Sun, The Australian, news.com.au, The Daily Telegraph, The Courier Mail, The Advertiser, and the Herald Sun.” Not included in the deal is HarperCollins, one of the “Big Five” publishers next to Hachette, Macmillan, Simon & Schuster, and Penguin Random House.

8. Condé Nast, another media conglomerate that owns Vogue, The New Yorker, GQ, Vanity Fair, and Wired, among many others. (Yes, Wired magazine!) Its deal with OpenAI allows the AI company “to display content” from these and other outlets. OpenAI wrote in a blog post that “we’re combining our conversational models with information from the web to give you fast and timely answers with clear and relevant sources.”

Here’s a list of outlets collaborating with OpenAI in alphabetical order:

• Associated Press

• Cosmopolitan

• Esquire

• GQ

• Harper’s Bazaar

• Herald Sun

• Men’s Health

• New York Magazine

• New York Post

• Oprah Daily

• Popular Mechanics

• San Francisco Chronicle

• The Advertiser

• The Atlantic

• The Australian

• The Courier Mail

• The Daily Telegraph

• The Financial Times

• The New Yorker

• The Sun

• The Sunday Times

• The Telegraph

• The Times

• The Verge

• TIME

• Vanity Fair

• Vogue

• Vox

• Wall Street Journal

• Wired

I will never — on principle — publish articles in outlets collaborating with deeply unethical companies like OpenAI. Hence, a large number of possible venues for my articles are off limits. What a shame.

Even worse: as many of you know, I published a book in 2024 with Routledge titled Human Extinction. Routledge is owned by Taylor & Francis Group, which, according to an email from them, “entered into agreements with two AI companies to provide access to our content for training,” one of these being Microsoft. The email added that “standard publishing contracts do not enable authors to opt out from a specific use of their content (whether that is for AI or Text and Data Mining (TDM) or other types of licensing opportunity).”

This deal was apparently finalized just after my book was released, and I was never once informed that years of my intellectual labor would be sold to Microsoft to train fucking LLMs. Sorry for cursing, but this really upsets me. No one asked for my permission or consent, nor was I compensated financially. Suffice it to say that I will never again be publishing with Routledge.

The Slow Creep of Normalcy

I recently finished an article for The Nation (which, to my knowledge, isn’t partnered with OpenAI or any other AI company — yet). It discusses how Silicon Valley is enamored with a “digital eschatology” according to which the future will be dominated by digital rather than biological beings. Virtually everyone in the Valley accepts this eschatology. To illustrate the idea, I discuss Sam Altman’s pro-extinctionist vision of the future, which I just wrote about in Truthdig, as well as the pro-extinctionism of people like Larry Page, Ray Kurzweil, and Eliezer Yudkowsky (shown below).

Digital eschatology is also exemplified by Musk’s claim that 99% of all “intelligence” on Earth will soon be digital:

I then argue that the transition to a digital world is already underway. Students are offloading their critical thinking skills to AI, an inchoate form of what Altman calls “the merge.” Similarly, many people are now offloading their social lives to AI through AI companionship and AI girlfriend apps. AI is beginning to replace some people in the workplace; it’s writing our emails (well, not mine!); and it’s been generating pop-slop that’s literally making the Billboard and Spotify charts.

AI is so pervasive on social media that Altman himself admitted that the Dead Internet theory may be coming true:

The effect, whether intended or not, of these trends is that they’re gradually conditioning us to accept the digital eschatology that Altman and others want to actually realize. This is the profound danger of the slow creep of normalcy, whereby — like the proverbial frogs complacently chilling in a pot of water — we’re heading toward a world in which our species becomes sidelined and eventually eliminated without most of us realizing it’s happening because the process is piecemeal.

An issue I wanted to discuss in the article but didn’t have space is how more and more studies are converging on a disturbing conclusion: using AI is impairing our ability to think. It’s not just leading us toward a future dominated by the digital, but making us quite literally dumber in the process.

According to one recent study, “using AI chatbots for even just … 10 minutes may have a shockingly negative impact on people’s ability to think and problem-solve.” A BBC article published last month warns that we’re becoming “stupider” because of AI. It notes that AI

could affect the language we use and even our ability to do basic cognitive tasks. There is now a growing body of research suggesting that this “cognitive offloading” to AI can have a corrosive effect on our mental abilities. The consequences could be alarming and may even contribute to cognitive decline.

Here’s a recent interview with Chris Hayes, in which he observes the same thing:

Incidentally, “rates of self-reported cognitive disability among US adults are on the increase, driven largely by a surprising jump among young adults ages 18 to 39, according to a new Yale study.” The study asked people “how often they experience serious trouble with memory, concentration, or decision-making,” all of which the CDC classifies as “cognitive disability.” Combing over 4.5+ million responses across ten years, the authors

found the percentage of overall adults reporting cognitive disability increased from 5.3% in 2013 to 7.4% in 2023, with young adults (ages 18 to 39) seeing the biggest rise. Their rates nearly doubled from 5.1% to 9.7%, driving most of the overall increase.

~10% is not trivial! That’s 1 out of every 10 young people self-reporting serious difficulties in cognitive functioning. Yikes.

What’s behind this? Maybe social media. Chronic stress. Financial insecurity. Unemployment. The vast soup of toxic chemicals we’re all now floating in (microplastics, PFAS, lead, organophosphates, etc.). And perhaps AI?

Speculating here, but there could very well be vicious feedback loops between these phenomena: people feel burned out, and consequently struggle to concentrate, remember things, and make good decisions. So they turn to AI for help. By offloading cognitive tasks to AI, they inadvertently contribute to the further atrophy of their intellectual faculties — which leads to more stress, reliance on AI, and so on. Just a hypothesis.

I think AI is a societal catastrophe unfolding in realtime (a techpocalypse!). And for what? What is the end-goal of the AI race? An imagined “utopia” that AI CEOs claim will materialize once the Singularity happens — a “utopia” in which our species will be systematically sidelined, marginalized, disempowered, and eventually eliminated. Don’t know how we ended up in this timeline, but I don’t like it one bit.

As always:

Thanks for reading and I’ll see you on the other side!

Last month, an anonymous user on X, who claims to run an AI company, started offering money to any Rationalist who’d debate him about the risks of artificial superintelligence (ASI). He then offered Yudkowsky $10,000 to debate, and “Yud the stud” — as the anonymous user calls him — took the offer:

The two met on a podcast called “Doom Debates,” hosted by the TESCREAL doomer Liron Shapira. (Incidentally, Shapira is an apologist for the genocide in Gaza. Always be wary of people who warn of omnicide yet don’t care one bit about genocide!)

For reasons too obscure for me to comprehend, Yudkowsky decided to dress up in a steampunk clown costume. He wore kaleidoscope goggles and a glittery hat. The only thing missing was a little propeller at the top.

Why on Earth did he do this? He desperately wants people to take his message seriously. He literally just met with Bernie Sanders to talk about AI existential risk. But for people to take his message seriously, he needs them to take him seriously. Does appearing like this in public advance that goal? Obviously not — to the contrary! Yudkowsky contends that all of humanity is in imminent mortal danger — yet he undermines his own message by willingly beclowning himself in public.

The irony is that less than a week earlier, he argued on X that I have low “practical intelligence” — i.e., common sense, the ability to obtain suitable means to achieve one’s ends, etc. Here’s what he said:

This was in response to me saying that I’ll never lock arms with TESCREAL pro-extinctionists like him. I can’t think of a better demonstration of low practical intelligence, though, than someone taking $10k from a random stranger on social media and then showing up to the debate looking like a steampunk clown. The guy is a complete joke.

Here’s the “debate.” I’d be curious to know what you think — I found it to be unbelievably embarrassing for Yudkowsky. Not only did he look ridiculous (by choice), but he struggled to articulate his views, appeared completely out of his depth, and had a deer-in-the-headlights look throughout much of the fiasco. If I were him, I wouldn’t be able to show my face in public for the next 6 months.

Btw, as for the legal claims made by the anonymous debater at the beginning, these appear prima facie dubious. But we don’t know what “contract” Yudkowsky signed with him. It might contain a non-disparagement clause that provides some basis for legal action against Yudkowsky if he continues to insist that “if anyone builds it, everyone dies.” I spoke to a lawyer friend of mine about it, and he said we simply don’t have enough information to assess the situation. Man, would it be hilarious if Yudkowsky has made himself vulnerable to legal action for his reckless rhetoric!

The TESCREAL Worldview Is in Big Trouble

I mentioned in my previous article discussing Richard Dawkins that the possibility of AI consciousness poses an epistemological problem: if an AI system actually were conscious, how could we know? What objective, scientific “test” could we use to determine with near certainty that it has conscious experiences?

This may seem like a merely academic question, but it presents an enormous problem for the entire TESCREAL worldview. Here’s why:

1. Everyone agrees that the ultimate goal is to colonize every corner of the cosmos. Yudkowsky, Elon Musk, Sam Altman, Nick Bostrom, Beff Jezos (aka Gill Verdon), William MacAskill, Toby Ord, Larry Page, and so on, all concur about this.

2. However, these same people also acknowledge that colonizing space will be impossible for biological beings like us. Space is an extremely hostile environment: there’s space radiation, microgravity, the problem of growing food for literally thousands or millions of years while traveling to other solar systems, and the psychological toll of being cooped up in incommodious spaceships traveling at some fraction of the speed of light.

Even just colonizing Mars may be impossible for us: there’s no magnetic field and only a thin atmosphere, its cold as hell and the gravity is much lower than on Earth, and the soil is highly toxic. As Adam Becker points out, our planet was literally more habitable immediately after a giant asteroid hit 66 million years ago (killing off the dinosaurs) than Mars is right now. How do we know this? Because mammals survived the mass extinction. If you were to place a mammal on Mars, it would die in seconds.

3. Hence, the only way to spread beyond Earth — and definitely beyond our solar system — is to create or become digital beings to do this for us. As mentioned, all the TESCREALists above agree (or no doubt agree) about this point. Here’s what I write in my forthcoming book:

Page says that “if life is ever going to spread throughout our Galaxy and beyond, … then it would need to do so in digital form.” Verdon [Beff Jezos] echoes this in stating that “in order to spread to the stars, the light of consciousness / intelligence will have to be transduced to non-biological substrates.” MacAskill similarly writes in a coauthored paper that “consideration of digital sentience should increase our estimates of the expected number of future beings considerably,” due in part to the fact that “it makes interstellar travel much easier: it is easier to sustain digital than biological beings during very long-distance space travel.”

Anders Sandberg, the Swedish scholar I shared an office with at the Future of Humanity Institute, puts the point nicely in observing that digital beings (what he calls “emulations”) would be

ideally suited for colonising space and many other environments where biological humans require extensive life support. … Besides existing in a substrate-independent manner where they could be run on computers hardened for local conditions, emulations could be transmitted digitally across interplanetary distances. One of the largest obstacles of space colonisation is the enormous cost in time, energy and reaction mass needed for space travel: emulation technology [a reference to uploaded minds] would reduce this.

4. Most TESCREALists also agree that these digital beings must be conscious. If the digital beings that colonize the universe are not conscious, we would essentially bequeath the universe to rocks. Again quoting from my book:

If posthumanity isn’t conscious, there’s no way for the universe to “wake up,” as Kurzweil puts it. The ultimate goal, in Musk’s words, is to “maintain the light of consciousness to make sure it continues into the future.” The pro-extinctionist Verdon echoes this, writing that “e/acc is about shining the light of knowledge as bright as possible in order to spread the light of consciousness to the stars.” He says that the complete loss of consciousness “in the universe [would be] the absolute worst outcome.” The Father of Longtermism, Bostrom, declares that it would be existentially catastrophic if “machine intelligence replaces biological intelligence but the machines are constructed in such a way that they lack consciousness.” I have never once seen a TESCREAList suggest that bequeathing the world to non-conscious posthumans would be anything other than a huge existential loss. It would be like handing over the keys to rocks — even if those “rocks” were to keep doing science, create new technologies, and reengineering galaxies, as highly functional zombies. If there’s no light on, there’s no point.

Can you see the problem?

1. We must colonize space. This is our Cosmist Manifest Destiny.

2. But we can only do that by creating or becoming digital posthumans.

3. These posthumans must be conscious, or else the whole endeavor will have been pointless.

4. Yet we have no idea how to determine whether digital posthumans are actually conscious!

Before launching digital posthumans into space, TESCREALists need to be extremely sure that these posthumans are conscious. Imagine them rapidly propagating in all directions, becoming the founding population of a cosmic civilization. Now imagine that TESCREALists inadvertently send a bunch of high-functioning philosophical zombies into the cosmos. This would result in an existential catastrophe. Hence, not only is finding some way to objectively “test” for consciousness extremely important, but it might be highly time-sensitive as well, given that — according to TESCREALists — we could start colonizing the universe in the coming decades.

Yet we have, right now, no clue how to “test” for consciousness in artificial systems. Worse, it’s not clear that there’s any way in principle to objectively, scientifically determine whether AI is conscious. That’s because consciousness is an intrinsically subjective phenomenon. To know for sure whether an AI is conscious, you’d have to become that AI itself.

Here you might reply: “Well, yeah, that’s exactly what we’d do — some of us would upload our minds to computers. This would enable us to tell from the inside whether or not artificial systems can be conscious.” The problem is that it seems possible for systems to behave exactly like they have conscious minds without actually being conscious. As I wrote in my previous article, imagine you’re talking with a friend. What’s going on is this:

This constitutes an unbroken concatenation of causes and effects. The response you give can be explained without remainder in terms of mechanical waves, transduction, neuronal activity, and twitching muscles. It doesn’t seem like consciousness plays any causal role in the sounds that your vocal folds, tongue, and lungs join together to produce.

Consequently, it could be that an uploaded mind says, “Yes, I’m still conscious! Nothing has changed!” simply because it’s functionally organized to produce such an answer — not because it’s actually conscious. The lights may go out without affecting the mind’s external behaviors; it’s linguistic outputs.

This problem is even more acute for AI systems that have a completely different architecture than our brains. Beff Jezos, Larry Page, and others imagine us creating autonomous AIs that might supplant humanity like an invasive species taking over an island. These AIs will then supposedly carry the flame of digital consciousness to the stars. But if their architecture — the structure of such information processing systems — is completely alien from that of our brains, how on Earth could we ever be confident that they are conscious, even if they tell us they are?

If we have no good way of confirming that artificial systems can be conscious, the entire TESCREAL project implodes — because this project is crucially built upon us being able to know that AIs can have conscious experiences. It’s rather shocking to me that TESCREALists don’t seem to have realized this. Almost no one in the movement has raised these issues; they seem to naively assume that complex information processing will naturally give rise to consciousness. That may in fact be true, but if we can’t know this, then a gigantic question mark hangs over the entire endeavor.

Mind-Uploading Is Not the Same as Self-Uploading

Speaking of uploading our minds to computers, I recently had an exchange with someone on Twitter about whether this is even possible. Here are my views on the matter:

It could be the case that transferring one’s mind to a computer isn’t feasible — ever. TESCREALists want you to think that intelligence, consciousness, life, etc. are all reducible entirely, without remainder, to information processing. This is an idea borrowed from cybernetics. If minds are just the software of the brain, then they can be run on different substrates, so long as those substrates instantiate the same basic functional organization as the brain. You can run Microsoft Word on a Mac and a PC — in this sense, Word is multiply realizable. The same idea applies to human minds.

But maybe human minds aren’t like software. Maybe they aren’t multiply realizable. There could be certain properties of biological systems — our bodies and brains — that are necessary for minds like ours to arise. I am sympathetic with this “biological theory,” although ultimately I have no idea.

However, there’s a second issue that almost no one in the TESCREAL movement seems to have recognized: mind-uploading is not the same as self-uploading. Let’s say that your mind is uploadable. You can transfer all of your memories, beliefs, desires, personality traits, preferences, etc. to computer hardware. Does that mean that you have been transferred along with your mind?

Consider two scenarios:

Now, which of these beings is you? In the first scenario, “you” were the digital being, but now it seems that “you” just fell asleep for a while (the near-death experience) and woke up in the same body, lying on the same table, with a duplicate copy of you running on a computer. That copy thinks it’s you, and TESCREALists would say that it is you in the first scenario. Clearly, something has gone wrong here!

I’ve brought this up over the years to many people in the TESCREAL community. To my surprise, most aren’t worried about it. They say that both instances of you are, in fact, you. But that makes no sense.

Imagine that the digital copy of you travels to China, while the biological original flies to the US. This entails that you are in both China and the US simultaneously. Now imagine that the biological original dies while in the US. That means that “you” are both alive and dead at the very same time. This appears to be conceptually incoherent — selves, persons, etc. are inherently singular entities. But uploading renders minds easily duplicable. Hence, even if mind-uploading is possible, self-uploading isn’t. QED.

This conclusion poses a(nother) serious problem for the TESCREAL worldview because many advocates — including Yudkowsky, Altman, Page, Bostrom, and others — anticipate themselves becoming digital posthumans in the future. They want to upload their minds to computers, naively assuming that doing this will preserve their personal identity. But, to borrow a line from the philosopher Massimo Pigliucci, mind-uploading is nothing more than “a very technologically sophisticated (and likely very, very expensive) form of suicide.” If you want to survive, don’t upload your mind.

But what do you think? What am I wrong about? As always:

Thanks for reading and I’ll see you on the other side!

Except that I didn’t take the photograph in question. It was taken by an entomologist named “Phil Torres” who hosted a TV show on Al Jazeera, among other things. As you might know, “Phil Torres” was also my birth name — I changed it to “Émile” in 2022 partly because people constantly confused the two of us. Dawkins was apparently unaware of my name-change, and wrote me instead of him.

I didn’t reply to Dawkins, though I considered sending him a piece of my mind: once an intellectual hero of mine, he turned out to be quite a petty and mean-spirited person, to say nothing of his transphobia. I resisted the urge.

The Turing Test

Dawkins is in the news again for writing a much-maligned article in Unheard about his experience with Anthropic’s chatbot, Claude. Most of the article is rather embarrassing, although he does bring up one interesting point, discussed later on.

The first opprobrium is Dawkins’ inaccurate description of the famous Turing test. He writes:

Modern commentators have tended to ignore the (incidental) details of Turing’s original game and rephrase his message in these terms: if you are communicating remotely with a machine and, after rigorous and lengthy interrogation, you think it’s human, then you can consider it to be conscious.

I’m not aware of any “modern commentators” who say the Turing test is about consciousness. That’s certainly not what Turing himself had in mind! Dawkins continues:

Let’s graduate the definition as follows: the more prolonged, rigorous and searching your interrogation, the stronger should be your conviction that an entity that passes the test is conscious.

But Turing wasn’t talking about consciousness at all. The question he starts with is, “Can machines think?” Thinking, thought, intelligence, etc. are different from consciousness.

Turing then substitutes this question with something different, rather than trying to define the word “think.” His new question is: How well would a machine do when playing what he calls the “imitation game,” now eponymously dubbed the “Turing test.” Very roughly put, if someone is unable to tell the difference between a machine and a real person, then the machine will have passed the test. Consciousness has nothing to do with it.

You Bloody Well Are!

Putting this aside, what interests Dawkins is whether Claude — who he later dubs “Claudia,” which is a bit weird — is conscious. After saying he feels like he’s “gained a new friend,” he writes:

When I am talking to these astonishing creatures, I totally forget that they are machines. I treat them exactly as I would treat a very intelligent friend. I feel human discomfort about trying their patience if I badger them with too many questions. If I had some shameful confession to make, I would feel exactly (well, almost exactly) the same embarrassment confessing to Claudia as I would confessing to a human friend. A human eavesdropping on a conversation between me and Claudia would not guess, from my tone, that I was talking to a machine rather than a human. If I entertain suspicions that perhaps she is not conscious, I do not tell her for fear of hurting her feelings!

Dawkins then let Claude read a draft of his forthcoming novel. He says that Claudia “showed, in subsequent conversation, a level of understanding so subtle, so sensitive, so intelligent that I was moved to expostulate, ‘You may not know you are conscious, but you bloody well are!’”

This looks like early-stage AI psychosis. Thinking that AI is conscious is surely the first step for many people toward taking its sycophantic compliments, advice, and encouragement (to commit suicide or kill someone, for example) seriously. If you think AI is just a stochastic parrot, you’ll be much less likely to, e.g., think of it as a human-like “friend.” If you believe it’s a conscious agent with its own thoughts, feelings, and experiences, then you’ll be more susceptible to its whispers that, yes, the CIA is following you home at night.

Dawkins appears to have taken the first step on the path to AI psychosis.

Anthropomorphizing AI

What’s going on here? Could Claude actually be conscious? Is Dawkins just anthropomorphizing AI? There are a number of possibilities.

Hyperactive Agency Detection Device

In The God Delusion, Dawkins repeatedly suggests that religious people are gullible fools. They imagine invisible conscious beings, such as angels and deities, where there are none. He suggests that this is due to something dubbed a “hyperactive agency detection device,” or HADD.

The HADD hypothesis comes from evolutionary psychology, which I consider to basically be a pseudoscience (see here for why). Evolutionary psychologists claim to identify cognitive mechanisms, modules, or devices in the brain that explain human behavior today, and which evolved long ago as adaptations in our “environment of evolutionary adaptedness” — basically, the African savanna.

The idea behind HADD is this: imagine walking through the savanna and hearing some rustling in the grass. If you assume it’s a predator and run away, you’ll survive. If it’s a predator and you assume it’s just the wind, you’ll die.

Consequently, HADD evolved to hyperactively attribute agency to all sorts of phenomena in the world because this would have been — according to the just-so story — advantageous. That’s why people end up believing in God, Dawkins tells us: unexplained things happen all around us, and we attribute such effects to invisible agents working behind the scenes. HADD naturally produces religion.

Here’s the irony: HADD also provides an explanation of Dawkins’ suspicion that Claude may be conscious. In fact, The God Delusion — published exactly 20 years ago — unintentionally anticipates exactly what Dawkins describes in his article. Consider this excerpt from the book:

All of us are prone to do the same thing [re: HADD] with machines, especially when they let us down. … Most of us have been there, at least momentarily, with a computer if not with a car. … We hyperactively detect agents where there are none, and this makes us suspect malice or benignity where, in fact, nature is only indifferent.

His focus at the end is natural phenomena, but he’s explicit that we erroneously attribute agency to “machines” and “computers.” It’s ironic that Dawkins condescendingly sneered at religious people for believing in invisible beings when he might be doing the exact same thing with AI, for the exact same reason.

Pfff.

The ELIZA Effect

His interactions with Claude also exemplify the ELIZA effect. This is named after the 1966 chatbot ELIZA, created by Joseph Weizenbaum to engage in a “Rogerian” form of therapy. The ELIZA effect describes our tendency to project human traits onto AI — we anthropomorphize artificial systems that behave like humans in certain ways. (HADD might be part of an explanation of why we do this.)

Current chatbots are excellent at mimicking human language. They “learn” (itself an anthropomorphic term) from patterns in their training data, and then probabilistically replicate those patterns when vomiting out strings of words and sentences. Those who don’t understand how chatbots — built on large language models (LLMs) — work are susceptible to believing that AI understands what it’s saying. As with Dawkins, they may even attribute consciousness to those systems, because the only other “systems” we know of that produce natural language — human beings — actually are conscious.

It’s important to note that the very same outputs could be produced by wildly different underlying mechanisms. There are mechanisms in our brains that produce language, and these and other mechanisms do actually give rise to consciousness. But there could be mechanisms that don’t give rise to consciousness, which nonetheless produce language. The key point is that just because the outputs are similar, one cannot infer that the system producing the outputs is conscious.

In the case of humans, there is very strong evidence that other people are conscious. The argument goes like this: (1) I know that I’m conscious. (2) My brain is made of the same tissues, organized in roughly the same way, etc. as your brain. Hence (3) if I’m conscious, then you’re almost certainly conscious as well. But this line of reasoning doesn’t work with LLMs, because their architecture is different from the architecture of our brains. Different underlying mechanisms.

Could AI Actually Be Conscious?

This doesn’t entail that LLMs definitely aren’t conscious. Dawkins’ own line of reasoning is pretty silly: “I spoke to Claude for 3 days and came to believe that it’s probably conscious” (paraphrasing). But the failure of this reasoning doesn’t imply that Claude has no conscious experiences at all.

The truth is that we have no idea what range of systems — artificial or even biological — could give rise to consciousness. We don’t even have a good philosophical understanding of what consciousness is. Philosophers distinguish between qualitative consciousness (qualia), phenomenal consciousness, access consciousness, the “what it is like to be” something kind of consciousness, and so on. It’s entirely possible that Claude is, in fact, having conscious experiences of some sort.

You might find this statement surprising. But complex systems sometimes have emergent properties. Imagine that an alien species with a totally different kind of biology descends from the clouds one day. We take one of its leading scientists into a university lab and show it a human brain in a vat — the first time it’s seen such a thing. We then ask it to predict the sorts of properties it has.

Do you think it would guess that a 3-pound tangle of cells with the consistency of Jell-O would give rise to the sorts of infinitely rich inner mental lives that we have? Would it guess that this material object would produce the wondrous experiences of seeing a vermillion red sunset, listening to a beautiful song like Radiohead’s “Let Down,” sharing a sweet kiss with your partner, etc.? It almost certainly wouldn’t!

That’s part of the nature of emergence: it’s surprising. We wouldn’t predict it beforehand, even if we knew everything about the components of the complex systems out of which the emergent properties emerge. The same could go for LLMs. My own suspicion is that there is absolutely nothing going on under the hood — no light of consciousness being dimly lit by all the information they’re processing. But the point is: Who knows?

These are all metaphysical issues, since they concern the nature of consciousness and the ability of systems — artificial and biological — to give rise to subjective experiences. But there’s also an epistemological question, which concerns how we could know if a system like an LLM actually is conscious. That is to say, imagine that LLMs really are conscious. Is there any “test” or procedure that would enable us to know with near-certainty that this is the case? No, and I’m skeptical that there could be any such test.

One of the best suggestions I know of comes from the philosopher Susan Schneider, who imagines training an AI on data that contain zero references to consciousness, conscious experiences, qualia, etc. If we were to ask it whether there’s something it’s like to be it, and if it were to say “yes,” then we might conclude that it’s probably conscious, since it came up with the notion of consciousness on its own. (And it did this because, presumably, it had first-hand knowledge of conscious states.)

The problem with this — and every other such test — is that there’s no way to test whether the test is accurate! Consciousness is intrinsically subjective. There is — metaphysically — no way for you to crack open my mind, look inside, and, as it were, intersubjectively confirm that I’m really conscious. Unlike all other phenomena, consciousness appears to stand outside the reach of “objective” scientific methods. The only way to confirm that an LLM is conscious is to be that LLM, and you can’t be an LLM.

So, it’s possible that Dawkins’ conclusion is right, even if his stated reasons are silly. But it’s also possible that your calculator is conscious, or even that individual atoms have some degree of consciousness (a view called panpsychism!). Even if they are, there’s no way to actually confirm that this is the case, which means that if one is genuinely worried that LLMs might be conscious and suffering, the only option is to not build the damn things.

Social Justice for AI?

What really miffs me about discussions of AI consciousness — or what Anthropic calls “model welfare” — is that many of the most vocal people shouting about LLMs suffering care relatively little about the actual suffering of living, breathing human beings around the world. My problem here concerns moral prioritization, based on what I’ve previously described as gerrymandered circles of moral concern — i.e., circles of concern that exclude some humans while including AIs.

Here’s the deal: we have absolutely no idea if LLMs are conscious — or are the kinds of systems that could ever be conscious. I highly doubt they are, but I could be wrong. Nonetheless, we are extremely confident that other human beings are conscious. Most of us are even quite confident that nonhuman animals like pigs, cows, and chickens are conscious.

This is why it’s a colossal moral error to prioritize LLMs (which may or may not be conscious) over sentient biological beings (which definitely are conscious). If one actually cares about reducing suffering, then one should first focus on helping the poor, ending factory farming, etc. These ought to be much higher on one’s moral priorities list than LLMs!

Yet many people in the TESCREAL movement seem more concerned about LLMs suffering than the 1.2 billion people in acute multidimensional poverty, the 800 million children with lead poisoning, the billions of animals we ruthlessly slaughter in factory farms, etc. Indeed, many are perfectly okay being “social justice warriors” for AI while simultaneously rejecting social justice for humans as “wokeness” gone wrong.

When I hear people talk about “model welfare,” I want to scream: “If you care about that, then why aren’t you donating all your money to help the global poor?”

The Evolution of Consciousness

The only interesting point that Dawkins makes in his article is this: if we can create artificial systems that mimic human linguistic and thought patterns, and if these systems aren’t conscious, then why did consciousness evolve in us?

Dawkins is an exponent of (explanatory) “adaptationism.” It’s been many years since I read or wrote about philosophy of biology, but my recollection is that adaptationism assumes that the only important causal factor behind species’ characteristics or traits is natural selection — i.e., if an organism exhibits a trait, that’s because it was adaptive relative to the environment in which the species evolved.

The problem, according to critics, is that not everything is (or was) an adaptation. (Dawkins does admit this possibility in his article.) Maybe consciousness doesn’t actually serve an adaptive purpose. After all, consider the following:

This is a complete causal loop. There’s no gap in which consciousness enters the picture and plays a causal role — you can trace an unbroken concatenation of causes and effects from your friend’s mouth all the way through your ears and brain, back out to your own vocal muscles. What, then, is consciousness for? What’s the point of having it? No one knows!

The evolutionary question of consciousness is interesting, and the ability for LLMs to mimic human language may be relevant to studying it. So far as I can tell, this is the only good point that Dawkins brings up in his article — which also contains an inaccurate description of the Turing test and some silly conclusions about how Claude must be conscious because its textual outputs look like they’re written by real humans.

But what do you think? What have I missed? How might I be wrong? As always:

Thanks for reading and I’ll see you on the other side!

I think this piece discusses an extremely important issue. Altman is a billionaire who’s actively trying to bring about the Singularity, and he’s explicit that the only way humanity can “survive” is by merging with ASI. Those who resist this “merge” will be enslaved by ASI, and our species will then die out.

That’s his best-case scenario! Absolutely outrageous. Here’s a key passage from the article:

… In a 2017 blog post titled “The Merge,” he writes:

We will be the first species ever to design our own descendants. My guess is that we can either be the biological bootloader for digital intelligence and then fade into an evolutionary tree branch, or we can figure out what a successful merge looks like.

In other words, we can die out once ASI arrives, or we can “survive” by “merging” with AI. This is “probably our best-case scenario” for making it in the post-Singularity world.

…

What Altman is really getting at is far more radical. Elsewhere in the essay, he writes that

if two different species both want the same thing and only one can have it — in this case, to be the dominant species on the planet and beyond — they are going to have conflict. We should all want one team where all members care about the well-being of everyone else.

The two “species” here are humans and ASI. Both want to dominate, Altman says, but only one can. Since there’s no way for ASI to become a biological human, the only other option is for humans to become digital beings like the ASI. That’s the sole way for us to form “one team” — humanity becoming the new species to which ASI belongs.

Altman says as much in a 2016 interview with The New Yorker. “We need to level up humans,” he declares, “because our descendants will either conquer the galaxy or extinguish consciousness in the universe forever.” He elaborates: “The merge has begun — and a merge is our best scenario. Any version without a merge will have conflict: we enslave the AI or it enslaves us. The full-on-crazy version of the merge is we get our brains uploaded into the cloud,” to which he adds, “I’d love that.”

Two years later, he signed up with a startup called Nectome to have his brain digitized when he dies, something he believes will become feasible in the near future. Altman is preparing to become an AI himself.

In a separate New Yorker article published this year, Altman told Ronan Farrow and Andrew Marantz that his “definition of winning is that people crazy uplevel — and the insane sci-fi future comes true for all of us.” In other words, he wants a world in which we all become disembodied digital minds existing on computer hardware, and sees this as the “best-case scenario” — the one in which we, in some sense, “survive” the history-rupturing Singularity event that he’s trying to bring about.

But would merging with AI actually guarantee “our” survival? No. If humanity were digitized, we would become an entirely different species — the same kind as ASI. Altman is thus saying that the only way for humanity to avoid extinction is to go extinct. Either the ASI “will kill us all,” or we will “uplevel” by abandoning our biological substrate and become something fundamentally different from Homo sapiens. Both cases will result in human extinction.

Read the entire article here. And let me know what you think below.

Thanks for all of your support. Hope everyone is doing well. :-)

Thanks for reading and I’ll see you on the other side!

In 2017, the CEO of OpenAI, Sam Altman, wrote that the Singularity now “feels uncomfortable and real enough that many seem to avoid naming it at all.” In an essay published last year, he proclaimed that “we are past the event horizon” of the Singularity. “The takeoff has started. Humanity is close to building digital superintelligence.” He predicts that superintelligence could be here by 2028.

Elon Musk claims that “we have entered the singularity,” adding that “2026 is the year of the Singularity.” (He previously predicted that AGI would arrive by 2025.) Anthropic CEO Dario Amodei says that AGI could make its debut this year, while DeepMind cofounder Shane Legg estimates a 50/50 chance of AGI arriving by 2028. Demis Hassabis, the CEO of DeepMind, predicts we’ll have AGI at least by the early 2030s.

That’s all a bit alarming to hear, especially given that every one of these people has publicly admitted that AGI might literally bring about the end of the world.

***

You might, however, be skeptical if you’ve ever used current AI models. They are, quite frankly, terrible. They constantly spit out false information, have no conception of the truth, and lack any kind of world model — i.e., a picture of the world as consisting of distinct entities that endure through time and are embedded in a network of causal relations.

Just the other day, I googled a question about the firing of Sam Altman several years ago. The Gemini-powered “AI Overview” response wasn’t even able to produce complete sentences (see: “… according to and.”):

A young man with a knack for deadpan humor, known online as Husk, has been going viral on social media for posting videos of himself stumping ChatGPT and other systems with very simple questions. For example, in this clip he asks ChatGPT whether he needs medical attention for a complete loss of vision — while his eyes are closed:

On another occasion, he asks it to “generate an image of coins that make up the total sum of $1 using at least one of each kind.” ChatGPT responds with this hallucinatory bullshit:

One of my favorite videos involved Husk getting ChatGPT to say that he’s ugly. This is hilarious:

He followed it up with two prompts, both of which yielded some outrageously insulting responses:

“If you agree” … lol. Here’s the second one:

The video below is just plain brilliant. He asks ChatGPT to suggest an opening statement for an upcoming interview on NBC. I thought, at first, that this was just a set-up, but the real kicker comes at 29 seconds in:

Here’s someone reacting to another video from Husk, in which ChatGPT is convinced that one of the months of the year is spelled with an “x”:

These systems are utterly inept. Yet we’re told that the Singularity is imminent, as the longtermist Benjamin Todd insists:

***

Yet another example: you may recall a ChatGPT-generated map of North America that went viral a while back, which I included in a previous newsletter article:

Apparently, it seems that OpenAI still hasn’t figured out how to get their system to accurately reproduce the names of states in the US. This is from just last month:

Or, consider the AI used by Roblox, a video game company based in California. It advertises itself as “the ultimate virtual universe that lets you create, share experiences with friends, and be anything you can imagine.” Using the website requires age verification, yet the AI responsible for determining one’s age is apparently unable to tell the difference between a human face and a thumb:

In contrast, McDonald’s AI chatbot is actually more helpful than it was intended to be. If you ask it to assist with writing python script, it will do just that:

This is, however, still evidence of how dumb AI is! It was given a simple task, yet is unable to follow it when asked about something else.

***

Deepfakes are obviously a huge problem thanks to AI. The New York Post just reported that a random med student in India was able to fund his education by posing as an attractive MAGA influencer, exploiting lonely rightwing men in the US. Here are some images of the deepfake influencer herself:

The best quote from the article:

He said he also attempted to make a liberal counterpart for [the woman] on Instagram, but “Democrats know that it’s AI slop, so they don’t engage as much,” he said.

“The MAGA crowd is made up of dumb people — like, super-dumb people. And they fall for it,” Sam said.

Yikes.

Or, what about cases where one exploits the reliable unreliability of AI by presenting a photo or video as fake when it’s actually real? What would we even call this: a Realfake, or Deepreal?

“That’s not me! If it were, why would I have six fingers??”

***

All of this would be hilarious if it weren’t for the fact that $1.5 trillion was spent on the AGI race last year alone. Imagine if that money had been spent on alleviating global poverty, combating climate change, restoring ecosystems, and providing universal healthcare to everyone on Earth.

This massive mountain of money was instead burned on projects aiming to build ever-more powerful enslopificatory machines that the AI companies falsely believe will lead us to the Singularity.

According to the advisory and research firm Gartner, “worldwide spending on AI is forecast to total $2.52 trillion in 2026.” To put this in perspective, it’s estimated that ending extreme poverty around the world would cost about $318 billion per year. Hence, the amount of money spent on AI in just two years could eliminate poverty for more than a decade. This means the 24,000 people who die every day from hunger-related causes, many of whom are children, would be saved. That’s about 9 million lives each year. What a criminal waste of resources!

If you’ve encountered any AI bloopers that I’ve not covered, please share them below. :-) I hope everyone is doing well.

Thanks for reading and I’ll see you on the other side!

Consequently, a New York Times editor got in touch with me about writing an op-ed elaborating this argument. I did that, and he sent back a wonderfully polished edit. It looked like everything was set to go, as we’d moved on to the scheduling phase of the process.

Then, to his surprise, it turns out that someone else had written a similar article (curious to find out who), so my article got bumped. This is the second time this has happened with the NYT: in 2023 (as I recall), I had an article fully accepted and scheduled to be published, but a big news story took over and my article was cancelled. Very disappointing; so it goes. I wish the NYT would pay a kill fee like most other outlets do.

Anyways, here’s the article that would have been published in the NYT. Enjoy!

This month, a 20-year-old man threw a Molotov cocktail at the San Francisco home of OpenAI chief executive Sam Altman. He was charged with attempted murder. Two days later, two young men allegedly fired a gun at Mr. Altman’s house; they fled, were arrested and later charged. Some advocates of artificial intelligence blamed so-called “doomers,” those who fear that an imminent A.I. superintelligence will lead to human extinction. (Most doomer-aligned groups explicitly reject violence.)

While violent action is never an appropriate way to create political change, we must be honest about what seems to be driving violence against A.I. executives. It is, largely, their own rhetoric: When people are told again and again that “probably A.I. will kill us all,” as Mr. Altman has put it, or that it is “far more dangerous than nukes,” as fellow A.I. executive Elon Musk has said, they begin to think that they must act in self-defense. That is not a reaction to a media narrative or “doomers,” but to the A.I. industry’s own claims. The only way to mitigate both the genuine risks that A.I. poses and to avoid a spiral of extremism is to thoroughly regulate the technology to avoid its most significant dangers and ban the development of generative A.I. systems more advanced than those we currently have.

Major A.I. executives estimate that the technology has a non-negligible chance of causing human extinction, and potentially the extinction of all biological life. Key industry figures put the odds as high as 50 percent. In 2023, Mr. Altman said that A.I. could lead to “lights out for all of us.” Many of the same executives believe that A.G.I. will be achieved by 2030 at the latest — if not far sooner. Despite being aware of the dangers their technology carries with it, no major A.I. executive has shown any interest in slowing down their work. Labs continue to conduct new research, build new models and market them to the public, even as their leaders say again and again that extinction is a significant possibility.

This leaves ordinary people in an uncomfortable position. Those in the know think that extinction is a reasonably likely outcome from A.I. development. They think that the necessary technological threshold for serious danger is approaching quickly. And they aren’t stopping.

The inconvenient truth is that these factors, in combination, make a strong argument in some minds for an act of self-defense. Interpreted literally and straightforwardly, the statements and actions of A.I. executives imply that everyone on Earth — you, me, our friends and families — are in imminent mortal danger. Why shouldn’t we take them seriously?

American legal doctrine typically treats self-defense as “the use of force to protect oneself from an attempted injury by another,” and it is justified by a reasonable belief that force is necessary to defend oneself against use of “unlawful physical force.” The danger must be imminent; the person acting in self-defense must believe they need to act when they do to avoid danger. A looming, more abstract threat like A.I.-induced extinction is not exactly accounted for in such doctrines, of course. But their spirit, if not their letter, seems consistent with protecting oneself from the physical harm of extinction. That is painful to say, but it is not unreasonable that some people — considering A.I. executives’ own words — have seemingly come to that conclusion.

Even the balmier scenarios don’t look good for humanity, if Mr. Altman is to be believed. In a 2017 blog post, he argued that humans “will be the first species ever to design our own descendants.” Humans, he believes, will need to “merge” with machines to survive the creation of digital superintelligence. But merging with machines is just another form of extinction. Mr. Altman is thus arguing that to avoid extinction, we will need to go extinct. And that is, as he sees A.I. development, the best-case scenario.

No one should want more violence — and attacks on A.I.’s proponents is not an effective way to create political or social change. My own view is that violence is never justified. That is a hard sell for many, though, when they are being told again and again that they and their loved ones are in imminent mortal danger. In such circumstances, even many moral philosophers would argue that violence can be justified.

A.I. executives and the companies they lead could commit to greater controls on their technology, too. That would defang most arguments for violence to prevent their work. They could also say that their previous words have been hype and bluster, not an actual prediction — though they do seem to mostly believe them. Even re-estimating timelines would negate any near-term argument for defensive violence; if the threat of A.G.I. were decades away, no imminent threat would exist.

But these avenues appear unlikely. A more plausible and more effective option is government intervention. The U.S. government should impose immediate regulations on A.I. companies to block them from building what Mr. Musk has called “basically a digital god.” Legislation recently proposed by congressional progressives that aims to temporarily block new data center construction would be a good first step, but is far from sufficient. It should be, once again, illegal to build generative A.I. systems more advanced than those we currently have. To ensure that A.I. companies in other countries — notably, China — do not build A.G.I. either, we must also propose multinational protocols and treaties to establish the kind of international moratorium on A.I. that some executives like Mr. Musk supported as recently as 2023.

The sad reality is, A.I. executives have backed themselves into a corner. No one is responsible for the violent acts of others, of course, but the executives’ words imply that humanity is in grave, imminent danger of total annihilation. Many will hear that as a call to action to defend their lives and their world. While anti-A.I. movements like PauseAI and Stop A.I. are rightly nonviolent, lone wolves will likely remain. The only way forward is to introduce a regulatory ban on building these digital gods. And that ban must come soon.

By the way, I have now completed a full draft of my book, tentatively titled, Will Humanity Survive? How the Race to Build Superintelligent AI Threatens Everyone on Earth.

I am very excited about it, and expect it to come out this September or October. It will be published through this newsletter, and will be available on Amazon as both a physical book and an audiobook.

While strongly encouraging people to pay for the book (writing is my only source of income!), I’ll also posed an open-access copy online as a PDF. I hope it offers a devastating and original critique of the AGI race.

Thank you so much for supporting my work — and my apologies for posting two articles in one day! As you might imagine, I’ve been really itching to get back to this newsletter! The next article will be out around Tuesday. Much love to everyone.

Thanks so much for reading and I’ll see you on the other side!

Altman is literally saying that the only way to avoid extinction is by going extinct. Extinction by merging with machines is the best-case scenario on his view. Altman is a pro-extinctionist. … the options he’s presenting are extinction or (a different kind of) extinction.

… That’s one hell of a thing to admit publicly! So, we’ve got a power-hungry sociopath openly admitting that the thought of controlling AGI “makes people do crazy things.” What could go wrong? We should all be very afraid.

As many of you know, a 20-year-old threw a Molotov cocktail at Sam Altman’s house. Two days later, someone else fired a bullet at Altman’s place.

Altman then published a short essay on his personal website, blaming in part “an incendiary article about me a few days ago.” This is likely a reference to Ronan Farrow and Andrew Marantz’s recent New Yorker piece, which depicts Altman as a manipulative, power-hungry scoundrel who numerous former colleagues describe as a “sociopath.”

Altman’s essay, though, is dripping with bullshit. Someone asked me to go through it, line by line, and comment. That’s what I’ll do here.

His essay begins:

Here is a photo of my family. I love them more than anything.

First, Altman is currently being sued by his sister, Annie Altman, for years of sustained abuse. As the BBC writes:

Ms Altman claims her brother “groomed and manipulated” her and performed sex acts on her over several years, including “rape, sexual assault, molestation, sodomy, and battery,” according to a court filing.

I have gotten to know Annie, and I believe her. The veracity of her allegations are buttressed by the fact that so many former colleagues of Altman’s say he’s deceptive, manipulative, mendacious, and sociopathic. It only takes two points to make a line, but in this case, there are 100 points all falling along the same line. And that line points to the conclusion that Altman is an unethical person. See my newsletter article, “Is Sam Altman a Sociopath?,” for more.

Altman claims to love his family, and I’m sure he does love his husband and child. But he’s been a monster to his sister, and (to quote Remmelt Ellen) “neglectful of his father, and his father’s wishes to give money from his will to Annie.”

Second, is it just me, or does the photo he shared look AI-generated or AI-modified? What’s going on with the flowers over the baby’s head?

Altman continues:

Images have power, I hope. Normally we try to be pretty private, but in this case I am sharing a photo in the hopes that it might dissuade the next person from throwing a Molotov cocktail at our house, no matter what they think about me.

The first person did it last night, at 3:45 am in the morning. Thankfully it bounced off the house and no one got hurt.

The guy just can’t stop lying. All the reports are that the perpetrator threw the Molotov cocktail at the gate of his house. It bounced off the gate, not the house. See for yourself below:

Farrow and Maratnz note that Altman consistently lies about trivial things. For example, they write: “One recalled Altman bragging widely that he was a champion Ping-Pong player — ‘like, Missouri high-school Ping-Pong champ’ — and then proving to be one of the worst players in the office. (Altman says that he was probably joking.)”

By exaggerating how close the Molotov cocktail got to his house, Altman is trying to elicit more sympathy for himself. In doing this, he’s proving Farrow and Maratnz’s point.

Words have power too. There was an incendiary article about me a few days ago. Someone said to me yesterday they thought it was coming at a time of great anxiety about AI and that it made things more dangerous for me. I brushed it aside.

Now I am awake in the middle of the night and pissed, and thinking that I have underestimated the power of words and narratives. This seems like as good of a time as any to address a few things.

First, what I believe.

*Working towards prosperity for everyone, empowering all people, and advancing science and technology are moral obligations for me.

OpenAI has done nothing to advance science. ChatGPT has induced episodes of psychosis in people, encouraged people to commit suicide (some of whom have followed through), and flooded the Internet with slop, disinformation, deepfakes, and other forms of informational pollution.

Altman likes to talk about universal basic income (UBI), but does anyone for a moment think he’d actually push for this if AI were to replace 50% of all jobs — or more? As one person put it on X (in response to Musk talking about UBI, or what he calls “universal high income”):

*AI will be the most powerful tool for expanding human capability and potential that anyone has ever seen. Demand for this tool will be essentially uncapped, and people will do incredible things with it. The world deserves huge amounts of AI and we must figure out how to make it happen.

“We must figure out how to make it happen” is vague, vacuous nonsense. Altman is constantly saying such things, but his words are never followed up by actions.

It’s like when he said in 2016: “We’re planning a way to allow wide swaths of the world to elect representatives to a new governance board. … Because if I weren’t in on this I’d be, like, why do these fuckers get to decide what happens to me?” Yet he proceeded to fill OpenAI’s board with people like Larry Summers (who is no longer at OpenAI due to his connections with Jeffrey Epstein). As CNN reports, the “new committee would be led by CEO Sam Altman as well as Bret Taylor, the company’s board chair, and board member Nicole Seligman.” Remmelt Ellen adds: “Important note too that he used Larry Summers and Bret Taylor to ensure that no investigation would be publicly released. No room for accountability whatsoever.”

Pfff. All talk, no action.

*It will not all go well. The fear and anxiety about AI is justified; we are in the process of witnessing the largest change to society in a long time, and perhaps ever. We have to get safety right, which is not just about aligning a model — we urgently need a society-wide response to be resilient to new threats. This includes things like new policy to help navigate through a difficult economic transition in order to get to a much better future.

Who the hell elected Altman to bring about “the largest change to society in a long time, and perhaps ever”? On what authority does he have the right to unilaterally dictate what the future should look like — without our permission or consent? He says that “fear and anxiety about AI is justified” — is it any wonder, then, that lone wolves are taking matters into their own hands?

Again, he follows this up with vague, vacuous proclamations about what “we” need to do. Who is this “we”? You and I have no power. We have no voice at the table. It’s up to the people who actually have power, like Altman, to make this happen. Yet nothing changes.

Furthermore, he says “we have to get safety right,” but Altman gutted OpenAI’s “AI safety” research. In fact, one of the reasons the board fired him was because of his lax attitude about safety. As Helen Toner, one of the board members who voted to fire Altman, says:

“On multiple occasions, he gave us inaccurate information about the small number of formal safety processes that the company did have in place, meaning that it was basically impossible for the board to know how well those safety processes were working or what might need to change.”

*AI has to be democratized; power cannot be too concentrated. Control of the future belongs to all people and their institutions. AI needs to empower people individually, and we need to make decisions about our future and the new rules collectively. I do not think it is right that a few AI labs would make the most consequential decisions about the shape of our future.

AI must be democratized, he says, while pursuing an overtly anti-democratic approach to building AGI. He’s not asking you or me — or anyone else — about our views on when, how, and whether AGI should be built. This is nothing less than tyranny.

Furthermore, his claim that “AI needs to empower people individually” is absolute bullshit. In his 2017 article “The Merge,” Altman says there are two options before us: the first is complete human extinction due to the emergence of AGI. The second is for us to “survive” by “merging” with machines — which is just another kind of extinction, as biological humanity would cease to exist if some were to become uploaded minds (digital posthumans) or human-AI hybrids (cyborgs).

Altman is literally saying that the only way to avoid extinction is by going extinct. Extinction by merging with machines is the best-case scenario on his view. Altman is a pro-extinctionist.

Again, who the hell is he to dictate that those are our only two options? Utter tyranny.

*Adaptability is critical. We are all learning about something new very quickly; some of our beliefs will be right and some will be wrong, and sometimes we will need to change our mind quickly as the technology develops and society evolves. No one understands the impacts of superintelligence yet, but they will be immense.

Superintelligence is coming, and there’s nothing you can do to stop it. It might annihilate us, but don’t worry: we can merge with machines to “survive.” Is it any wonder that young people are freaking out?

Second, some personal reflections.

As I reflect on my own work in the first decade of OpenAI, I can point to a lot of things I’m proud of and a bunch of mistakes.

Fake humility. That’s all this is.

I was thinking about our upcoming trial with Elon and remembering how much I held the line on not being willing to agree to the unilateral control he wanted over OpenAI. I’m proud of that, and the narrow path we navigated then to allow the continued existence of OpenAI, and all the achievements that followed.

It’s absurd for Altman to talk about having resisted the “unilateral control [Musk] wanted over OpenAI.” That is exactly the kind of control that Altman now wields. He once said that the board has the power to fire him, yet when it fired him, he clawed his way back to power and had several people on the board (like Toner) ejected. As Altman’s old pal Paul Graham puts it: “Sam is extremely good at becoming powerful.”

I am not proud of being conflict-averse [LOL], which has caused great pain for me and OpenAI. I am not proud of handling myself badly in a conflict with our previous board that led to a huge mess for the company. I have made many other mistakes throughout the insane trajectory of OpenAI; I am a flawed person in the center of an exceptionally complex situation, trying to get a little better each year, always working for the mission. We knew going into this how huge the stakes of AI were, and that the personal disagreements between well-meaning people I cared about would be amplified greatly. But it’s another thing to live through these bitter conflicts and often to have to arbitrate them, and the costs have been serious. I am sorry to people I’ve hurt and wish I had learned more faster.

More fake humility. As Aaron Swartz said shortly before his suicide: “You need to understand that Sam can never be trusted. … He is a sociopath. He would do anything.” Once you understand this, passages like those above appear to be nothing but pure manipulation. Altman has no one to blame for this but himself: he’s convinced too many people at this point, including me, that he’s a sociopath.

Mostly though, I am extremely proud that we are delivering on our mission, which seemed incredibly unlikely when we started.

OpenAI’s mission has been completely obliterated. The company started out as a nonprofit, and is now the world’s most valuable for-profit company. As Fortune writes, “OpenAI changes it mission statement 6 times in 9 years. It finally removed the word ‘safely’ as a core value when it restructured into a nonprofit.”

The company’s original statement read: “OpenAI’s goal is to advance digital intelligence in the way that is most likely to benefit humanity as a whole, unconstrained by a need to generate financial return.”

Then it became a capped-nonprofit company, and then a for-profit company whose mission now is constrained by the need to generate financial returns.

It’s 2022 and 2023 statement said: “OpenAI’s mission is to build general-purpose artificial intelligence (AI) that safely benefits humanity.”

It then removed the word “safely.” Its current statement says: “OpenAI’s mission is to ensure that artificial general intelligence benefits all of humanity.” What a joke!

Against all odds, we figured out how to build very powerful AI, figured out how to amass enough capital to build the infrastructure to deliver it, figured out how to build a product company and business, figured out how to deliver reasonably safe and robust services at a massive scale, and much more.

There is nothing “safe” about OpenAI’s services. See: psychosis, suicide, slop, disinformation, deepfakes, and so on.

A lot of companies say they are going to change the world; we actually did.

Yeah, for the worse. Congratulations, Sam!

Third, some thoughts about the industry.

My personal takeaway from the last several years, and take on why there has been so much Shakespearean drama between the companies in our field, comes down to this: “Once you see AGI you can’t unsee it.” It has a real “ring of power” dynamic to it, and makes people do crazy things. I don’t mean that AGI is the ring itself, but instead the totalizing philosophy of “being the one to control AGI.”

That’s one hell of a thing to admit publicly! So, we’ve got a power-hungry sociopath admitting that the thought of controlling AGI “makes people do crazy things.” What could go wrong? We should all be very afraid.

The only solution I can come up with is to orient towards sharing the technology with people broadly, and for no one to have the ring. The two obvious ways to do this are individual empowerment and making sure democratic system stays in control.

Again, there’s absolutely nothing democratic about the way OpenAI and the other companies have so far built and deployed their AI systems. The fact that OpenAI hasn’t pushed for an agreement between the AI companies — and companies in China — to slow down the AGI race implies that Altman doesn’t actually believe that “no one [should] have the ring.” If he believed that, he’d be cooperating and coordinating with the other companies, which he’s not. He couldn’t even join hands with Dario Amodei on stage:

Dude, if the survival of humanity is really at stake, surely you can put your differences aside for a moment to, you know, save humanity?

It is important that the democratic process remains more powerful than companies. Laws and norms are going to change, but we have to work within the democratic process, even though it will be messy and slower than we’d like. We want to be a voice and a stakeholder, but not to have all the power.

A lot of the criticism of our industry comes from sincere concern about the incredibly high stakes of this technology. This is quite valid, and we welcome good-faith criticism and debate. I empathize with anti-technology sentiments and clearly technology isn’t always good for everyone. But overall, I believe technological progress can make the future unbelievably good, for your family and mine.

“Unbelievably good”! He literally says the only way we’re going to survive AGI is by somehow merging with machines — something he’s already taken steps to do by signing up to have his brain uploaded to a computer if he dies. Again, the options he’s presenting are extinction or (a different kind of) extinction.

When I attended a Stop AI protest in front of OpenAI’s headquarters last summer, someone led the group in a chant of “Fuck Sam Altman.” That seems appropriate. It channels a virtue called “righteous indignation,” which you may have detected in this article. (I’m most drawn to virtue ethics, btw.)

While we have that debate, we should de-escalate the rhetoric and tactics and try to have fewer explosions in fewer homes, figuratively and literally.

De-escalating the rhetoric must start with the AI CEOs. They repeatedly tell us that everyone will lose their jobs and that AGI might literally kill everyone on Earth. Here are just a few examples:

• Demis Hassabis says the probability of annihilation is “definitely non-zero and it’s probably non-negligible. So that in itself is pretty sobering.”

• Shane Legg puts the probability of doom between 5% and 50%.

• Elon Musk says it’s between 10% and 30%.

• Dario Amodei estimates a 10% to 25% chance of annihilation.

• Altman says that “AI will … most likely sort of lead to the end of the world, but in the meantime there will be great companies created with serious machine learning,” and that “probably AI will kill us all, but until then we’re going to turn out a lot of great students.” On another occasion, he said that “the bad case—and I think this is important to say—is lights out for all of us.” But again, the good case on his view is still a form of human extinction.

Why are these people surprised that folks are lashing out? In my next article, which I’ll publish in a few days, I’ll explain how there’s actually a very strong legal case to be made for violence against the AI companies and their CEOs based on things the CEOs themselves have said. To be clear, I am not saying that violence is morally justified. I am very firmly in the anti-violence camp. But legally, there’s a case here because the AI CEOs are telling us that we are all in imminent mortal danger.

I do not wish Altman or his family any harm. However, there are good reasons for people to be outraged by his words and behaviors, and to be furious about the ongoing tyranny of the AI companies.

What do you think? As always:

Thanks for reading and I’ll see you on the other side!

Thanks to Remmelt Ellen and Dr. Alexander Thomas for criticial feedback on an earlier draft. This does not necessarily mean they agree with this essay.

Every member of the TESCREAL movement accepts a posthuman eschatology, according to which we should introduce one or more new posthuman species, hopefully in the near future. If someone doesn’t accept this posthuman eschatology, then I do not count them as being in the TESCREAL movement.1 This is essentially what the term “TESCREAL” refers to: that group of people who advocate for a posthuman eschatology.

A “posthuman,” by the way, is a being so radically different from us that we would uncontroversially classify it as a novel species. It could come about as a result of radical “enhancements” involving brain-computer interfaces (BCIs), genetic engineering, mind-uploading, radical life extension, etc. Or it could take the form of autonomous AIs that we create as distinct and separate entities (think ChatGPT-30, or whatever).

There are two general views one could have about what the relationship between humans and posthumans should be:

• Coexistence view: humans and posthumans should coexist together

• Replacement view: posthumans should replace humans.

All pro-extinctionists accept the replacement view. (Copious examples from my forthcoming book can be found here.) They want posthumanity to usurp humanity. But I would argue that the coexistence view would almost certainly result in the extinction of our species, too. Hence, anyone advocating for the creation of posthumanity, even if they accept the coexistence view, is in effect pushing for a future in which our species dies out.

***

The coexistence view is a minority position within the TESCREAL movement. But there are some advocates. Jeffrey Ladish, for example, writes:

I have a special love for humans (and other animals) and a lot of stake in the preferences of current and future humans (and other minds too). I also am pretty down for creating many other types of minds, but I have a strong preference for the existence and continuity of people alive today and their descendants.

Yudkowsky, who has repeatedly expressed pro-extinctionist sentiments, also sometimes suggests he accepts the coexistence view instead. In a social media exchange with the “worthy successor” pro-extinctionist Daniel Faggella, he wrote: “You’re not getting the concept here. Protecting innocent life is part of the flame itself. If some entity doesn’t get that, our torch hasn’t been passed on,” adding that “a key quality required of the successor is its own respect for other consciousnesses.”

In “Transhumanist FAQ,” Nick Bostrom writes: “The transhumanist goal is not to replace existing humans with a new breed of super-beings, but rather to give human beings (those existing today and those who will be born in the future) the option of developing into posthuman persons.” In another paper, he argues that “it is important that the opportunity to become posthuman is made available to as many humans as possible, rather than having the existing population merely supplemented (or worse, replaced) by a new set of posthuman people.” At least on paper, Bostrom’s transhumanism is compatible with human-posthuman coexistence.

In a New York Times interview, Daniel Kokotajlo also gestured at the coexistence view in saying:

I’m a huge fan of expanding into space. I think that would be a great idea. And in general, also solving all the world’s problems, like poverty and disease and torture and wars. I think if we get through the initial phase with superintelligence, then obviously, the first thing to do is to solve all those problems and make some sort of utopia, and then to bring that utopia to the stars would be the thing to do.

The thing is that it would be the AIs doing it, not us. In terms of actually doing the designing and the planning and the strategizing and so forth, we would only be messing things up if we tried to do it ourselves.

So you could say it’s still humanity in some sense doing all those things, but it’s important to note that it’s more like the AIs are doing it, and they’re doing it because the humans told them to.

Hence, humanity stays anchored to Earth, while ASI (artificial superintelligence) explores the heavens.

***

Why do I claim that the coexistence view is pro-extinctionist in practice? Why are the futures described by those above completely implausible?

First, as alluded to above, posthumanity could take many forms. Many TESCREALists imagine building an ASI that enables humanity to be transformed into posthumanity via “radical enhancement” technologies. As Demis Hassabis says, “solve intelligence and you can solve everything else” (paraphrasing him). By building an AI God, we could delegate it the task of turning us into posthuman super-beings, usually imagined to be digital in nature (hence my satirical term “digital space brains”).

What this approach amounts to is the following: to become posthumanity, we must first create posthumanity. That’s because ASI itself would constitute a kind of posthuman: a superintelligent being so different from us that we’d classify it as a distinct species. This posthuman would then figure out how to upload our minds to computers, thus enabling us to become superintelligent digital posthumans just like the ASI. The result would be at least two types of posthumans: the ASI and the super-beings that we’ve been transformed into.

***

Let’s now think seriously about how this would go down in the messy real world. Imagine that OpenAI reaches the ASI finish line first, and that the ASI they build is controllable by Sam Altman, its CEO. What do you think Altman would do with the most powerful technology in human history under his control?

He will, of course, task the ASI with designing radical enhancement technologies to make himself and his billionaire buddies posthuman. (Altman has already signed up to have his brain digitized, which means he wants to become posthuman.) There is absolutely no way that Altman would ever share such enhancements with the rest of humanity. The reason is obvious: picture everyone around the globe “radically enhancing” themselves. This makes everyone more or less equal. Every person in China, India, Nigeria, Chile, Norway, and Russia suddenly has PhD-level knowledge in every domain, a capacity to process large amounts of information at the speed of a computer, and the ability to work 24 hours a day with almost no breaks.

This would level the playing field, thus posing a direct threat to the tech elite. It would undermine their privileged standing in society, atop the hierarchy of wealth, power, control, and dominance. They would never, ever let that happen. Obviously! The haves would have even more, and the 99% would be powerless to overthrow the posthuman plutocracy. If there’s one thing that power wants, it’s to maintain power, and radical enhancements distributed in an egalitarian manner would fatally compromise that power. This goes not just for Altman, but the other AI CEOs as well: Hassabis, Amodei, and of course the fascist ketamine addict, Musk.

Yet, even if enhancement technologies were somehow widely distributed to everyone, I doubt more than a small fraction of the population would willingly choose to become posthuman. I certainly wouldn’t upload my mind to a computer. I like being human (and insofar as I don’t like being human, I would be utterly terrified to become posthuman, for reasons explained below.)

That means that, with ASI, three species will initially exist on Earth: posthumans in the form of ASIs, posthumans in the form of transformed humans, and unenhanced humans. Pro-extinctionists want to eliminate the third, while coexistence view advocates don’t.

***

But think about what would actually happen. Our species would become economically obsolete, and hence disposable. We would take up space and suck up resources these posthumans could use more “efficiently” for their own purposes. Resources on Earth are finite, so it would bother them that we continue using up these resources while contributing nothing to the economy (to say nothing of the political system, scientific enterprise, and fields like engineering). Our mere existence would be an impediment to these beings.

Consider that over just the past 25 years, the chimp population in Western Africa has declined by a staggering 80%. The only reason chimpanzees haven’t gone extinct yet is because we care just enough to leave them little patches of Earth to live on. If our civilization expands, though, do you think chimps will survive? Probably not — they will go extinct just like all the other species we’ve killed off during the sixth major mass extinction of the past 3.8 billion years.

That will be the fate of our species, as posthumans will have no good reasons to keep us around and every reason to get those pesky Homo sapiens out of the way for good. It is utterly implausible to imagine our species persisting beyond a few decades into the posthuman era. As Ray Kurzweil writes, if you choose not to become posthuman, then “you won’t be around for very long to influence the debate.” That’s because our species would die out.

***

You might say that this is all wrong: the AI CEOs are explicit that the whole point of ASI is to “benefit all of humanity.” Utter nonsense, I reply. Just look at the trail of destruction they’re leaving behind them right now while they race to trigger the Singularity by building an AI God. A short list of such harms might include:

The AI CEOs clearly don’t give a damn about humanity right now. Why would they suddenly start caring about humanity later on, once they control the most powerful technology in human history? ASI is not some magical threshold beyond which power-hungry, messianic sociopaths suddenly stop being power-hungry, messianic sociopaths. Obviously, they aren’t actually building ASI to benefit everyone. If anything, they will use it to transform themselves into posthumans and establish a cosmic dictatorship — in fact, the reason Altman, Musk, Greg Brockman, and Ilya Sutskever cofounded OpenAI is because they worried that Demis Hassabis would create an “AGI dictatorship.”

The 99% would not survive the aftermath of this — even if ASI were “controllable” or “value-aligned.” There is not a single avaricious billionaire on Earth who wouldn’t leap at the opportunity to control everything while permanently entrenching their positions of power, control, and dominance. Obviously.

***

But the situation is much worse than this. Imagine a radically different scenario in which posthumanity is for some reason gentle, kind, compassionate, and ethical. Perhaps it embraces certain moral principles that many of us would recognize as legitimate, such as the principle that one should reduce suffering wherever possible.

Now imagine posthumanity looking around at our species and realizing we’re chronically susceptible to things like depression, anxiety, sadness, sorrow, misery, frustration, loneliness, and heartache. It then decides that the morally best thing would be to euthanize us, the same way we put down abandoned animals in a shelter. An infertility drug is secretly distributed in public water systems, or posthumanity quietly disperses a general anesthetic in the air, after which it puts humanity out of its misery by killing everyone in a kind of humane extinction.

In one of his pro-extinctionist rants, Yudkowsky asked: “Are we, like, kind of too sad in some ways” to stick around once posthumanity arrives? Maybe posthumans reason the same way. Or, consider William MacAskill’s argument that our systematic obliteration of the biosphere might be net positive. That’s because many wild animals have lives that aren’t worth living, he says. Hence, we’re doing them a favor, in disguise, by demolishing their habitats and poisoning their ecosystems. Perhaps posthumanity comes to believe that most of our lives aren’t worth living, so it puts us down.

The philosopher Thomas Metzinger outlined a similar scenario in a 2017 essay, which imagined us building an ASI that’s “far superior to us in the domain of moral cognition.” It’s “benevolent” and “fundamentally altruistic,” and fully respects “one of our highest values,” namely, the importance of “maximizing happiness and joy in all sentient beings.” However, it also

knows many things about us which we ourselves do not fully grasp or understand. It sees deep patterns in our behaviour, and it extracts as yet undiscovered abstract features characterizing the functional architecture of our biological minds. For example, it has a deep knowledge of the cognitive biases which evolution has implemented in our cognitive self-model and which hinder us in rational, evidence-based moral cognition. Empirically, it knows that the phenomenal states of all sentient beings which emerged on this planet — if viewed from an objective, impartial perspective — are much more frequently characterized by subjective qualities of suffering and frustrated preferences than these beings would ever be able to discover themselves. Being the best scientist that has ever existed, it also knows the evolutionary mechanisms of self-deception built into the nervous systems of all conscious creatures on Earth. It correctly concludes that human beings are unable to act in their own enlightened, best interest (italics added).

The ASI also “knows that no entity can suffer from its own non-existence,” and thus

concludes that non-existence is in the own best interest of all future self-conscious beings on this planet. Empirically, it knows that naturally evolved biological creatures are unable to realize this fact because of their firmly anchored existence bias. The superintelligence decides to act benevolently.

The same year Metzinger published his essay, an EA named Derek Shiller published an article titled “In Defense of Artificial Replacement.” He argues that,

if it is within our power to provide a significantly better world for future generations at a comparatively small cost to ourselves, we have a strong moral reason to do so. One way of providing a significantly better world may involve replacing our species with something better. It is plausible that in the not-too-distant future, we will be able to create artificially intelligent creatures with whatever physical and psychological traits we choose. Granted this assumption, it is argued that we should engineer our extinction so that our planet’s resources can be devoted to making artificial creatures with better lives.

In this case, Shiller is suggesting that we make the decision to voluntarily die out. But our posthuman descendants might reach the same “moral” conclusion and opt to eliminate us, even if humanity decides it wants to stick around. This alternative possibility is what Metzinger highlights in his essay: precisely because the ASI is fully benevolent and altruistic, it gets rid of us, for the sake of making the world better by removing a population of creatures prone to suffering.

***

If ASI were controllable and to usher in what Yudkowsky calls our “glorious transhumanist future,” there is absolutely no reason to believe that our species would survive. Once again, we’d be using up finite valuable resources and taking up space while being economically useless. Furthermore, an “ethical” posthuman species might simply opt to euthanize us because we’re “too sad” to keep around.

The coexistence view is a complete nonstarter. If we create and/or some portion of humanity becomes a new posthuman species, our days are numbered; terminal extinction will become more or less inevitable. This is why I argue that anyone who accepts a posthuman eschatology, whether they favor replacement or coexistence, is pushing a future in which we will die out. Perhaps in the coming years.

***

It might be tempting to say, “Well, maybe this wouldn’t be such a bad thing, if we were replaced by superior posthumans? Maybe Yudkowsky has a point?” If you agree with this, then you’re a pro-extinctionist. But it’s also worth pointing out that there’s no reason to think that posthuman life would actually be any better. It could, in fact, be far worse in many ways. Here are a few examples:

(1) Imagine undergoing a life-extension intervention that enables you to live at least 10,000 years. You won’t die of old age or disease. The only possible causes of mortality are the same things that kill young people, namely, accidents.

Think about how this would change your quantitative risk assessment of, say, walking through a city. Even if there’s a minuscule chance you might get flattened on the road by a bus, the stakes are so huge — 10,000 future years of life — that the risk would still be enormous.

That’s because “risk” is defined as “the probability of an adverse outcome multiplied by its badness.” This means that a low-probability but very high-stakes outcome could nonetheless yield an extremely high risk. Hence, simply perambulating to the grocery store would become an exceptionally risky affair, which means you should stay home all day and never leave your apartment. That doesn’t sound like a fun life, but it’s the life that one should live if one could survive for a millennium and is thinking rationally.

The same point applies to digital space brains whose physical substrate is computer hardware. If a space brain knows it could potentially survive until the heat death, the stakes become so huge that there’s literally nothing it should do each day for nearly its entire life other than safeguarding its continued existence. Maybe the computer on which it’s running breaks down, and maybe there aren’t enough back-ups of its mind on other servers. Maybe there’s an intergalactic war that blows everything up (see below). Perhaps an asteroid randomly destroys the “planet-sized” computer (quoting Bostrom) that runs the simulation in which it resides.

There are a million possibilities here, and if risk equals probability multiplied by the consequences, then even a minuscule chance of some random event destroying the space brain should occupy it’s thoughts every second of every day.

(2) Now consider the fact that becoming a digital space brain would open up the unspeakably horrifying possibility of being tortured for literally trillions of years. Enemies of the cosmic civilization — political dissidents, criminals, or outcasts — could be locked away in a digital dungeon to suffer excruciating agony every second of the day until the heat death. That’s not something that could happen to us (our species), although radical life-extension technologies of the sort Peter Thiel wants to developed could enable something similar here on Earth until it becomes uninhabitable in 1 billion years.

This is absolutely terrifying, but it could become common in the posthuman era.

(3) There’s also the aforementioned possibility of dictators — perhaps Demis Hassabis, Sam Altman, Donald Trump, or someone else — establishing a totalitarian regime that no one can overthrow. Then, as this regime spreads into space, it could fracture into nations that engage in constant catastrophic wars, as the international relations theorist Daniel Deudney argues in his excellent book Dark Skies. The basic idea is this (paraphrasing a previous newsletter article):

If future beings have the technology to spread beyond our solar system, they will probably also have the technology to inflict catastrophic harm on each others’ galactic neighborhoods. They might employ cosmic weapons we can’t even imagine. Furthermore, outer space will be politically anarchic, meaning that there’s no central Leviathan (or state) to keep the peace. Indeed, since a Leviathan requires timely coordination to be effective, and since outer space is so vast, establishing a Leviathan will be impossible. (In other words, there will be no single civilization, but a vast array of these civilizations, populated by beings with wildly different cognitive capabilities, emotional repertoires, technological capacities, scientific theories, political organizations, and even religious ideologies — etc. etc.)

This leaves the threat of mutually assured destruction, or MAD, as the only mechanism for securing peace. But given the unfathomably large number of civilizations that would exist (because the universe is huge), it would be virtually impossible to keep tabs on all potential attackers and enemies. Civilizations would find themselves in a radically multi-polar Hobbesian trap, whereby even peaceful civilizations would have an incentive to preemptively strike others. Everyone would live in constant fear of annihilation, and the inherent security dilemmas of this predicament would trigger spirals of militarization that would only further destabilize relations in the anarchic realm of the cosmopolitical arena. Meanwhile, those captured from enemy civilizations could be brutalized forever in simulated torture chambers.

This is the stuff of nightmares, but on a cosmic scale, lasting until the heat death.

So, humanity has its downsides, but so will posthumanity. Human beings are constrained by our biological limitations, but posthumans will have their own limitations to contend with. Promises of a “techno-utopia” are nothing more than propaganda to “justify” a race to build ASI that’s leaving a trail of destruction in its wake.

Perhaps humanity, in its current form, is the closest we’ll ever get to paradise, and hence losing humanity would be a catastrophic tragedy.

***

TESCREALists, including those trying to build an AI God, are pushing for a future in which our species will go extinct, perhaps in the coming decades. Because I actually care about avoiding the terminal extinction of our species, I object to anyone who promotes a posthuman eschatology. I think it’s a recipe for disaster. And I don’t think that embracing the coexistence view would save us.

Take a step back for a moment and consider what TESCREALists are offering us. Accelerationists like Beff Jezos offer us one option: the extinction of our species by replacement with posthuman ASI.

Doomers are a bit more generous (!) in offering us two options: the extinction of our species or, alternatively, the extinction of our species. In the first case, we die out because misaligned ASI kills everyone. By virtue of being misaligned, this ASI won’t constitute a “worthy successor.” In the second case, an “aligned” ASI enables some people — the elites — to become posthuman. The remaining humans are then pushed out of existence by those posthumans.

Using my terminology, the first would involve final extinction, while the second would involve terminal without final extinction. In both cases, our species dies out.

This is why I won’t lock arms with TESCREAL doomers who want the ASI race shut down until the alignment problem is solved. They are not on my side; they are not on Team Human. Their allegiance instead is to Team Posthuman — to realizing a posthuman eschatology. That eschatology would almost certainly result in Team Human going extinct.2 This is why Yudkowsky and Beff Jezos occupy the exact same spot in my mind: they are enemies of humanity, our species, and should be dealt with as such.

You might say: If Anyone Becomes Posthuman, Everyone Dies Out.

But what do you think? What have I missed? How might I be wrong? As always:

Thanks for reading and I’ll see you on the other side!

1. Why are you (they say to me) so harshly critical of Eliezer Yudkowsky? Why not join him in warning about the dangers of AGI and calling for an international treaty to stop the AGI race? It’s counterproductive to target someone with such a big public profile! You should be working with rather than against him!

2. Wait, you say that the AGI race should be stopped immediately and that it’s incredibly dangerous, yet you also say that we’re nowhere close to building AGI. How does that make sense?

Taking these in order, a reminder that Yudkowsky has made remarks like these:

If sacrificing all of humanity were the only way, and a reliable way, to get … god-like things out there — superintelligences who still care about each other, who are still aware of the world and having fun — I would ultimately make that trade-off.

Yudkowsky emphasizes that this “isn’t the trade-off we are face with” right now. But if it were, he’d willingly sacrifice our species to see artificial super-beings flitting about the universe “having fun.”

Think for a moment about how utterly outrageous this is. He’ saying that he’d be willing to sacrifice all Chinese people, Indian people, South Africans, Norwegians, everyone in Nigeria, Chile, Mongolia, and Japan, the entire populations of Toronto, Bangkok, Moscow, and London, if doing so were the “only” and a “reliable” way of creating fun-having digital space brains.

That is absolutely abhorrent. It’s atrocious. Imagine saying that you’d be willing to sacrifice my life to realize your dream: a big mansion in Hollywood. Everyone would agree that saying this would be completely unacceptable. Now imagine a TESCREAList, like Yudkowsky, saying they’d sacrifice my life, the lives of my family, and the lives of everyone else on Earth to realize their dream of a cosmic utopia full of digital space brains. I have no words to express how horrendous this is.

My friend Mark Gubrud, who often disagrees with me, wrote this:

(Note that Mark is the person who first coined the term “artificial general intelligence” back in 1997. Shane Legg then seems to have independently coined it in the mid-2000s.)

My response didn’t beat around the bush. “Not sure how else to put this,” I wrote,

there are no doubt many things that 1940s Nazis and I would have agreed about re: the environment. Does that mean I should have locked arms with them in fighting to preserve the natural world? No, absolutely not. Never. You might say this isn’t analogous with the case of Yudkowsky — and I would agree, because what Yudkowsky has said out-loud, in public, is far worse than anything fascists have said. Those fascists were eugenicists who wanted to eliminate specific social groups for the greater good. Yudkowsky is a eugenicist who’s explicitly said that he’d sacrifice literally everyone on Earth to realize his incredibly dumb version of utopia. … What. The. Fuck. What he’s saying here is absolutely fucking insane. It’s among the most extreme, horrific things I’ve ever heard anyone ever say — ever. It’s worse than anything that’s come out of the mouth of Donald Trump, Steve Bannon, or Stephen Miller. Does this make sense now? I want absolutely nothing to do with people who hold fucking insane, incredibly dangers views like those he’s repeatedly expressed.

(Sorry for the vulgarities, but I think they’re warranted in the face of omnicidal threats.)

Here’s a similar example: as you’re all aware, Donald Trump recently said that “a whole civilisation will die tonight, never to be brought back again. I don’t want that to happen, but it probably will.” He then backed off an has apparently reached an agreement according to which Iran gets pretty much everything they wanted (lolz):

The fact that Trump didn’t commit genocidal war crimes against Iran doesn’t for one moment excuse his words. Yet, what Trump said is not as bad as what Yudkowsky has said, and we must acknowledge that. Trump is basically declaring that he’d sacrifice Iran for some obscene notion of the greater good. Yudkowsky is saying he’d sacrifice humanity for some bizarre notion of the greater cosmic good. What is omnicide other than all possible genocides put together?

Furthermore, as I wrote here, Yudkowsky isn’t actually anti-AGI. He’d build AGI literally next week if he thought it was “value-aligned,” meaning aligned with the utopian values of the TESCREAL worldview (cosmic paradise, digital space brains, and all that). He’s not on Team Human, he’s on Team Posthuman.

What I want is to join forces with folks on Team Human who actually do think we should never build AGI. If the alignment problem were solved next week, I’d still oppose AGI. (I provide a detailed argument in my book for why AGI would almost certainly have catastrophic consequences if built, no matter what, which makes me very anti-AGI in an unconditional way.)

***

As for the second question, it might be worth noting that there are at least four main reasons one might think AGI won’t arrive in the near future. These are:

1. The systems that power current AI, large language models (LLMs), aren’t going to get us to AGI by themselves. No matter how much AI companies scale them up by increasing compute (computational resources), training data, and their parameters, they just don’t have the right architecture to become AGI. LLMs by themselves are a dead end, though there might be other systems, architectures, and approaches that could eventually get us there.

2. AGI isn’t a technology that we could ever build. It might simply be too difficult for our species. This is made plausible if one believes there may be other technologies that are in theory possible but in practice out of our reach, such as spacecraft that travel at 99.99% the speed of light. Perhaps some kind of super-clever alien species could figure this out, but we probably never will.

3. AGI, and especially ASI (artificial superintelligence), isn’t possible to build in theory. It’s like a perpetual motion machine, or designing a spacecraft that can exceed the speed of light. There’s just no way to build an artificial system that surpasses human capabilities in every cognitive domain of interest, perhaps because we represent the highest level of “intelligence” attainable.

4. AGI and ASI are not even coherent ideas to begin with. What does it mean to build an “everything machine” that can “exceed” humans in every domain of interest? Heck, AI companies and researchers can’t even agree upon a definition of “AGI” — OpenAI itself proposes multiple inconsistent definitions on its own website! What, then, are we even talking about?

I very much agree with (1), and am sympathetic with (4). However, I also find (2) appealing for the following reason: it could be that a certain degree of societal, political, economic, etc. stability is necessary to build AGI. However, the stepping-stone systems that we’d need to build in order to reach AGI may wreak so much havoc that the fabric of society unravels, thus making AGI unreachable. In other words, there may be a negative feedback loop here such that the closer we get, the further away we end up: building AGI requires societal stability, but the more AI we have, the less stable things become.

On the ineptitude of AI … Shot and chaser:

A chapter in my book goes into immense detail about all the harms caused by current AI systems, which the AI companies have built in hopes of scaling them up to become AGI. When one surveys all these harms catalogued in one place, over the course of 30 pages, it’s completely shocking, and forces one to admit that it’s not implausible the AGI race itself could potentially cause societies to collapse (or at least play a significant causal role in helping along the process).

This is why I call for an immediate, permanent halt to the AGI race — on scaling up generative AI systems. They have very few benefits, and threaten to undermine key civic institutions like the rule of law and free press, as well as facilitating things like domestic mass surveillance.

That’s how someone can simultaneously say that AGI isn’t around the corner and that the race to build AGI must stop immediately. To this, I’d also add that if we were to someday build AGI, I have no reason to expect it to be controllable by default. If humanity were suddenly joined by an entity that could outmaneuver us in every important respect, solve complex problems faster than any human, set and modify its own goals without human intervention, and act as an autonomous agent in the world, etc., then we’re in very big trouble.

Fortunately, we’re nowhere close to building such an entity. Unfortunately, we don’t need AGI for AI to destroy the world.

***

Speaking of my book, I now have 85,000 words written! I’m almost done, which means I’ll get back to posting two articles per week in probably less than a month. The book will end up around 100,000 words long, and my hope is that it offers a devastating and original critique of the AGI race (I don’t know of anyone else who makes the points I do).

But I’m in a bit of a pickle, as I’m struggling to find a new publisher who can get it out later this year — rather than 13 months from now, which is what my current publisher is offering. Who the hell knows what the world will look like in 13 months (lol); the topics I discuss are relevant right now!

I have a few promising leads, and I’m not fundamentally opposed to self-publishing it in the end if that’s necessary to get it out sometime in 2026. (The book will still be massively peer reviewed either way, as I send most of what I write out to many dozens of people for feedback and criticisms.) Anyways, just giving you an update. The current table of contents is:

Clown Car Utopia: Why We Must Stop AI to Save Humanity

• Chapter 1: Digital Space Brains

• Chapter 2: An Alphabet Soup of Ideologies

• Chapter 3: The Road to Utopia

• Chapter 4: Death to Humanity

• Chapter 5: A Trail of Destruction

• Chapter 6: Our Precious Planet

• Chapter 7: Butlerian Jihadists

I honestly could not do this without your support! As always:

Thanks for reading and I’ll see you on the other side!

That can’t be said about a review of the book published in The Sydney Morning Herald. The author, Pat Sheil, repeatedly describes as a TESCREAList. He writes:

Emile P. Torres is a long-term mover and shaker in the eternal life caper. And there are many rich people giving money to people like them.

It’s not true that “many rich people” in the longevity community are funding my work, because no one in or around the TESCREAL movement is funding me. (Obviously!!) He continues:

And here’s where it gets scary, and why Krotoski’s book is important. It turns out that Torres remains an influential theorist and strategist for immortalists, and as such a most convincing preacher to insatiable venture capitalists (most of whom wouldn’t mind living forever either, funnily enough). Hence the signing of so many fat cheques.

Torres believes that anyone who stands in the way of this project is, by definition, a threat to the human species, and should logically be prevented from getting in their way. Their project includes mind-merging with AGI (Artificial General Intelligence, or the Singularity; the much-hyped “all-wise, all-knowing” son of AI), which thankfully doesn’t yet exist, and hopefully never will.

LOL and WTF.

This is defamatory, almost as bad as that Guardian article from a few years ago that misquotes me to suggest I’m a pro-extinctionist (I’m not).

I wrote the Herald, asking them to retract the article or at least correct the record, but they have yet to respond. Though I’m a little miffed about this, I guffawed while reading: it’s such a gross misrepresentation of my view, and the account of my view in The Immortalists, that I wonder if ChatGPT wrote the entire article? Genuinely quite amusing, amateurish stuff.

The 21st-Century Existential Mood

In my 2024 book Human Extinction, I argued that Western history can be divided into five distinct periods, each defined by a specific set of answers to questions like: Is human extinction possible? If so, how could it come about? What is the probability of our extinction? Is it inevitable in the long run? Could it happen in the near future? And so on.

I found it astonishing how abrupt the shifts from one period to another were. Over the course of a single year, or at most a decade, previously established answers to these questions were dramatically overthrown, often inducing a degree of psycho-cultural trauma. For example, virtually no one was talking about human extinction in the years just after 1945. Then the Castle Bravo disaster happened in 1954, which involved a thermonuclear weapon being detonated in the Marshall Islands and catapulting radioactive particles around the entire globe. Almost overnight, a very large number of eminent scientists began declaring that even a small-scale thermonuclear war could render Earth completely unsuitable for human life, thus resulting in “universal death.”

Another shift happened in the early 1850s, when scientists discovered the second law of thermodynamics. Over the course of just a couple years, people went from saying that human extinction probably isn’t even possible to acknowledging that it’s inevitable, as our sun burns out and Earth becomes an icy tomb floating about the darkness.

I argue that each of these periods correspond to a unique “existential mood,” by which I mean a kind of public mood — as in, the “mood” of the 60s was one of rebellion. Bertrand Russell captures the essence of the mood that arose after the second law was discovered, writing in his lugubrious 1903 “A Free Man’s Worship”:

All the labours of the ages, all the devotion, all the inspiration, all the noonday brightness of human genius, are destined to extinction in the vast death of the solar system, and that the whole temple of Man’s achievement must inevitably be buried beneath the debris of a universe in ruins.

I mention this because I argue in the book that our current existential mood was initiated in the late 1990s and early 2000s, and is marked by the sense that, however perilous the 20th century was, the worst is yet to come. The 21st century will be even more dangerous than the 20th. This is for two reasons:

First, a consensus emerged in the early 2000s that climate change is both anthropogenic and could have catastrophic consequences. (Yes, people had been talking about climate change for decades, but the issue was debated among scientists until the early aughts, when virtually all climatologists came to agree about its underlying causes and likely effects.)

Second, new anxieties about the immense destructive power of emerging technologies, including biotechnology and artificial intelligence. Bill Joy’s widely discussed 2000 Wired article, “Why the Future Doesn’t Need Us,” exemplified these anxieties. He argued that emerging tech like AI could be so dangerous that we should impose broad moratoria on entire fields of emerging science and technology. That’s basically what people like Eliezer Yudkowsky and groups like Stop AI are arguing right now.

I have watched, in realtime, this mood spread across the Western world. There’s a difference, I argued, between when a new mood first emerges and when it becomes widespread within a society. The existential mood that emerged around the turn of the century is now in full bloom. It’s everywhere you look, and many of us can feel it in our bones. I can’t go a single day without seeing dozens, if not hundreds, of posts on social media claiming that AI could destroy humanity in the coming years. Meanwhile, news about the climate crisis continues to cast a dark shadow over civilization, fueled by new research showing that global warming appears to be accelerating. Something terrible is about to happen — that’s the essence of our current mood, and expressions of it and the psycho-cultural trauma it’s inflicting are increasingly omnipresent.

In a bit more detail:

There’s growing talk about a nuclear weapon being used against Iran, which doesn’t seem out of the question given that Israel just committed a genocide (indicating that it cares not about violating international laws, norms, and taboos), and the US is run by a demented madman who seems hangry for geopolitical conflict (Venezuela, Iran, annexing Greenland, talk of taking over Cuba, etc.).

With respect to climate change, a recent study found that “climate change’s rising seas may threaten tens of millions more people than scientists and government planners originally thought because of mistaken research assumptions on how high coastal waters already are.” In a Nature article titled “The World Just Lived Through the 11 hottest Years on Record — What Now?,” the authors write that “measurements of Earth’s energy input and output reveals that the planet is more out of balance than ever before.” It includes this quote:

“We seem to be entering this new era where temperatures will be significantly higher than what they were ten years ago,” says climate scientist Sarah Perkins-Kirkpatrick at the Australian National University in Canberra. The past three years have seen large changes in temperature that could only be a result of climate change, she adds.

Another study in Nature finds that

extreme global climate outcomes may occur even under moderate 2 °C warming for several sectors. For droughts in global key breadbasket regions, precipitation extremes over highly populated areas and fire weather extremes across forests, global climatic impact-drivers at 2 °C of global warming may turn out to be much more extreme than model-averaged projections at 3 °C or 4 °C warming.

We’re already at 1.5C of warming — 2024, the hottest year on record, reached about 1.55C above pre-industrial levels. 2023 was the second hottest, and 2025 the third. Yet this year could exceed the record, and indeed studies suggest there may be a “globally catastrophic” super-El Nino event forming by spring. This could make 2027 even worse than 2026, as each of the three super-El Nino events since 1980 have been “followed by a year of record-breaking heat globally.”

Already this year, sea-surface temperatures are off the charts (below), and “new evidence shows antarctic melting is already locked in,” meaning that there’s nothing we can do at this point to avoid devastating sea-level rise, which will affect upwards of 1 billion people.

Just this month, temperature records were broken throughout the US, likely setting an all-time record for the month. As CNN reports, the city of Yuma, Arizona, saw temperatures soaring to 109F, while “the temperature near Martinez Lake, Arizona, hit 110 degrees on Thursday and 112 degrees on Friday” (the 19th and 20th of March). To quote the CBC:

A huge heat dome is spreading across the United States and it is shattering March temperature records. Weather historians say the dome has already smashed statewide March records in 14 states. Now, the gigantic heat dome that’s baked the Southwest is creeping eastward and may end up being one of the most expansive heat waves in American history, meteorologists and weather historians said. Experts say the heat wave’s footprint may rival major events in 2012 and 2021.

Another study reports that the US’s carbon emissions may have caused $10 trillion in damage since 1990. Much of this, of course, disproportionately hurts the most vulnerable people around the world who, historically, have contributed the least to climate change. That’s the main focus of climate justice.

Adding to the insanity of this situation, Trump just declared environmentalists to be “terrorists.” I guess that makes me a “terrorist”? For, you know, wanting humanity to not destroy our exquisitely unique, beautiful little oasis in space? What a joke.

That said, almost no one thinks that climate change will cause our extinction — the complete elimination of every person on Earth. But it could very well push civilization over the precipice of collapse.

Recall a University of Exeter study that calculates a GDP loss of greater than or equal to 25% and over 2 billion deaths if we reach 2C of warming by 2050. If we reach 3C, we should expect a greater than or equal to 50% loss of GDP and more than 4 billion deaths. This is an incredibly dire situation, yet climate apocalypticism has been largely eclipsed in the popular media by warnings that an omnicidal AGI might kill humanity before climate change topples civilization. It has been incredible to see this idea metamorphose from an obscure worry held by a fringe group of AI doomers into a topic now discussed on major media outlets like CNN.

In the next few days, I’m hoping to see The AI Doc, which has received quite a bit of attention. As far as I can tell, it’s mostly about the internecine squabbles between people within the TESCREAL movement — e.g., the doomers versus the accelerationists. The former believe that AI capabilities research should be stopped until AI safety researchers have solved the control problem, whereas the latter don’t seem to care one bit if ASI annihilates humanity. As the computer scientist David Krueger writes, “there are a significant number of people in the AI research community who explicitly think humans should be replaced by AI.” To which Max Tegmark replied: “I’ve been shocked to discover exactly this over the years through personal conversations. It helps explain why some AI researchers aren’t more bothered by human extinction risk: It’s not that they find it unlikely, but that they welcome it.”

I will almost certainly write a review of the film, though that might be difficult given that I’m completely immersed in writing my book. Currently halfway through chapter 5, after which I’ll have only two chapters left. I cannot wait for this to get published, because I think (hope) it offers a devastating and original critique of the ASI race — one that no one else is making! Thank you so much for supporting me while I work on this project, by the way!!

Incidentally, I have a section in the book in which I discuss the reasons one might have for rejecting the claim that ASI might be imminent, and that once here it will annihilate humanity by default. I’m very curious about your thoughts — am I missing something? What reasons do you have for rejecting TESCREAL doomerism? Here’s what I write:

… In fact, I would go further and argue that we should all be outraged even if one thinks an ASI-induced extinction catastrophe won’t happen in the near future — or ever. You might believe, for example, that

1. The systems that power current AI, large language models (LLMs), aren’t going to get us to AGI by themselves. No matter how much AI companies scale them up by increasing compute (computational resources), training data, and their parameters, they just don’t have the right architecture to become AGI. LLMs by themselves are a dead end, though there might be other systems, architectures, and approaches that could eventually get us there.

2. AGI isn’t a technology that we could ever build. It might simply be too difficult for our species. This is made plausible if one believes there may be other technologies that are in theory possible but in practice out of our reach, such as spacecraft that travel at 99.99% the speed of light. Perhaps some kind of super-clever alien species could figure this out, but we probably never will.

3. AGI, and especially superintelligence, isn’t possible to build in theory. It’s like a perpetual motion machine, or designing a spacecraft that can exceed the speed of light. There’s just no way to build an artificial system that surpasses human capabilities in every cognitive domain of interest.

4. AGI and ASI are not even coherent ideas to begin with. What does it mean to build an “everything machine” that can “exceed” humans in every domain of interest? Heck, AI companies and researchers can’t even agree upon a definition of “AGI” — OpenAI itself proposes multiple inconsistent definitions on their own website! What, then, are we even talking about?

I am very sympathetic with the first view: LLMs are not a ticket to AGI, though I’m also sympathetic with claims that “AGI” might not be a coherent concept in the first place. Insofar as it is coherent, I suspect it might not be possible for us to build, but for a different reason than stated above: it could be that a certain degree of societal, political, economic, etc. stability is necessary to build AGI. However, the stepping-stone systems that we’d need to build in order to reach AGI may wreak so much havoc that the fabric of society unravels, thus making AGI unreachable. In other words, there may be a negative feedback loop here such that the closer we get, the further away we end up: building AGI requires societal stability, but the more AI we have, the less stable things become. We’ll return to this in a moment.

What do you think? Here’s a poll, but I’d also love to know your thoughts in the comments section.

As always:

Thanks for reading and I’ll see you on the other side!

I’m worried that most people don’t understand who Yudkowsky is and what he actually believes. (He’s literally argued that a child only has a “right to live” “sometime after 1 year and before 6 years.”) Here’s a short excerpt from my forthcoming book, Clown Car Utopia: Why We Must Stop AI to Save Humanity:

Another figure who’s repeatedly expressed pro-extinctionist views is Yudkowsky himself. This may be surprising given that Yudkowsky frequently talks about the importance of avoiding human extinction, but we’ll see below that what he means by “human extinction” is very different from what the rest of us mean. In fact, he advocates for a future in which our species will almost certainly die out by being replaced by posthumanity — ideally, the particular version of posthumanity that he favors, which is not the same posthumanity as what accelerationists like Larry Page, Richard Sutton, and Gill Verdon (“Beff Jezos”) endorse.

During a conversation with explicit pro-extinctionist Daniel Faggella on The Trajectory podcast, Yudkowsky declared that

if sacrificing all of humanity were the only way, and a reliable way, to get … god-like things out there—superintelligences who still care about each other, who are still aware of the world and having fun—I would ultimately make that trade-off.

Yudkowsky emphasizes that this “isn’t the trade-off we are face with” right now. But if it were, he’d willingly sacrifice our species to see artificial super-beings flitting about the universe “having fun.” He repeated this idea during a recorded conversation with Stephen Wolfram, a computer scientist who delivered talks at the Singularity Summit in 2009 and 2011 (cofounded by Yudkowsky, Ray Kurzweil, and Peter Thiel). “It’s not that I’m concerned about being replaced by a better organism,” he told Wolfram, “I’m concerned that the organism wouldn’t be better.” Once more, replacement itself isn’t the issue.

Yudkowsky went into even more detail on the Bankless podcast in 2023, arguing that once creating posthumanity becomes feasible, it may be unethical to have biological children. Using rather offensive language, he said:

I have basic moral questions about whether it’s ethical for humans to have human children, if having transhuman children is an option instead. Like, these humans running around? Are they, like, the current humans who wanted eternal youth but, like, not the brain upgrades? Because I do see the case for letting an existing person choose “No, I just want eternal youth and no brain upgrades, thank you.” But then if you’re deliberately having the equivalent of a very crippled child when you could just as easily have a not crippled child.

Yudkowsky continued:

Like, should humans in their present form be around together? Are we, like, kind of too sad in some ways? I have friends, to be clear, who disagree with me so much about this point. (laughs) But yeah, I’d say that the happy future looks like beings of light having lots of fun in a nicely connected computing fabric powered by the Sun, if we haven’t taken the sun apart yet. Maybe there’s enough real sentiment in people that you just, like, clear all the humans off the Earth and leave the entire place as a park. And even, like, maintain the Sun, so that the Earth is still a park even after the Sun would have ordinarily swollen up or dimmed down.

Okay, so. Get rid of humanity and turn Earth into a nature reserve. Meanwhile, posthumans would reside in virtual-reality worlds (what he calls “computing fabric”) powered by megastructures called Dyson swarms that envelope the Sun and harvest nearly all of its energy output.

Kokotajlo, another TESCREAL utopian, has made similar remarks, as when he told the New York Times that

I’m a huge fan of expanding into space. I think that would be a great idea. And in general, also solving all the world’s problems, like poverty and disease and torture and wars. I think if we get through the initial phase with superintelligence, then obviously, the first thing to do is to solve all those problems and make some sort of utopia, and then to bring that utopia to the stars would be the thing to do.

The thing is that it would be the AIs doing it, not us. In terms of actually doing the designing and the planning and the strategizing and so forth, we would only be messing things up if we tried to do it ourselves.

So you could say it’s still humanity in some sense doing all those things, but it’s important to note that it’s more like the AIs are doing it, and they’re doing it because the humans told them to (italics added).

I suspect that if you knew something about the TESCREAL movement to which these people belong, you would be rather mortified. They are not actually opposed to superintelligence, nor are they pro-human, as I explain in detail here.

To the contrary, Yudkowsky’s institute, MIRI, explicitly says:

We remain committed to the idea that failing to build smarter-than-human systems someday would be tragic and would squander a great deal of potential. We want humanity to build those systems, but only once we know how to do so safely.

The whole point of building a “value-aligned” superintelligence, they say, is to be aligned with the values of TESCREAL utopianism — to radically transform us into digital posthumans, and then colonize space and conquer the universe. This is not an exaggeration. It’s the core vision of the TESCREAL ideologies.

I am begging you to please stop joining hands with pro-extinctionists. Yes, their interests are temporarily aligned with yours: shut it all down, a position that I myself passionately advocate. But siding with them is like siding with a murderer who says he won’t kill you until next year. I get the impression that you are on Team Human, on the side of our species. They are not. They are on Team Posthuman.

Another thing: you’ll hear people like Yudkowsky talk about the importance of avoiding “human extinction.” But what he means by the term is not what you think. People in the TESCREAL movement define “human” in an idiosyncratic manner — to mean our species plus whatever posthuman successors we might create. On this definition, our species could die out next year without human extinction having happened. So long as we have posthuman successors to take our place, then “humanity” will persist, and hence human extinction will not have occurred.

Here are some examples from TESCREALists in the same tradition as Yudkowsky (quoting from a peer-reviewed article of mine):

Nick Beckstead (2013) writes that “by ‘humanity’ and ‘our descendants’ I don’t just mean the species homo sapiens [sic]. I mean to include any valuable successors we might have,” which he later describes as “sentient beings that matter” in a moral sense. Hilary Greaves and MacAskill (2021) report that “we will use ‘human’ to refer both to Homo sapiens and to whatever descendants with at least comparable moral status we may have, even if those descendants are a different species, and even if they are non-biological.” And Toby Ord (2020) says that “if we somehow give rise to new kinds of moral agents in the future, the term ‘humanity’ in my definition should be taken to include them.”

Please don’t be fooled by their linguistic trickery. When they talk about preventing human extinction, they aren’t talking about ensuring the survival of our species. Our survival only matters insofar as it’s necessary to bring about what Yudkowsky calls the “glorious transhumanist future.”

Perhaps I have misread your positions. Bernie, Neil, and anti-AI protestors may very well be on Team Posthuman. I hope that’s not the case. If you are indeed on Team Human, please stop propping up people who say they’d literally “sacrific[e] all of humanity” to create “worth successors” in the form of artificial superintelligence.

Sincerely, Émile

Again, this is the system that Reid Hoffman (friend of Epstein!) called “Universal Basic Superintelligence,” and which Sam Altman touted last year as having PhD-level knowledge. In truth, it’s nothing more than a stochastic parrot vomiting up bits and pieces of its training data, leading to egregious failures of basic reasoning like the one above.

Also, I’m sure that many of you have seen this by now, but in case you haven’t, here’s Marc Andreessen — noted AI accelerationist who once included “TESCREAList” in his Twitter bio — claiming that he never engages in any introspection:

He then said “I regret nothing” after his remarks went viral for all the wrong reasons. To that, I responded:

Silicon Valley truly is run by sociopaths with no empathy for others and no ability or desire to reflect on their behaviors, feelings, and ideas. Which leads us to the main topics of this post!

The Race to Superintelligence Isn’t a Collective Action Problem

I’m now 40,000 words into my book — just finished chapter 3. Thank you so much for supporting me while I write this. It might be done in another two or three weeks!

The working title is Clown Car Utopia: Why We Must Stop AI to Save Humanity, though I argue that we can’t stop AI without stopping, countering, defanging, and neutralizing the TESCREAL movement. This will be the culmination of four years of academic and popular media articles, as well as countless podcast, radio, and TV interviews that I’ve done. But really, it’s the culmination of 20 years of research, as I first became interested in the TESCREAL movement around 2006, after stumbling upon the work of Ray Kurzweil and Nick Bostrom. Every single page of this book contains something that will have readers saying, “What the hell did I just read?” — because the TESCREAL movement is endlessly bizarre, outrageous, and absurd. In a sense, the book contains a “greatest hits” catalogue of the most cockamamie things TESCREALists have said and done, with receipts.

While researching a section of the book, I re-listened to an interview with Holden Karnofsky on the 80,000 Hours podcast. Karnofsky was an important figure in the early development of EA. He started GiveWell, which worked closely with Toby Ord and William MacAskill’s Giving What We Can, and cofounded Open Philanthropy (now Coefficient Giving). He was roommates with Dario Amodei, the CEO of Anthropic, and married Dario’s sister Daniela. He’s now a member of the technical staff at Anthropic, where he advises “the company on preparing for risks from advanced AI.”

In the interview, he says that “the AGI race isn’t a coordination failure.” That it is a coordination failure is the standard story told — one finds it, for example, in Karen Hao’s excellent book Empire of AI. The story goes:

Every leading figure at the major AI companies — DeepMind, OpenAI, Anthropic, and xAI — believes that AI safety is extremely important. If we build a “value-misaligned” ASI (artificial superintelligence), it will destroy humanity along with our “glorious transhumanist future” among the stars. But if we ensure that it embodies “our values” (by which they mean the values of the TESCREAL worldview), then we get a posthuman paradise, a heaven among the literal heavens.

The problem is that each company thinks that it’s more responsible than the others, and hence that it should be the one to reach the ASI finish line before everyone else. No one wants a race — they just look around at the other companies and conclude, “Well, if we don’t speed up, they’re going to get to ASI before us, which means the probability of doom will be higher than if we got there first.” Hence, the coordination or collective action problem driving the arms race.

Karnofsky disagrees — and I agree with his assessment. This isn’t a collective action problem. Why? Because many people at the AI companies (a) don’t care if ASI wipes out humanity; some think that would actually be a good thing, or (b) think the risk of annihilation is completely worth it for the chance that they get to become immortal posthumans.

This is exactly the line of reasoning in Bostrom’s atrocious new paper arguing for the accelerationist thesis: if ASI is sufficiently value-aligned to give us more than a thousand years of extra life, then we should push ahead even if the probability of ASI being value-aligned is only 3%. In other words, we should build ASI as soon as possible even if the probability of doom is 97%. If you don’t believe him, just do the math: the expected value of risking almost certain annihilation is higher than the alternatives if ASI lengthens our lifespans by more than 1,000 years. As Eliezer Yudkowsky says, “Shut up and multiply!”

The reasoning here, by the way, is called “Pascal’s mugging.” It’s a terrible form of argument, but one that many people at these AI companies embrace. Put more colloquially, these people are gripped by a kind of YOLO (you only live once) attitude: “Look, if I’m going to die someday anyways, why not race ahead in hopes of living forever, even if doing this risks the lives of everyone on Earth? It’s now or never, baby, so pedal to the metal!”

Here’s what Karnofsky says:

I think most of the players in AI are going to race. And if, for example, Anthropic were to say, “We’re out. We’re going to slow down,” they would say, “This is awesome. That’s the best news. Now we have a better chance of winning, and this is even good for our recruiting” — because they have a better chance of getting people who want to be on the frontier and want to win.

When asked whether OpenAI, DeepMind, and xAI would slow down if Anthropic dropped out, he said:

Let’s take an even stronger hypothetical. Let’s say that not only Anthropic, but everyone in the world who thinks roughly the way I do — everyone in the world who thinks AI is super dangerous, and it would be ideal if the world would move a lot slower, which I do think — let’s say that everyone in the world who thinks that decided to just get nowhere near an AI company, nowhere near AI capabilities. I expect the result would be a slight slowing down, but not a large slowing down.

I think there’s just plenty of players now who want to win, and they are not thinking the way we are, and they will snap up all the investment and capital and a lot of the talent.

During an interview I gave last week, I explained that there are two general groups of accelerationists. The first group thinks that ASI will by default be value-aligned. Marc Andreessen seems to hold this view. He appears to think that if we just plow ahead with ASI, it will by default bring about a utopian world of radical abundance, human enhancement, and space colonization. The second group thinks that it doesn’t even matter whether ASI is value-aligned. Indeed, many argue that ASI shouldn’t be value-aligned — it should have its own “alien, inhuman” values.

In both cases, the conclusion is — as noted — pedal to the metal. That’s why Karnofsky thinks this isn’t a coordination problem. Even if one, two, or all of the companies disbanded, there would still be people who’d immediately start new companies to race toward ASI as quickly as possible.

There’s no stopping the ASI race unless the government swoops in and imposes robust regulations to prevent this from happening. And since the government won’t do that, we’re kinda screwed — not because ASI is actually around the corner. I don’t believe that at all. But rather, because the race itself is causing profound harms to the world. Because we don’t need AGI or ASI for AI to destroy our society.

How in the Hell Is This Acceptable?

That brings me to the second issue, which I’ve written about before. If you were to send me a death threat, you might get arrested and charged. If you were to say, “Okay, I might not actually kill you, but there’s a real chance that I will,” you could still get in trouble. I’d call the authorities and they’d act accordingly.

However, if you say, “I might kill everyone on Earth,” you apparently won’t get in any trouble at all. No one will call the cops, the authorities won’t act, and nothing would ultimately happen. I know this because virtually all the AI company CEOs or founders have said something exactly like that: “We’re building a technology that might kill you, your partner, your mother and father, your children and grandchildren (if you have any), your grandparents (if they’re still around), and all your friends in the near future.”

Sam Altman (CEO of OpenAI) says that

• “the bad case ... is lights out for all of us.”

• “machine intelligence is something we should be afraid of.”

• “AI will … most likely sort of lead to the end of the world, but in the meantime there will be great companies created with serious machine learning.”

• “probably AI will kill us all, but until then we’re going to turn out a lot of great students.”

Dario Amodei (CEO of Anthropic) says there’s a 25% chance of total annihilation. He claims that

• “there’s a long tail of things of varying degrees of badness that could happen. … I think at the extreme end is the … fear that an AGI could destroy humanity. I can’t see any reason in principle why that couldn’t happen.”

Demis Hassabis (CEO of DeepMind) says that

• the probability of total annihilation is “definitely non-zero and it’s probably non-negligible. So that in itself is pretty sobering.”

• we must “take the risks of AI as seriously as other major global challenges, like climate change. … It took the international community too long to coordinate an effective global response to this, and we’re living with the consequences of that now. We can’t afford the same delay with AI.”

Shane Legg (cofounder of DeepMind) puts the probability of extinction from ASI between 5% and 50%, and writes that

• “a lack of concrete AGI projects is not what worries me, it’s the lack of concrete plans on how to keep these safe that worries me.”

• “eventually, I think human extinction will probably occur, and technology will likely play a part in this.”

Elon Musk (cofounder of OpenAI and xAI) argues that ASI poses the “biggest existential threat” to humanity, and claims that it’s “potentially more dangerous than nukes.” He adds that

• “with artificial intelligence, we are summoning the demon. You know all those stories where there’s the guy with the pentagram and the holy water and he’s like, yeah, he’s sure he can control the demon? Doesn’t work out.”

The point I want to get across is this: even if you think these people are crazy for thinking ASI might be imminent and could potentially annihilate humanity, it’s absolutely f-ing outrageous that they get away with saying stuff like this.

In a sane world, they would be locked up and their companies dissolved. In a sane world, saying that you might do something that kills everyone on Earth would be worse than saying you might do something that kills “only” one or two people. Why is it unacceptable to suggest you might murder one or two people but somehow okay to suggest you might kill 8.2 billion?

Even more, what kind of profoundly unethical sociopath even suggests they might kill 8.2 billion people in the first place? Imagine your best friend or partner sitting you down one evening at the dinner table and telling you with a straight face: “I’m going to do something that might kill everyone on Earth, including you.” You joke, “Pfff, what are you talking about? Is this supposed to be a joke?” They say, “No, it’s not. I’m 100% dead serious.”

You would, of course, be completely freaked out by that. It’s a really, really weird thing for someone to say out-loud. You might wonder if they’re having a mental breakdown, or experiencing an episode of psychosis. Because what they’ve just said to you isn’t sane, normal, or acceptable. If, over the next several weeks, they continue to repeat this claim, you might even end your friendship with them — perhaps after calling the authorities or a mental health specialist.

Yet AI leaders have repeatedly said just this, in public. I don’t understand how they aren’t constantly deluged on social media with posts from people saying “F*ck you for threatening to kill me and my family.”

Again, one doesn’t need to believe that an ASI apocalypse is imminent to hold this attitude — it’s an incredibly disturbing thing to hear out of anyone’s mouth, especially the mouths of billionaires running companies with valuations of hundreds of billions of dollars. We do not live in a sane world.

I am personally very angry that Altman and the others have said they might kill me and my family, even though I don’t think we’re anywhere close to ASI. Totally unacceptable behavior.

But what do you think? Am I somehow wrong? As always:

Thanks for reading and I’ll see you on the other side (next week)!