I’d like to answer, in writing, two questions that journalists have been asking me the past few days:

• Do people like Jacob Coxon, Dario Amodei, and Sam Altman sincerely believe that ASI (artificial superintelligence) could bring about human extinction?

• Why won’t they just slow down? As David Sacks said on X, they don’t need the government to voluntarily slow down to, you know, not kill everyone on Earth.

The Whole Discourse Is Fucked

The first point I’d make is that the whole public conversation about “human extinction” is fundamentally misleading. When people who espouse the Silicon Valley worldview — which is to say, the TESCREAL worldview — talk about “human extinction,” they don’t mean what the overwhelming majority of people think they mean.

Most people interpret this as saying: We must save the human species — biological humans. That’s not what they mean. If our species were to die out, this would constitute terminal extinction. But there’s a different kind of extinction that the TESCREALists are actually talking about: final extinction, whereby we die out and leave behind no “worthy” posthuman successors to take our place.

This distinction really, really matters. There’s a massive difference, they believe, between us dying out and leaving behind successors, and us disappearing without doing so.

The whole point of the Silicon Valley worldview is to build an ASI that will bring about a cosmic posthuman utopia. They want it to transform us into digital posthumans who can then colonize the universe. (Some just want ASI to simply replace us like an invasive species let loose on an island. See this article for details.)

The only type of extinction that would permanently prevent the creation of a cosmic posthuman utopia is final extinction. The only reason we should avoid terminal extinction — the disappearance of our species — is that if terminal extinction were to happen tomorrow, it would entail final extinction.

But if terminal extinction were to happen after “worthy” digital posthumans arrive, then it wouldn’t entail final extinction. Since final extinction is the only way a cosmic utopia would be permanently expunged, TESCREALists care about terminal extinction only insofar as it would entail final extinction.

Once posthumanity is here, who gives a shit about what happens to our species? As I write in great detail here, many of these people hold the “replacement view” (also called “pro-extinctionism”), according to which our species should die out once posthumanity arrives.

To emphasize, many, many hold this view. Yudkowsky holds this view. So does Sam Altman, Larry Page, and Gil Verdon — and so on, and so forth. Using my terminology from above, what this amounts to is that these people want terminal extinction without final extinction — which is just to say, they want our species to stick around long enough to create or become our posthuman successors, and then to kick the bucket.

There are two reasons for this: (1) once posthumans rule and run the world, Homo sapiens will become obsolete and irrelevant in every respect: technologically, scientifically, economically, and even politically. We will serve literally no purpose. (2) Even worse, we will continue to suck up resources that our “superior” posthuman successors could use “more efficiently” — for things like covering Earth in data centers (which Altman says is likely), building a Dyson swarm around the sun (as Demis Hassabis says will happen starting in the early 2030s), and spreading the light of digital consciousness to the stars (which all of them endorse).

There may even be “ethical” reasons for getting rid of us. As Yudkowsky declares, “Are we, like, kind of too sad in some ways” to keep around? He adds that once posthumanity is here, continuing to create biological humans would be like someone right now intentionally deciding to have a child with disabilities. (Yes, this is outrageously ableist. But then again, he is a eugenicist of the most radical kind.1)

***

So, this is why the conversation about “human extinction” is completely fucked. Making matters worse, the TESCREAL academic literature explicitly defines “human” as our species plus whatever posthuman successors we might have. This immediately implies that our species could completely die out next year without “human extinction” having happened. As long as there are posthumans to take our place, “humanity” will persist and “human extinction” will have been avoided. When they talk about “preserving humanity,” they really mean “securing posthumanity.”

What does this all mean? That if the TESCREAL extremists with their bizarre Silicon Valley fantasies about a cosmic posthuman utopia were to succeed in building an ASI that’s “aligned” with the values of TESCREALism — and that’s their goal — then human extinction would almost certainly result, because we’d be pushed out of existence by digital posthumans and ASI, exactly as Altman predicts.2

Despite their screams about “avoiding human extinction” if ASI is “misaligned,” most of them actually advocate for human extinction, where “human” here specifically means “our species.” Put another way: if what one cares about is avoiding terminal extinction — and I do! — then these people are pro-extinctionists. (My view, to be clear, is that our biological species is what matters, for reasons I’ve outlined elsewhere.)

The media needs to understand this, fast. What the TESCREAL doomers are worried about is an ASI that wipes out our species, thus foreclosing the possibility of us becoming or creating “worthy” successors: posthumans. What they want is an ASI that enables us to become or create “worthy” successors that usurp humanity.

In the first case, the outcome would be final extinction (and hence terminal extinction, since final extinction entails terminal extinction). In the second case, the outcome would be terminal extinction but not final extinction. Once again, the goal for many if not most of them is terminal without final extinction.

Do They Sincerely Believe?

Now we can be more precise with the first question: do Coxon, Amodei, Altman, and the others sincerely believe that “misaligned” ASI could bring about final human extinction?3 Or are they just using extinction worries to further concentrate power: “We’re building this thing that could kill everyone (and erase utopia), and we hold the only solution, so give us more money and power!”

These are not mutually exclusive! I spent years in the TESCREAL movement, and can confirm that everyone is genuinely scared shitless about the possibility of a misaligned ASI murdering everyone. They aren’t lying when they say that an existential catastrophe caused by ASI is a real possibility.

That said, I have no doubt that many AI CEOs realize the “extinction risk” narrative also gives them more power. And that’s what they want. Longtermists have literally talked about trying to acquire the greatest possible fraction of the world’s wealth and power.

These people are, essentially, religious extremists: they believe fervently in their vision of the future; they believe that literally “astronomical” amounts of value hang in the balance re: getting ASI right; they believe they have special access to fundamental “truths” about what the future ought to look like; they think the masses are too stupid to understand the stakes, and hence that democratic input from the rest of humanity is unnecessary and irrelevant.

This is one of the dangers of worshiping “rationality.” Those who see themselves as having superior rationality assume they have a fully objective “view from nowhere.” It’s not a point of view, it’s the point of view of objective truth, so to speak. Why do you need to ask people in India, Nigeria, China, Bolivia, etc. what the future should be like when you can clearly see what it should look like from your special, objective view from nowhere?

Because they believe that realizing a cosmic posthuman utopia is so important, they will stop at nothing to acquire as much wealth and power as possible to ensure this comes to pass. And warning about “ASI extinction risk” helps them acquire the wealth and power they so desperately want, for what they see as fundamentally ethical (utilitarian) reasons (“maximize value in the cosmos!”).

Do not underestimate how power-hungry these people are. As someone who spent a decade in the TESCREAL movement, I can personally attest to many conversations behind closed doors about how to do this!

Why Not Stop?

If the AI CEOs think that misaligned ASI might kill everyone and, in the process, permanently erase our cosmic posthuman future, why don’t they just stop? Why continue building ASI?

There are several reasons. First, many of these people suffer from extreme death anxiety. They think that if we build an ASI that is “aligned,” it will grant them eternal life, enabling them to upload their minds to computers, etc. This is one reason.

Second, they all love expected value (EV) theory. Given the supposed “astronomical” value of cosmic utopia, even a relatively high probability of total annihilation in the future could still justify moving forward. Imagine there’s a 20% chance of your plane going down and you dying. (This is Amodei’s estimate of final extinction in the near future from misaligned ASI.) Now imagine that if it doesn’t go down, you get to become an immortal super-being who literally owns a galaxy forever in the literal heavens with all your best friends. The immense “value” of this positive outcome makes an 80% chance of surviving worth taking.

Third, you have to understand how these people define an “existential risk” or “existential catastrophe.” Introduced specifically in the context of libertarian transhumanism, it refers to any event that would permanently prevent us from creating a cosmic posthuman utopia. Final human extinction is the most obvious type of existential risk/catastrophe. But there are an infinite number of existential catastrophes in which our species, Homo sapiens, survives.

To illustrate: imagine that we somehow establish world peace. Nuclear nations abolish their arsenals. There are no more wars, climate change is reversed, and the ecological situation is radically improved. All diseases are cured, and people live wonderful lives into their late 90s. Loneliness disappears, everyone is happy, and we all live in sustainable eco-technological villages amid meadows and gardens watching the sunset every night. After 1 billion years of this, we collectively decide to “die with dignity” by allowing our species to fade away as the sun becomes a Red Dwarf that renders Earth uninhabitable. Despite a rocky start to civilizational history, everyone around the world is at peace with how well our story turned out.

This would be an existential catastrophe whose badness is on par with final human extinction next year in some kind of devastating bloodbath. Why?

Because in both cases we will have failed to realize more than a teeny-tiny fraction of all the cosmic “value” that could have otherwise existed within our future light cone. And it’s the realization of this future “value” that ultimately matters — value for whom? That’s the wrong question, they’d say. This “value” is just intrinsically valuable, from “the point of view of the universe.” The more “value” the universe contains across space and time, the better it becomes, full stop. (If you find this unsatisfying, then join the club!)

This is the real reason the AI companies won’t stop: ASI is almost certainly necessary to create a cosmic posthuman utopia. As Bostrom puts it, “all the plausible paths to a really great future involve the development of machine superintelligence at some point.”

You’ll hear the AI companies give different reasons like:

1. We can’t stop because we’re in a race with other US companies. These other companies are less responsible than us. If they reach the ASI finish line first, we might all die (final extinction). If we reach it first, then at least we have a chance.

2. We can’t stop because we’re in a race with Chinese companies. If China wins, then it will establish a global authoritarian communist state.

But think about what underlies these excuses. Both are reducible to the TESCREAL goal of creating utopia.

With respect to (1), the companies are saying that if another, less responsible company creates ASI first, it will probably be misaligned, and misaligned ASI would be very, very bad because it would destroy utopia.

With respect to (2), the companies are saying that if China were to build an ASI aligned with its political ideology, this would also prevent the realization of the utopian TESCREAL worldview. Would China proceed to reengineer humans to create a new species of digital posthumans, who then go out and plunder the universe’s vast resources like a good utilitarian and capitalist? Maybe not — probably not. No one in China, so far as I know, espouses the TESCREAL worldview. Their goals are different, and that poses a threat to the TESCREAL movement.

The ultimate reason these companies won’t stop is because ASI is believed to be necessary to create utopia, and these people desperately want utopia as soon as possible. Even the doomiest doomers want ASI — specifically, “aligned” ASI — as soon as possible. As Yudkowsky’s Machine Intelligence Research Institute puts it, borrowing the language of longtermism,

we remain committed to the idea that failing to build smarter-than-human systems someday would be tragic and would squander a great deal of potential [i.e., posthumans living in computer simulations4]. We want humanity to build those systems, but only once we know how to do so safely.

***

This is why I have been shouting for years that, to successfully stop the ASI race, we need to address the underlying ideological drivers that are pushing it forward. So long as those drivers remain in place, influential and powerful, championed by power-hungry billionaire oligarchs, the ASI race will move forward.

The question of “pausing” or “slowing down” the race specifically concerns the race between AI capabilities research and AI safety research. The former aims to build an actual ASI, while the latter aims to ensure that it’s “aligned” with the TESCREAL worldview. The point of a slow-down would be to hit the breaks on capabilities research so that safety research can catch up, because this is how they maximize the probability that ASI is going to help us realize a cosmic posthuman utopia.

This isn’t about not building ASI. That would be existentially catastrophic. And as I argued above — and explain in immense detail here — if these TESCREAL extremists succeed in building an “aligned-with-TESCREALism” ASI, the likely outcome would be terminal extinction: the expungement of our species.

If there is an extinction risk here, it arises from the successful realization of the TESCREAL vision. This is what everyone is missing.

When these people warn about “human extinction,” they’re talking about final human extinction, whereby we don’t get to create or become “worthy” posthumans who will then replace our species. The probable outcome of ASI creating a new species of posthumans is that our species will be sidelined, marginalized, disempowered, and ultimately eliminated. As Ray Kurzweil once put it, if you choose not to become posthuman “you won’t be around for very long to influence the debate.”

Conclusion

So, TESCREAL folks really are worried about ASI causing final extinction. But they also no doubt understand that shouting about “extinction risk” will increase their power to control the situation — to control, ultimately, how the entire future of (post)humanity unfolds in the cosmos.

And they won’t stop trying to build ASI because they think it’s necessary to realize their techno-utopian fantasies. At most, they’ll slow down a bit to let safety research catch up, where the central focus of “AI safety” is to guarantee a cosmic utopia rather than total annihilation (where a cosmic utopia would probably mean the disempowerment and elimination of our species).

Finally, they exploit the ambiguity of the term “human extinction” to make their views sound infinitely more reasonable than they actually are. Members of the public who know nothing about the TESCREAL movement hear Altman and the others warning about “human extinction” and think: “I also don’t want our species to die out. I guess we’re on the same side.”

No! False! That is not what they mean. Again, Altman is explicit that the only scenario in which “we survive” is by becoming digital posthumans — something Altman himself has already taken steps to become by signing up with Nectome to have his brain digitized.

I am begging the media to investigate these issues more; I am pleading with them to make the effort to understand the Silicon Valley worldview that nearly every major figure in the ASI race embraces. This will not end well for humanity even if the ASI race is slowed down and an “aligned” ASI is successfully built.

Thanks for reading and I’ll see you on the other side!

Immense thanks to Alexander Thomas, author of the brilliant book The Politics and Ethics of Transhumanism (which you can read for free here), for thoughtful feedback on this post. Any errors are my own.

The (Existential) Mood of the Times

Back in ~2017, I was speaking to a friend of mine — a famous senior political scientist at Johns Hopkins — and I asked how he was doing. His response has haunted me ever since: “Every day feels like a funeral.”

The global predicament — or what the Club of Rome called the “world problematique” — has only worsened since then, and the slide appears to be accelerating. In my 2024 book Human Extinction: A History of the Science and Ethics of Annihilation, I introduced the idea of an “existential mood.” This is a kind of public mood — a “mood of the times” — that arises from a specific set of assumed answers to questions like:

• Is human extinction possible? Or is humanity fundamentally indestructible?

• If possible, how probable is it?

• Is this probability increasing or decreasing?

• Could extinction happen in the near term? Or are we safe for the time being?

• Is our extinction inevitable in the long run?

• How could our extinction happen? What are the various “kill mechanisms” that could catapult us into the eternal grave?

• Have we discovered all possible kill mechanisms? Will we create new kill mechanisms in the future?

• And so on.

  Realtime Techpocalypse Newsletter is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.

A Brief History of Thinking About Human Extinction

For nearly all of Western history, virtually everyone believed that humanity is indestructible(!). Even those who affirmed that human extinction could happen also believed that humanity cannot be destroyed. How does that make sense?

Well, they accepted a version of extinction that I call “demographic extinction,” whereby our species disappears entirely — but not necessarily forever.

For example, the Ancient Greek philosopher Xenophanes propounded a cyclical cosmology according to which at some point in the future all humans will disappear from the earth. However, as the cycle proceeds, we will inevitably return — until we disappear again. He thus accepted a kind of human extinction while nonetheless affirming the indestructibility of our species.

Once Christianity became widespread in the West during the 4th and 5th centuries (boosted by the 313 CE Edict of Milan and 380 CE Edict of Thessalonica), even the most elementary conception of our extinction — demographic extinction — became more or less unthinkable. It simply wasn’t part of God’s plan for the universe, for balancing the cosmic scales of justice at the end of time. Yes, the world as we know it will end, but humanity will endure through the cataclysmic paroxysms of the eschaton to join God in heaven on Earth forever.1

The point is that naturalistic extinction (think: dodos and dinosaurs) is rendered impossible by the eschatological narrative of Christianity and its metaphysical assertion that all humans possess immortal souls, which implies that humanity itself is immortal.

It wasn’t until the 19th century, as Christianity began to retreat, that people — for the first time in all of Western history — started to accept that humanity might not be fundamentally indestructible.

You find, for example, Lord Byron’s 1816 poem Darkness — inspired by the Year Without a Summer caused by the Mount Tambora eruption in Indonesia — which depicts the world completely bereft of all humans. Around the same time, the “last man” motif became widespread, known today mostly because of a book by Byron’s friend, Mary Shelley, titled The Last Man (1826).

Then, in the early 1850s, something devastating happened: physicists discovered the second law of thermodynamics (“Second Law”). They immediately recognized the apocalyptic implications: that Earth — if not the entire universe — will someday become uninhabitable. One scientist wrote of “the terroristic nimbus of the second law,” which he described as “a destructive principle of all life in the universe” (at the time — surprisingly — most reputable scientists believed in extraterrestrial civilizations). Another reported that

species after species of animals and plants will first degenerate and then become extinct, as the worsening conditions of life render it impossible for them to continue the struggle for existence; a few scattered families of degraded human beings living perhaps in snowhuts near the equator, very much as Esquimaux [dated language] live now near the pole, will represent the last wave of the receding tide of human existence before its final extinction; until at last a frozen earth incapable of cultivation is left without energy to produce a living particle of any sort and so death itself is dead.

Wild stuff!

Not long after, Bertrand Russell penned his famous 1903 essay “A Free Man’s Worship” — which basically plagiarized another article from Arthur Balfour (yes, that Balfour2). In it, he poetically lamented that

all the labors of the ages, all the devotion, all the inspiration, all the noonday brightness of human genius, are destined to extinction in the vast death of the solar system, and that the whole temple of Man’s achievement must inevitably be buried beneath the debris of a universe in ruins.

This gestures at what I mean by an “existential mood” — a vague sense of, in this case, inevitable doom that pervaded intellectual culture, if not society more generally. (Indeed, H. G. Wells brought this mood to those outside of academia with his 1895 The Time Machine, which depicts the end of earthly life.)

Many people at the time felt this mood in their bones. It lingered in the background. It cast a shadow on the present. It mumbled faintly in the back of people’s minds. The terrible realization that, in the end, all will be lost, and not even a trace of humanity having once existed will remain, framed people’s thinking about the cosmically hopeless human situation. Russell even describes it as a kind of “unyielding despair.”

Four Shifts in Mood

In the book, I argue that there have been exactly four well-defined shifts in existential mood, driven by four sudden changes in how people answered the various questions listed above. These are:

1. The first shift happened in the 1850s, due to the Second Law. It caused a double psycho-cultural trauma: not only is human extinction really possible, but it’s inevitable in the long run.

2. The second in 1954, due to the Castle Bravo debacle. Almost no one was talking about human extinction from nuclear war between 1945 and 1954. Immediately after Castle Bravo, though, it was on the minds of many leading scientists, who conveyed their horror to the public through letters like the Russell-Einstein manifesto and Mainau Declaration.

3. The third in the early 1990s, when scientists realized that global-scale natural catastrophes can happen; see the Alvarez hypothesis. (Scientists didn’t believe in natural global catastrophes prior to this because nearly everyone in the Earth sciences accepted a paradigm called “uniformitarianism,” which says that natural catastrophes are always localized. A new paradigm sometimes called “neo-catastrophism” took its place, which is our current reigning paradigm.)

4. And the fourth began in the early 2000s, when (a) a consensus about climate change first arose, and (b) people — in particular, early TESCREALists — began to speculate about how 21st-century technologies will introduce novel, unprecedented threats to human survival far more terrifying than the risks faced during the 20th century. The opening act of this mood was exemplified by Bill Joy’s 2000 Wired article, “Why the Future Doesn’t Need Us.”

The existential mood that resulted from the fourth shift — our current mood — is marked, I argue, by a haunting sense that however perilous the previous century was — with its nightmarish chemical, biological, and nuclear weapons, along with the giant hemoclysm of the two world wars — the present century will be far more perilous.

An Astonishing Transformation

In the book, I highlight how advanced AI would reinforce the new existential mood — and, insofar as this was a prediction, it happens to have been spot on.

Can you feel this existential mood? I bet you can! Just this week, a former Anthropic employee named Jacob Coxon has been appearing on major news outlets like CNN, MSNBC, and Fox News because he quit Anthropic and declared on X that “the people building AI earnestly believe that it could kill us all by the end of the decade.” As of this writing, his tweet has 119,000 likes and has been viewed 24.3 million times.

The current lead researcher of Alignment Science at Anthropic, Evan Hubinger, quote-tweeted Coxon’s post saying:

This also went mega-viral, as you can see.

It’s hard to overstate how astonishing this transformation is. Just ~two centuries ago — perhaps 7-8 generations in the past — nearly everyone would have claimed that humanity is fundamentally indestructible. Human extinction is not something to worry about, because it quite literally can’t happen. God wouldn’t allow it!

Even atheists during the 19th-century Enlightenment didn’t believe that we could disappear forever — or even temporarily. The French encyclopedist Diderot, arguably the first atheist in our modern sense, speculated that someday humanity might indeed vanish, but that surely we would re-emerge on Earth. He was channeling a secular version of the old way of thinking: it’s just not plausible to think that our species could disappear forever.3

Meanwhile — as you know — virtually every notable figure in the ASI (artificial superintelligence) race has expressed existential concerns similar to those of Coxon. Dario Amodei says the probability of extinction in the near future is 25%. Musk puts it at 20%. Demis Hassabis says the likelihood is “definitely non-zero and it’s probably non-negligible. So that in itself is pretty sobering.” Sam Altman says that “probably AI will kill us all,” and argues that the only way to “survive” the advent of ASI is to become digital posthumans by “merging” with machines (in other words, we will need to undergo terminal extinction). Shane Legg puts the probability of doom at 5-50%. Geoffrey Hinton says it’s at least 10%.

I don’t think that ASI is about to kill everyone on Earth. I see absolutely no reason to believe this. Just today, I read a RAND report from last year arguing that it’s extremely implausible that a rogue ASI could murder literally every last person — and I think they’re right.

But that’s not the point. The point concerns the public mood that now pervades Western society. This mood is undeniably reinforced by the growing proclamations from prominent researchers (many of whom are affiliated with the TESCREAL movement) of imminent doom.

Even if one rejects such proclamations, it’s still hard to avoid feeling the existential mood that’s now percolating into every corner of society. Just three days ago, two people wrote me — in the midst of Coxon’s media blitz — to ask whether this is it for humanity. One person is an old friend who I hadn’t spoken to for some 10 years. Meanwhile, university students are despairing; people are spending their retirement savings because they don’t believe there will be a future. And so on, and so forth.

***

It’s not just AI, as noted above. Another major cause of the current existential mood is of course climate change. Thanks to the record-shattering Super El Niño brewing in the Pacific, on top of all the warming that’s happened so far, 2027 is now projected to be 1.8C above pre-industrial levels.

Ecosystems are collapsing; the ocean is acidifying 4.1 times faster than it did during the Great Dying, the worst extinction event in Earth’s history; we’ve crossed 7 of 9 planetary boundaries; catastrophic, irreversible (on human timescales) tipping points are looming; the entire planet is covered in microplastics (even found in the Mariana Trench); and human activity (mostly in the Global North) has initiated the sixth major mass extinction event in life’s 3.8-billion-year history on Earth.

Entire regions of the planet will be — and are being — subject to heatwaves exceeding the 95-degree wet bulb temperature, meaning that you’d die even if you were naked, in the shade, standing in front of a giant fan. And one study from the University of Exeter estimates 4 billion premature deaths if we reach 3C of warming by 2050, and 2 billion if we exceed 2C.

Humanity won’t die out because of catastrophic climate change, either — unless there’s a runaway greenhouse effect, though climatologists believe the probability of this is very low. But many people don’t clearly distinguish between civilizational collapse and human extinction (in fact, these have been conflated by people going back at least to the early 1900s), and hence climate change is also intimately connected to the current existential mood, according to which the probability of extinction is unprecedentedly high and almost certainly rising.

Additional factors include:

• The continued risk of nuclear conflict, which appears to have risen due to the Russia-Ukraine war. Iran is also likely, I think, to pursue nuclear weapons even more vigorously given Trump’s dumb decision to start a war with them. The “hardliners” in Iran were essentially proven right by the US’s fatuous actions.

• The possibility of the US-Iran and Russia-Ukraine wars expanding.

• The possibility of engineered pandemics, perhaps enabled by AI. (Recall that scientists just created brand new viruses using AI.)

• The possibility of mirror life, which could potentially destroy most life on the planet.

• And so on.

***

A somewhat useful indicator of the current existential mood is the Bulletin of the Atomic Scientists’ Doomsday Clock, which aims to track our collective proximity to total annihilation. The clock was created in 1947, and was originally set at 7 minutes before midnight (or doom). It inched closer in 1953 — reaching 2 minutes before doom — the year after the US built the first thermonuclear weapon (far more powerful than the atomic bombs dropped on the Japanese archipelago), and then was set to a reassuring 17 minutes in 1991, after the Cold War officially ended.

Over the past few years, it has ominously moved closer and closer to midnight:

• In 2012, it moved to 5 minutes.

• In 2015, it moved to 3 minutes.

• In 2017, it moved to 2.5 minutes.

• In 2018, it moved to 2 minutes, tying the record.

• In 2020, it broke the record, being moved to 100 seconds.

• In 2023, it moved to 90 seconds.

• In 2025, it moved to 89 seconds, and …

• In 2026, it was set at a mere 85 seconds before doom.

Why? Because of nuclear proliferation, wars, climate change, synthetic biology, Trump and fascism, and advanced AI.

To put this in perspective, prior to the 1950s, the only plausible kill mechanisms were natural: astroid and comet impacts, volcanic supereruptions, gamma ray bursts, etc. These are real dangers, but they are vanishingly improbable. An asteroid 12km wide, which is how big the asteroid that killed off the non-avian dinosaurs was, only strikes Earth once every 100 million years or so! (This is actually a higher estimate than what I’ve read elsewhere.)

Hence, if there had been a Doomsday Clock in, say, 1850, it would have been set at something like 0.5 seconds after midnight on the day of doom — i.e., roughly 24 hours before the clock ticks to the dreaded moment of annihilation. Think about how wild this is. Never before in our 300,000-year history has humanity been so at risk of global catastrophe.

Was This All a Mistake?

Again, I don’t believe that human extinction — total annihilation — is actually that probable. A full-on nuclear war, ASI takeover, engineered pandemic involving synthesized designer pathogens, and so on, very likely wouldn’t kill every last person on Earth.

But, at the same time, I do think that the probability of a major, horrific, global-scale catastrophe is higher now than ever before. In 1961, just before the Cuban missile crisis that the historian Arthur Schlesinger Jr. called arguably “the most dangerous moment in human history,” John F. Kennedy said this in an addressed to the UN General Assembly:

Today, every inhabitant of this planet must contemplate the day when this planet may no longer be habitable. Every man, woman, and child lives under a nuclear sword of Damocles, hanging by the slenderest of threads, capable of being cut at any moment by accident or miscalculation or by madness. The weapons of war must be abolished before they abolish us.

The “nuclear sword of Damocles” can be generalized to many other threats, of course — those listed above. Even if some humans survive (and they almost certainly will), we live amid unprecedented levels of global risk. Trump could literally wake up tomorrow and decide to launch a nuclear missile. Or, perhaps I’m wrong and one of the AI companies succeeds in triggering recursive self-improvement next year, resulting in an ASI that does somehow — perhaps in a way we cannot possibly imagine and hence anticipate — exterminate humanity or “merely” obliterate civilization.

All of this makes me wonder whether “civilization” has been worth it. In Goliath’s Curse, Luke Kemp — an old friend of mine — points out that many of the most horrendous events in human history have been the result not of natural catastrophes but the emergence and expansion of civilizations.

Furthermore, we once lived in (a) abundant environments, where food was relatively plentiful, everyone’s teeth were almost perfectly straight, and “diseases of civilization” like “acid reflux, acne, anxiety, asthma, certain cancers, depression, type 2 diabetes, flat feet, high blood pressure, irritable bowel syndrome, lower back pain and osteoporosis” were rare or non-existent, and (b) societies that were non-hierarchical and egalitarian. The hallmark of civilizations — which Kemp dubs “Goliath’s” — is dominance hierarchies, manifest in phenomena such as slavery, patriarchy, exploitation, etc.

Yes, infant mortality was much higher among hunter-gatherers, and they didn’t have “cool” things like TV and social media. But they also didn’t have doomscrolling, nor did they perpetually live under the nuclear/etc. sword of Damocles. The Doomsday Clock would have been set at something like 12:01 am, not 85 seconds before calamity later that day.

I know asking the question, “Was civilization worth it,” is opening a can of worms! I don’t have a settled answer, and I’m curious to know what you think. Perhaps I’ll dedicate an entire essay to it in the future. As always:

Thanks for reading and I’ll see you on the other side!

First, huge congratulations to my friend Gil Duran for The Nerd Reich becoming an instant New York Times bestseller. That is an incredible achievement, made even more impressive by the fact that this is his debut book! (I’ve publish four so far, and never come close to this!)

I see it as a kind of moral duty to celebrate the extraordinary achievements of friends,1 but also, on a non-moral, purely strategic level, Gil’s immense success is good for everyone writing about Silicon Valley because the book highlights the various utopian-apocalyptic ideologies circulating among Valley dwellers. His success is a tide that will lift all other boats in the harbor.

I started reading The Nerd Reich the other day and can attest that it is absolutely fantastic. It’s a gripping page-turner that reads like a thriller. There is no fluff. Every sentence is well-crafted. Every point is terrifying. It’s honestly one of the best books I’ve read in a very long time. If you haven’t bought a copy, I’d recommend doing so right away. So much of what’s happening in US politics and Silicon Valley will start to make (more) sense.

Gil’s thesis also dovetails nicely with my own focus on the TESCREAL ideologies — an acronym he mentions throughout the book. Gil has also very graciously mentioned me and Timnit in some interviews, such as this one with Chris Hayes.

I wish I’d taken a screenshot when his book was higher on the list, but here’s proof that it’s already a smash hit:

Amazing stuff! You can — and should! — buy a copy here.

The Utter Tyranny of the ASI Race

I just published a new article in Truthdig about how the race to build artificial superintelligence (ASI) is tyrannical and fundamentally undemocratic. I also point out that the AI company CEOs — Sam Altman and Dario Amodei, in particular — are damn hypocrites for claiming to care about democracy and the democratization of AI.

I know this isn’t a particularly novel claim. The value of the article, I think/hope, is that it articulates the point in a compelling manner. I’ve copy-pasted part of it below. The full article can be read here.

Given the apparent stakes of ASI, why do we have no say in whether it’s developed? If ASI initiates the Singularity, and the post-Singularity world is fundamentally different in every way from our current world, why do we not have a seat at the decision-making table? If this technology is going to affect all of us, everywhere around the world, shouldn’t we have a voice?

A friend of mine, Monika Bielskyte, likes to say, “You cannot design for. You must always design with.” In other words, people of every nation, tribe, culture, tradition, religion, race, sex, gender, political ideology, socioeconomic demographic and so on, must be included in the design process from the very start.

It is not acceptable for a tiny elite of techno-utopian extremists in Silicon Valley to promise that at some future point, after ASI has arrived, democracy will kick in. Let’s call this what it is: a form of tyranny. The AI companies are subjecting us to a tyranny of monumental proportions by unilaterally building what they claim will be an unprecedentedly powerful technology without the permission or consent of the people.

But again, these “fuckers” — quoting Altman — don’t actually care about what we think because they’re driven by bizarre TESCREAL fantasies of uploading their minds to computers, colonizing the cosmos, and buying entire galaxies to own until the heat death of the universe. If Altman or Amodei genuinely valued democracy, they would be asking us right now whether we want the future they’re trying to realize. Of course, we don’t want that future, which is why they don’t ask us. They want to design a future for themselves rather than one with us.

This is a farce — a dystopian scenario come to life. Given that all the AI CEOs claim that superintelligence will arrive within the next few years, we may have only a small window of time to fight back. Boycott AI, join an anti-AI protest, call your representatives and tell them that this tyranny must end. There is still time to avert a bad outcome for humanity, but we must act with a sense of moral urgency.

Incidentally, an employee at Anthropic posted this just today:

Dario Amodei, the CEO of Anthropic, gives an even higher probability of total annihilation from ASI: 25%. He also says that ASI might arrive this year, meaning within the next ~3.5 months.

I know I’ve discussed this before, but I can hardly express how angry it makes me that these people — these “fuckers” — get away with threatening the lives of me, my family, my friends, and everyone else on Earth. If one takes their warnings seriously, it means that you and I, everyone we know, all the residents of Toronto, Paris, Bangkok, Casablanca, Shanghai, Caracas, Mexico City, Mumbai, etc. could all die in the next ~3.5 months.

Sorry for cursing, but how in the fuck is this acceptable? Why aren’t these people on trial for threatening humanity? Why haven’t they been arrested? Are there any lawyers reading this who have some insights about how to sue them for their rhetoric? I refuse to believe that there’s nothing to be done about this. If my neighbor says there’s a 25% chance that he might kill me and my family in the next ~3.5 months due to negligence, I’d call the cops and he’d be taken to a police station for interrogation. Surely there’s some way to hold these people accountable for their dangerous, unsettling words?

That said, I don’t think ASI will be created by the end of the year. It’s not even clear to me that ASI — whatever that is, exactly — will arrive in the next 5 or 10 years. But this is beside the point. Even if I believe my neighbor’s threat is unfounded, they should still be held accountable for uttering that threat.

The Swiss Alps

Finally, I took off the last two weeks of August to vacation with some relatives in the Swiss Alps. They generously paid for everything, which made the trip possible.

The Swiss Alps are, in my opinion, the closest thing there is to heaven on Earth. The vast pastures, the towering snowcapped mountains, the sprawling meadows and verdant forests, the dreamy ferns and mossy rocks are all glorious notes in a magnificent symphony.2

The way I’ve described the Swiss Alps to friends is this: I grew up in a horizontal world — Frederick, Maryland, 30 minutes away from the puny Appalachian Mountains.

The lucky residents of the Swiss Alps live in a vertical world. As you drive or hike up the switchbacks, you pass sprawling towns almost stacked on top of each other. Enormous mountains, you find, are actually the valleys of mountains above them, which you only discover once you reach the false summit as seen from below. There are, as I like to say, entire worlds atop worlds atop worlds. It’s like walking a circuitous stairway to heaven: at each twist in the journey, you find another, previously hidden staircase ahead of you, and looking back you see mountains and valleys layered on top of mountains and valleys. It’s really something extraordinary, and a reminder of how beautiful our precious oasis in space can be.

Below are some photos and videos from the Swiss towns of Lauterbrunnen, Adelboden, and Leukerbad. The first was visited by J.R.R. Tolkien, and inspired the town of Rivendell. I also visited Zurich at the end of the trip, staying with a good friend of mine (Ged). Zurich is frequently ranked as the best, or one of the best, cities in the world to live in, and Switzerland is consistently ranked as the best country to be born in. Enjoy:

Adelboden below. I hiked a sinuous trail that winds around the huge waterfall. It takes one to a mind-bogglingly huge cirque, i.e., “an amphitheatre-like valley formed by glacial erosion.” You cannot see the cirque from this photo — it is a hidden world. A good example of discovering a world (the cirque) towering above a world (Adelboden) towering above yet another world (the valley below Adelboden).

Lauterbrunnen, next to the accommodations in which I stayed. High up to the left is a sprawling village called Wengen, which looks down on Lauterbrunnen, but is dwarfed by huge mountains that you can’t even see from the Lauterbrunnen valley (itself far above valleys further below).

Wengen, the world over a world over a world, beneath a series of other worlds high above it.

Here’s another picture I took from Wengen when I visited back in 2021. Perhaps you can see why Tolkien was inspired by this place!

Further down the Lauterbrunnen valley:

A video of the Lauterbrunnen valley.

Expansive green pastures in Adelboden (no audio). The snowcapped mountain peaks behind the waterfall are obscured by clouds.

Another video of Lauterbrunnen (no audio):

Another video of Adelboden:

Picture of Leukerbad:

Video of the vast cliffs surrounding Leukerbad. To the right (occluded by the buildings, are snowcapped mountains).

As always:

Thanks for reading and I’ll see you on the other side!

Leftist Conspiracy Theories

On June 12, 2023, the democratic-socialist transhumanist James Hughes and then-graduate student Eli Sennesh posted an essay on Medium in which they argued that the TESCREAL thesis is little more than a leftist conspiracy theory.

This essay was written and posted before Timnit Gebru and I had published even one article on the TESCREAL concept, either together or individually. We hadn’t even given a single interview discussing the idea. Hughes and Sennesh’s entire piece was based on nothing more than a handful of tweets from me and Gebru. This would be like me writing a lengthy, detailed review of Hughes’ book, Citizen Cyborg, based solely on the title and a 200-word blurb.

Naturally, then, their critique completely misrepresents the TESCREAL thesis. Like an LLM, they hallucinated what they thought the TESCREAL thesis might be, and then launched a barrage of rhetorical missiles at the nonexistent target.

That said, the claim that TESCREAL is something of a conspiracy isn’t far off the mark. (This, however, is not what they argued.) It’s worth remembering here that not all conspiracies are conspiracy theories. In a section titled “Some Conspiracies Are Real” of her excellent book Doppelganger, Naomi Klein reminds us that the world has been shaped by “real conspiracies” hatched and executed by the socioeconomic and political elite. She writes:

The poisoned water system in Flint, Michigan, was covered up by state officials year after year. British Petroleum and Halliburton cut corners in the operation of the Deepwater Horizon offshore drilling rig, and the result was the largest accidental oil spill in history in the Gulf of Mexico, with the companies scrambling to cover up the full extent of the damage. Volkswagen ran a conspiracy for years to cover up how much polluting carbon its diesel vehicles were emitting (the cars were programmed to fool the testers). Most consequentially, Exxon and several other oil majors ran a conspiracy to spread doubt and confusion about the reality of climate change for decades, taking a page from the tobacco giants. These are just the barest of examples.

A Dish of Spaghetti Thrown at the Wall

The fact is that the TESCREAL movement is an incestuous tangle of researchers, activists, policymakers, wealthy donors, tech CEOs, think tanks, organizations, institutes, and AI companies all driven by a common eschatological belief system: we must build a Machine God, create a new posthuman species, colonize the universe, and establish a sprawling multigalactic civilization high up on the Kardashev scale.

There is a sense in which this is a conspiracy. The people sharing this belief system all know each other. They’ve all hung out at conferences and private parties. They’ve exchanged ideas on the same email lists. They run the major AI companies, as well as the “AI safety” organizations trying to stop those AI companies.

Even those in the community who appear to dislike each other, such as Eliezer Yudkowsky and Dario Amodei, have extensive personal histories, and despite their disagreements about ASI existential risk, both Yudkowsky and Amodei are trying to realize the very same cosmic “utopia” through the creation of artificial superintelligence (ASI).

To make this more concrete, let’s zoom in on a tiny segment of the social network that comprises the TESCREAL movement, focusing on names like:

• Shane Legg

• Ben Goertzel

• Eliezer Yudkowsky

• Demis Hassabis

• Dario Amodei

• Leopold von Aschenbrenner

• Holden Karnofsky

• William MacAskill

• Nick Bostrom

• Ray Kurzweil

• Elon Musk, and

• Jaan Tallinn

This will get messy, but the messiness is part of the point. I feel confident in saying that you’ll want to read this entire section carefully, maybe twice. It gets more outrageous the further along you read.

***

We begin with Shane Legg, whose 2008 PhD dissertation was titled “Machine Super Intelligence.” For this, he received a $10,000 award from the Canadian Singularity Institute for Artificial Intelligence. That’s the (now-defunct) Canadian branch of Eliezer Yudkowsky’s Singularity Institute, founded in 2000 and later rebranded as the Machine Intelligence Research Institute (MIRI).

From 1999 to 2001, Legg worked for Ben Goertzel’s company WebMind. In the mid-2000s, Goertzel co-edited a book about human-level AI that included a contribution from Yudkowsky, among others. The original title was Real AI, but Goertzel wanted something better. So he asked around for suggestions, and Legg proposed “artificial general intelligence” (AGI, the precursor of ASI). This is how the term “AGI” became part of our shared lexicon, although “ASI” has become more popular among AI hypesters over the past year or so.

Three years after Artificial General Intelligence (2007) was published, Legg cofounded DeepMind with Demis Hassabis, a fellow transhumanist and singularitarian. DeepMind’s explicit goal was the creation of a Machine God. Like many in the ASI race, Legg first became interested in superintelligence after reading one of Ray Kurzweil’s books, though he reports that Yudkowsky got him “obsessed” with ASI. He even exchanged emails with Yudkowsky via the SL4 mailing list, which Yudkowsky created in 2000 as “a refuge for discussion of advanced topics in transhumanism and the Singularity.”

Before founding SingularityNET, Goertzel was an active member of the Extropian movement of the 1990s, along with Yudkowsky, Kurzweil, and Nick Bostrom. At various Singularity Summit events, he shared the stage with these people as well as his former employee Legg and Legg’s DeepMind cofounder, Hassabis (more on this below). The Singularity Summit was an annual conference cofounded by Yudkowsky, Kurzweil, and Peter Thiel.

In 2008, Goertzel was hired as the “Director of Research” at Yudkowsky’s Singularity Institute (now MIRI). The following year, he convinced Jeffrey Epstein to donate $50k to the organization (this money being passed on to another research group, I believe). In fact, Epstein had been financially supporting Goertzel since 2001.1

Yudkowsky was one of the most prominent singularitarians of the early 2000s, alongside Kurzweil. He founded the Singularity Institute (MIRI) to accelerate the techno-rapture and what he frequently calls our “glorious transhumanist future.” He also participated heavily in the Extropian movement, founded Rationalism, and has been highly influential within the Effective Altruist (EA) and longtermist communities. (William MacAskill once described him as a “moral weirdo,” which is among the highest praise one could get from a leading EA.)

The Singularity Institute was initially funded by Thiel ($1,627,000). It also received money from Vitalik Buterin, Jaan Tallinn, OpenPhil (i.e., Open Philanthropy, now called Coefficient Giving), the last of which distributes money from Facebook cofounder Dustin Moskovitz, a hardcore EA-longtermist. OpenPhil has employed Nick Beckstead, Holden Karnofsky, and Luke Muehlhauser, who was previously the Executive Director of Yudkowsky’s MIRI and subsequently became a Board Member at Anthropic.

Beckstead, one of the earliest members of the EA movement,2 cofounded longtermism along with Nick Bostrom, and was a Research Fellow at Bostrom’s Future of Humanity Institute (FHI). After leaving OpenPhil, Beckstead became the CEO of Sam Bankman-Fried’s FTX Future Fund, co-run by Leopold Aschenbrenner, Amitav Balwit, and William MacAskill. (Note that Bankman-Fried was once the most prominent EA-longtermist in the world.3)

The FTX Future Fund had a close relationship with FHI, both sharing the same EA-longtermist ideology. As an article in Asterisk Magazine notes,

for a single, delirious summer, it seemed that every other person at Trajan House [where FHI was located], including FHI staff, was to-ing and fro-ing between Oxford and the Bahamas, where Sam Bankman-Fried was planning a barrage of longtermist philanthropy.

According to the New York Times, “Bankman-Fried and his colleagues had fed at least $500 million into Anthropic” (which we’ll get to in a moment), and the FTX Future Fund offered Nate Soares, a MIRI employee who coauthored a recent book with Yudkowsky, $6k for advice on regranting. Another Bankman-Fried run company, Alameda Research, donated over $113k to MIRI, as well as another $19k to an organization “housed” within MIRI called AI Impacts.

In addition to working with Beckstead at the FTX Future Fund, Aschenbrenner was also an intern at Bostrom’s FHI and its sibling organization at Oxford, the Global Priorities Institute, funded by OpenPhil and cofounded by William MacAskill — an EA-longtermist and transhumanist who was the moral “advisor” of Bankman-Fried.4 (Whoops!) Aschenbrenner subsequently worked at OpenAI as part of the “Superalignment” team, co-led by Ilya Sutskever.

The CEO of OpenAI, Sam Altman, has been pictured partying with Yudkowsky and, like Shane Legg, explicitly credits Yudkowsky with inspiring him to pursue ASI. The New York Times describes him in a profile as being a “product” of the EA (meaning, in this context, longtermist) and Rationalist movements.

In 2024, OpenAI fired Aschenbrenner for sharing sensitive material from OpenAI with Holden Karnofsky, who left OpenPhil in 2024 (as Director of AI Strategy) and joined Anthropic in 2025. In his 2022 book on longtermism, MacAskill credits Karnofsky with having “a particularly profound impact on my broader thinking about longtermism.”

Following in the footsteps of Bankman-Fried, Aschenbrenner pursued the “earn to give” career path invented by EA. He started a hedge fund called Situational Awareness, which imploded this summer, losing $35 billion almost overnight.

This was just days before Aschenbrenner’s wedding to Avital Balwit, the former FTX Future Fund employee who, like her now-husband, previously worked at Bostrom’s FHI and is currently the Chief of Staff to Dario Amodei, Anthropic’s CEO. In attendance at the wedding were Amodei and “more than a dozen Anthropic employees … as well as MacAskill and others from the FTX Future Fund, plus effective altruists and the billionaires who had put their money into Situational Awareness.”

Alongside Beckstead and Karnofsky (who now works at Anthropic), Dario Amodei was one of the earliest members of the EA movement. According to the New York Times,

all of the major AI labs and safety research organizations contain some trace of effective altruism’s influence, and many count believers among their staff members. … No major AI lab embodies the EA ethos as fully as Anthropic. Many of the company’s early hires were effective altruists, and much of its start-up funding came from wealthy EA-affiliated tech executives.

Like Legg, Dario Amodei cites Kurzweil as having piqued his interest in ASI. He’s also shared the stage with Yudkowsky in discussing the promises and perils of ASI, and he spent years living with Karnofsky (they were roommates). Indeed, Karnofsky would later marry Dario’s sister, Daniella, who cofounded Anthropic with her brother after both quit working for OpenAI.

In addition to money from the FTX Future Fund, Anthropic has been funded by OpenPhil (where Karnofsky worked) and Jaan Tallinn (one of the most prolific funders of the TESCREAL movement).

Another noted figure at Anthropic is the utilitarian longtermist Amanda Askell, the former wife of William MacAskill, who was involved in the early EA movement and received her undergraduate degree at Oxford, where FHI and the Global Priorities Institute were based. (MacAskill changed his surname from “Crouch” upon marrying Amanda.)

In 2017, while still an employee at OpenAI, Dario Amodei joined a conference panel on ASI with Nate Soares (of MIRI, also Yudkowsky’s coauthor) and Shane Legg (of the Thiel-funded DeepMind). The conference was hosted by the Future of Life Institute (FLI), which was cofounded by Tallinn and Max Tegmark, the latter of whom has collaborated with Bostrom. Musk and Bostrom were both external advisors at FLI.

Also present at the conference — sponsored by Musk, Tallinn, and OpenPhil — was:

• Musk (who has promoted the work of Bostrom and MacAskill, even calling longtermism “a close match for my philosophy”)

• Eliezer Yudkowsky

• Holden Karnofsky

• Ray Kurzweil

• William MacAskill

• Nick Bostrom

• Max Tegmark

• Jaan Tallinn

• Demis Hassabis

• Sam Altman

• Paul Christiano (who was also roommates with Karnofsky and Amodei)

• Jason Matheny (a former FHI researcher who went on to run IARPA and is now the CEO of RAND Corporation)

• Ilya Sutskever (a noted figure at OpenAI until he quit — Sutskever has parroted Yudkowsky in arguing that ASI could kill everyone on Earth. Recall that Aschenbrenner worked under Sutskever on the “Superalignment” team)

• Helen Toner (a hardcore EA-longtermist who served on OpenAI’s board and voted to fire Altman in 2023)

• Larry Page (a former close friend of Musk’s who personally hired Kurzweil to work at Google)

• Eric Schmidt

• Reid Hoffman

• Vitalik Buterin (recipient of the Thiel Fellowship who donated $665 million to FLI and once participated in an FHI conference)

Going back in time a bit: Seven years before the FLI conference, Hassabis and Legg both gave talks at the 2010 Singularity Summit alongside Goertzel, Yudkowsky, and Kurzweil. (In fact, Legg and Yudkowsky were on the same panel.) Recall that Yudkowsky, Kurzweil, and Thiel founded the summit.

After the summit ended, Hassabis followed Thiel back to his mansion and asked for money to start DeepMind, which Thiel gave him. Hassabis later delivered a talk at the 2011 Winter Intelligence conference hosted by Bostrom’s FHI (image below), and hired Bostrom to be a member of DeepMind’s Ethics & Society research unit.

Taking a closer look at Bostrom, who constitutes a central node in the TESCREAL network: Alongside Yudkowsky, Goertzel, and Kurzweil, he was an early member of the Extropian movement, and he’s become one of the most influential transhumanists today. His 2014 book on the dangers of ASI was blurbed by Musk, and — as noted — he cofounded the longtermist ideology with Beckstead (of FHI, FTX, and OpenPhil).

Bostrom has also been a major figure in Yudkowsky’s Rationalist and MacAskill’s EA movements. He’s coauthored papers with Yudkowsky, and many of his “original” ideas about ASI came from the Rationalist founder. (As Goertzel once wrote, “scratch the surface of Bostrom, find Yudkowsky.”)

Before it was shut down — thanks in part to a racist email from Bostrom that I uncovered in late 2022 — FHI shared office space in Oxford’s Trajan House with the Centre for Effective Altruism, cofounded by MacAskill, who also worked for Bostrom at FHI and, once again, cofounded the Global Priorities Institute that hired Aschenbrenner. In 2015, Musk donated $10 million to FLI, which funneled some of these riches to FHI.

Recall that FHI had close connections with FTX, run by MacAskill’s advisee, Bankman-Fried, who’s now in federal prison. It also had connections with OpenAI. (Altman once described Bostrom’s 2014 book as “the best thing I’ve seen on this topic.”) In 2018, a group of FHI researchers coauthored a paper about AI risk with OpenAI.5 Notable names on the paper include Helen Toner, Dario Amodei (still at OpenAI, along with his sister, Karnofsky’s wife), and Carrick Flynn.

As it happens, Flynn was a former FHI researcher who ran for congress several years ago in Oregon. His campaign boasted of more than $10 million from Bankman-Fried, which “is considered an astronomical figure for a House Democratic primary.” His campaign manager? None other than Balwit, the former employee under Beckstead at the FTX Future Fund, who now works for Dario Amodei and is married to Aschenbrenner.

A Conspiratorial Afterimage

Believe me when I tell you that this is a tiny, tiny glimpse at how incestuous the ASI race — driven by the TESCREAL movement — is. An exhaustive account would require, I’d say, 100 pages of dense reading like that above.

If one were to compile such a document, you’d find the same people, think tanks, companies, etc. popping up over and over again. As mentioned earlier, these people all know each other. They’re all part of the same fringe eschatological movement (though that movement is hugely powerful in Silicon Valley, and hence the world more generally) with the shared goal of building a Digital Deity and conquering the cosmos as radically transformed digital space brains. Even a cursory glance at this network leaves a conspiratorial afterimage, like the floating dots of colors you see after looking at a bright object and then closing your eyes.

Indeed, it’s worth adding briefly that many notable media figures have connections to this movement, including Ezra Klein, Sam Harris, and Kelsey Piper. The last of these knew Bankman-Fried, is friends with Yudkowsky, and writes for the Vox vertical Future Perfect, which was funded by FTX. Even more striking is the fact that Ross Douthat, formerly of the New York Times and now a correspondent at CBS’s 60 Minutes, officiated Aschenbrenner’s wedding. (Side note: the New York Post reports that Aschenbrenner promised “his future wife he would someday buy her not a yacht or a mansion, but a galaxy.” Lolz.)

Here’s how Wikipedia defines a “conspiracy”:

[A] plan or agreement between people … for an unlawful or harmful purpose, such as murder, treason, or corruption, especially with a political motivation, while keeping their agreement secret from the public or from other people affected by it. In a political sense, conspiracy refers to a group of people united in the goal of subverting established political power structures. This can take the form of usurping or altering them, or even continually illegally profiteering from certain activities in a way that weakens the establishment with help from various political authorities.

I don’t think the TESCREAL movement, which admits of many minor variations, is cohesive enough to justify calling it a “conspiracy” without significant qualifications. But it is conspiracy-like. The difference between a real conspiracy and a conspiracy theory is, of course, epistemological: it depends on the totality of the available evidence. As Naomi Klein implies, there is ample evidence to justify describing Big Oil’s efforts to mislead the public about climate change as a conspiracy. But there isn’t any good evidence (!) to support the claim that commercial airplanes are spewing biological or chemical agents into the atmosphere, or that 9/11 was an inside job.

If one seriously examines the cluster-fuck of connections within the TESCREAL movement, and between the TESCREAL movement and the leading AI companies, it becomes clear that calling it “conspiracy-like” is epistemologically warranted. Or so I would argue.

Subscribe now

But what do you think? What am I wrong about? As always:

Thanks for reading and I’ll see you on the other side!

Since I’m technically on vacation right now (until September 1), I thought I’d publish a relatively short piece on a phrase I hear a lot these days: “the danger of the race to build ASI” (artificial superintelligence). It’s worth noting, I think, that this phrase is ambiguous. It can mean two radically different things.

On the one hand, people in the field of AI safety — many of whom are “doomers”1 — believe the ASI race is dangerous because it will result in ASI. Specifically, they believe it will likely result in misaligned ASI unless significant steps are taken to ensure that ASI is “value-aligned” (by which they typically mean “aligned with the TESCREAL worldview”). Many of these people will add that ASI is imminent, and hence that everyone on Earth will die in the very near future.

On the other hand, when people like me talk about “the dangers of the ASI race,” we typically mean that trying to build ASI itself poses significant threats to individuals, democratic societies, and perhaps even humanity itself. Many of us don’t believe that ASI is imminent — if it’s even possible to build, assuming the term is meaningful in the first place. But this is beside the point. The fact is that rushing toward what the AI companies call “ASI” has inflicted, and will continue to inflict, serious harms on society and the individuals who comprise it.

Hence, worrying about the dangers of the ASI race doesn’t commit one to a belief that ASI will arrive anytime soon, if ever. By analogy, you could imagine multiple companies racing to build a perpetual motion machine (PMM). They pour huge amounts of money into their respective projects, polluting the environment, exacerbating climate change, destroying communities, etc. in the process. Despite the fact that PMMs are physically impossible, according to the laws of thermodynamics, one could still talk about “the dangers of the race to PMM” — an imaginary end-goal that these companies will never reach.

The same goes for the ASI race. I think this is important to note because referring to “the dangers of the ASI race” could be misunderstood as suggesting that one believes ASI is possible and imminent. One interpretation of the critiques of scholars like Emily Bender, Alex Hanna, Timnit Gebru, etc. is that they are deeply concerned about the ASI race — that is, it’s deleterious consequences — despite them believing (as I understand their views) that ASI isn’t even a coherent concept.

Whenever you hear someone talking about the ASI race being dangerous, it’s worth taking a minute to decipher what they mean. Are they referring to the race itself, or the anticipated outcome of this race?

AI Ethics vs. AI Safety

This ties into another issue worth mentioning, I think. It concerns the distinction between AI ethics and AI safety. One often hears this distinction elucidated as follows: AI ethics focuses on near-term risks whereas AI safety focuses on long-term risks.2

But this is misleading. For example, AI ethicists worry about the environmental consequences of current AI systems — seen by the AI companies as the stepping-stones to ASI. (Hence, to build ASI, we must first build systems like ChatGPT-5, Claude Mythos, etc.) AI has a nontrivial carbon footprint, and indeed the CO2 emissions of companies like Google and Microsoft have risen — despite promises to be carbon-neutral or carbon-negative by 2030 — due almost entirely to AI.

But, of course, climate change is a long-term risk. The CO2 injected into the atmosphere right now will linger for millennia, affecting the livability of Earth for the next 10,000 years — a longer period of time than civilization has so far existed.3

On the flip side, many AI safetyists and doomers believe that ASI could arrive in the next few years, and that, since we don’t yet know how to “align” it with our (i.e., the TESCREAL worldview’s) values, it will annihilate humanity by default. This is clearly a near-term risk, as it would mean the untimely death of you and me, our children and parents, and everyone else alive a few years from now.

A better way of understanding the difference between AI ethics and AI safety is in terms of the distinction made above. Those in the first camp are mostly concerned with the externalities of attempting to build ASI, while those in the second tend to ignore or minimize the inimical effects of the race itself, focusing instead on the “existential” stakes of actually building a superintelligent machine.

Herein lies the most salient difference between these two fields.

My Current View

What are my views on the matter? I mentioned above that my primary concern is the harms caused by trying to build ASI. Such harms include:

• exploiting workers in the Global South,

• stealing our intellectual property,

• exacerbating climate change,

• sucking up massive amounts of energy,

• flooding the Internet with slop, disinformation, and deepfakes,

• inducing episodes of psychosis in vulnerable populations,

• perpetuating harmful stereotypes due to algorithmic bias,

• enabling unprecedented levels of mass surveillance,

• working with the US military,

• and so on.

I see no reason for believing that ASI is imminent, and indeed the TESCREAL pro-extinctionist Sam Altman recently acknowledged that the hype around ASI has been overblown. Altman, of course, has been one of the biggest hypesters out there, even arguing last year that the Singularity has already started.

I’m not even sure that the term “ASI” is meaningful, as no one can agree on a definition and the concept of intelligence itself is poorly defined. (To a great extent, our contemporary notion of intelligence has been influenced by the eugenics movement, which has morphed into the TESCREAL movement. Hence, Gebru and I characterize TESCREALism as the most recent iteration of the “eternal return of eugenics.”)

That said, it also seems clear to me — I think — that if the AI companies were to build a system that could outmaneuver us in every important respect, solve complex problems much faster than we can, set and modify its own goals without human intervention, and act as autonomous agents in the world, then we’re in very big trouble. I don’t see any good reason for thinking that our species would survive for very long competing against an artificial system of this sort. You can call that “ASI,” but you don’t need to, to grasp the danger.

Hence, if the ASI race succeeds in building an “ASI,” the outcome would likely be terrible. (Even the “good” outcomes would be terrible, as these would also inevitably lead to the extermination of our species.) But I don’t think the ASI race will “succeed,” at least not anytime soon, and thus when I talk about “the dangers of the ASI race,” I mean “the dangers of trying to build this thing called ‘ASI,’ whatever that is exactly.”

But what do you think? What am I missing? Is this distinction useful? (It is in my own mind!) As always:

Thanks for reading and I’ll see you on the other side!

Summary1:

A woman named Cami Clark, who hung out with EA-longtermists and married one of the most powerful EA-longtermists in the world (Dario Amodei), exchanged emails with Jeffrey Epstein after his 2008 conviction in hopes of getting him to fund her porn company, whose tagline was: “intellectually promiscuous.”

Another leading EA-longtermist, Leopold Aschenbrenner, who worked for Bankman-Fried’s FTX Future Fund and married a former colleague at the Future Fund, Avital Balwit — who now works for Clark’s husband, Amodei — started a hedge fund that invested heavily in Anthropic and recently lost $35 billion of its $45 billion AUM.

Lol.

Dario Amodei’s Wife Tried to Get Jeffrey Epstein to Fund Her “Luxury Porn” Company

Last February, I wrote a Truthdig article about Jeffrey Epstein’s extensive connections to the transhumanist community. Epstein himself was a transhumanist and eugenicist (but I repeat myself) who wanted to have his head and penis cryogenically frozen and aimed to create a “super-race of humans with his DNA” by impregnating up to 20 women at a time on his Zorro Ranch.

He was in contact with Bryan Johnson, and appears to have met Aubry de Grey (the longevity researcher) over Skype. He was also pals with Peter Thiel and Peter Diamandis, and exchanged emails with Elon Musk, who practically begged Epstein to let him visit the pedophile and sex trafficker’s home-base in the Virgin Islands.

Epstein was also long-time buddies with Ben Goertzel, who congratulated Epstein when he emerged from prison in 2009, convinced Epstein to transfer $50,000 to Eliezer Yudkowsky’s Singularity Institute (now the Machine Intelligence Research Institute), and wrote the following to Epstein after sending him birthday wishes:

Now if you want to have 100000 more birthdays you should fund my AI project so we can create a thinking machine that will solve the problem of mortality!!!

These are just a few examples of Epstein’s connections to the TESCREAL movement.

***

It turns out that Epstein was also friendly with Cami Clark, the wife of Anthropic CEO Dario Amodei. Previously, Clark had dated Eric Schmidt, the former Google CEO who claimed we should build ASI (artificial superintelligence) despite the environmental costs. She married Amodei in 2022 (having been married once before to an architect who was 64 years old while she was just 20).

As Forbes reports:

• Her previous attempted businesses include founding a self-described “free luxury porn” company that catered to women and a “social dieting app.”

• Clark met Epstein in March 2011 through John Brockman, a literary agent who knew Epstein, with emails in the Epstein files showing Brockman telling Epstein he “should connect for dinner with my girls,” referring to Clark and her business partner Michelle Capocefalo. [The full quote is: “You should connect for dinner with my girls — Cami and Michelle. They’re in LA raising money for porn movie aimed at women’s market.”]

• “We would love to have dinner with you this evening,” Cami said in an email responding to Brockman’s introduction, then sending an email the next day that passed along her company’s funding requirements and exclaimed, “It was FANTASTIC meeting you last night!”

• Clark then asked Epstein in a March 2012 email exchange about whether he’d be interested in investing in her and a colleague’s businesses, though there’s no indication in the Epstein files that the financier made any investment.

The porn company was apparently called Eddice. It “described itself as a ‘revolutionary porn company’” that emphasized “sex positivity in a male-oriented porn industry, and carried the tagline: ‘intellectually promiscuous.’”

All of this happened after Epstein pled guilty to soliciting prostitution from a minor and spent 13 months in prison — out of the 18 months he was sentenced to in a “sweetheart deal.” Upon being released, he almost immediately began raping an 18-year-old woman who had “recently arrived from Uzbekistan.”

As with Noam Chomsky, who became close friends with Epstein, Clark appears not to have cared that Epstein was a registered sex offender.

Forbes continues:

After their initial introduction through Brockman, Clark sent Epstein the script and treatment for her company’s first films a few days later in March 2011, noting she thought Epstein “and the ladies” might enjoy them and saying the content was “a little nsfw.” Clark’s March 2012 exchange with Epstein is the lengthiest conversation between them that appears in the Epstein files, with Epstein initially asking, “i’m sorry, who is this,” after Clark reached out and asked to put Epstein in touch with another female entrepreneur launching a lingerie line. When Clark reminded him that she has the “free luxury porn company” and asked if that “ring[s] a bell,” Epstein responded, “yes, a loud gong.” (“I was going to say, you would be the first person ever that didn’t remember us :),” Clark responded.) … Epstein then asked what happened to Clark’s business plans with the porn company and she said they were moving “full speed ahead,” pitching Epstein on investing in her company’s foray into video. “Can’t do TV sex,” Epstein responded, to which Clark said, “I didn’t think so :).”

Forbes adds that “other emails in the Epstein files show Clark keeping the financier on her email list — inviting him to a housewarming party in July 2012 — and that Epstein connected with her on LinkedIn a year later in 2013.”

***

Clark has almost no online footprint. She keeps a low profile, but appears to be influential behind the scenes, even playing a role in getting Anthropic early funding. As the Wall Street Journal, which broke the story, observes:

When Indian Prime Minister Narendra Modi invited AI leaders to a meeting in New Delhi earlier this year, security protocols allowed each executive to bring one additional person with them. Most brought colleagues, but Anthropic CEO Dario Amodei brought his wife, Cami Clark.

Clark doesn’t work at Anthropic, but she is often seen sitting in the front row while Amodei talks at events such as Davos or can be found chatting up investors at gatherings such as the Allen & Co. conference in Sun Valley.

She acts as a sounding board and strategic adviser for Amodei, according to people close to the company. She also brought Anthropic a key early investor, former Google CEO Eric Schmidt — whom she had dated — as it was getting off the ground in early 2021, some of the people said.

WSJ also notes that

Clark spent time at the group house in San Francisco where Amodei, his friend Holden Karnofsky and his sister, Daniela — who would later marry Karnofsky — lived with other key figures from the AI safety and Effective Altruism worlds. The movement, centered largely in Silicon Valley, aims to use data and reason to determine how to do the most good possible.

And:

Clark has recently told people involved in politics that the company sees its mission as protecting America and isn’t as “woke” as its detractors might think it is, according to people familiar with the matter.

At the Sun Valley conference in July, where she spent much of the week by her husband’s side, she had lunch with Ivanka Trump, a friend, and chatted with Jared Kushner, whom Amodei had approached earlier in the year to invest, people familiar with the matter said.

She strikes me as a hyper-ambitious person with low moral standards, which of course describes many power-hungry vultures in Silicon Valley — including her husband. I wonder who else in the TESCREAL/ASI race has had connections with Epstein that haven’t yet been reported.

Womp Womp: Situational Awareness Loses $35 Billion

Leopold von Aschenbrenner — a German-born investor and AI safety researcher whose name I jokingly add the aristocratic “von” to — is an Effective Altruist (EA) and longtermist who gained a bit of Silicon Valley fame for his wildly hyperbolic 2024 essay, “Situational Awareness.” In it, he hyperventilates about ASI being imminent and argues that it’s very important for the US to win the ASI race against China.

Here’s what I write about his essay in my forthcoming book on the TESCREAL ideologies:

Aschenbrenner’s “impressive” CV includes a stint at Sam Bankman-Fried’s cryptocurrency exchange platform, FTX. Specifically, he worked for the FTX Future Fund, run by Nick Beckstead — the longtermist who argued that we should prioritize saving the lives of people in rich countries. The Future Fund also gave $500 million in funding to Anthropic.

While still a teenager, he wrote a paper titled “Existential Risk and Growth,” and cofounded the Effective Altruism club at Columbia University when he was a student there. He spent time at the Future of Humanity Institute (before it was shut down due in part to a racist email from it’s founder that I uncovered in late 2022), as well as Will MacAskill’s Global Priorities Institute. He later joined OpenAI’s “Superalignment” team, but left after OpenAI accused him of sharing confidential documents (which von Aschenbrenner disputes).

Following his viral-in-the-Valley 2024 essay, he founded a hedge fund of the same name: Situational Awareness, which he describes as “an investment firm focused on AGI.” At its peak, Situational Awareness had $45 billion assets under management (AUM), but last month it lost a catastrophically huge $35 billion. That still leaves an eye-watering $10 billion behind, but the massive drop in assets reminded many of the ignominious collapse of Aschenbrenner’s former employer: Bankman-Fried.

As CNBC reports,

critics pointed out that Aschenbrenner had no experience running money prior to launching his fund in July 2024, calling him more lucky than smart. Some noted that his early work experience was at the doomed crypto firm FTX, where he helped now-disgraced founder Sam Bankman-Fried run a charity out of a Bahamas penthouse.

“A lot of people saw this blow-up as a matter of not if, but when,” said Jerry Diao, who runs a Wall Street coaching firm. “Maybe his views on AI are correct in the long run, but in the public markets, you have to be prepared for the short-term.”

The same article opens with the rather sardonic line: “Two years ago, Leopold Aschenbrenner argued he was one of few people in the world who saw the future clearly.” Now, von Aschenbrenner has made himself the butt of jokes and thrust EA into the spotlight once again. (Aschenbrenner was essentially earning to give, the very same career path that led Bankman-Fried into crypto.)

As my friend writes in a quite amusing article, “Son of Sam: Aschenbrenner was SBF 2.0”:

An infamous Effective Altruist thought experiment is “the coin flip to end the world.” Sam Bankman-Fried interpreted this as an injunction to take catastrophic risk if the average “expected value” of winning a long bet was even slightly positive.

That high-risk mindset also seems to have ruled at Situational Awareness, a huge, AI-focused hedge fund that exploded in blood and guts last week. It was run by a deeply involved Effective Altruist named Leo Aschenbrenner.

Aschenbrenner’s downfall very closely tracks that of Sam Bankman-Fried. Specifically in that they were both young, loved using very high leverage in volatile markets, believed trend-lines only go up, and apparently thought risk management is for cowards.

Morris continues:

Aschenbrenner still got caught by the exact same ratchet that killed SBF.

Situational Awareness LP grew incredibly fast after 2024, to a peak of $45 billion in AUM, by using high leverage to bet on the continued rapid growth of AI. Aschenbrenner had kept making the same bet for years, at larger and larger size, to the point where his leverage may have been propelling the broader market.

Then July was choppy for AI stocks — incredibly, even $Nvidia was flat - and that was all it took to destroy Situational Awareness. …

[T]he linear, predictionist modelling that rules EA, alongside the idea that its adherents are uniquely genetically gifted, lead believers to take on extreme risk and, inevitably, to blowups like Aschenbrenner’s.

He further notes that, almost comically,

Aschenbrenner’s unluckiest investors lost something like 110% of the current mark-to-market value of their assets.

That means Leo Aschenbrenner was far, far worse for many investors than Sam Bankman-Fried was for FTX depositors.

***

Of note is that von Aschenbrenner’s wedding to Avital Balwit happened just weeks after the $35 billion loss. Balwit is Dario Amodei’s chief of staff at Anthropic and a former employe at Bankman-Fried’s FTX Future Fund — just like her husband was. Note also that Situational Awareness had a multi-billion-dollar investment in Anthropic. As Morris observes, “the largest remaining asset in Situational Awareness LP portfolio is reportedly a large stake of illiquid Anthropic equity.” (This community is so incestuous!)

Balwit, I should add, wrote a long response back in 2021 to a couple of articles I published critiquing longtermism — an ideology that both she and her husband obviously subscribe to. Her critique of my critique is pretty abysmal, spurring me to write a response in which I dissect each paragraph, showing how far her criticisms miss the mark. I had some personal communication with her because I sent her my response prior to publication for comments. (A courtesy I always extended to EAs and longtermists, though they never reciprocated.)

Anyways, the whole von Aschenbrenner debacle is rather hilarious. As I wrote years ago, EA is the gift that keeps giving to critics.

As always:

Thanks for reading and I’ll see you on the other side!

PS. In case you need a chuckle, I saw this on Facebook the other day. :-)

CORRECTION: In the original post, I accidentally wrote “$45 billion” on a few occasions instead of “$35 billion,” which is how much Situational Awareness lost. My apology for the error!

The following piece is far longer than usual. It will be most relevant to those with a strong interest in the TESCREAL thesis. The footnotes, in particular, contain a wealth of information about how I’ve outlined and elucidated this thesis over the years. A shorter, more accessible article will be published tomorrow. Thanks so much, friends!

tl;dr

1. Connor Leahy offers an intellectual history of the Silicon Valley worldview that is strikingly similar to the history outlined by Timnit Gebru and myself.

2. He highlights the TESREAL (no “C”) ideologies as having played an integral role in launching the race to build ASI (artificial superintelligence), whereas we highlight the TESCREAL ideologies.

3. In copious footnotes, I show that many of Leahy’s claims are nearly identical to claims I’ve made in articles and interviews over the past many years.

4. This cannot be a coincidence, since it’s impossible to work in the space of AI and not have come across my work (solo and with Gebru) on the TESCREAL bundle.

Very, very few people have told this story in any good sense. I would love for more people to tell this story ‘cause it’s [a] fucking crazy story, man. Oh man. So the story as we have it today with, like, you know, this, like, ChatGPT and AGI and ASI and AI risk. All of this stuff descends from this one weird offshoot 1980s group of futurists called the Extropians.

— Connor Leahy, November 6, 2025.

Connor Leahy is the cofounder of Conjecture, an AI safety organization at which he was CEO until this year, and he cofounded a nonprofit AI research company called EleutherAI. He is currently the US Director at ControlAI, which is seeking $50 million in annual funding. According to Wikipedia, “he has testified before the House of Commons of Canada and participated in the inaugural AI Safety Summit in 2023.” He has appeared multiple times on CNN, as well as CBS, ABC News (Australia), Al Jazeera English, BBC, and countless popular podcasts, stacking up millions of views.

He mentions in many interviews that he speaks to policymakers in DC, where he’s based, about AI existential risk. If you read or write about AI, there’s no way you’ve missed him. He has become one of the leading voices warning about the annihilatory dangers of misaligned ASI (artificial superintelligence).

Recently, he gave an interview on the Onezero podcast hosted by Robert Wright, whose work I admire. It’s titled “The Ideology Behind AI,” which happens to be a primary focus of my research over the past many years. I found this interview to be deeply problematic, and am inclined to describe Leahy’s conduct as unethical.

But first, it will be useful to provide an overview of Leahy’s basic argument. Take note, if you’d like, of the copious footnotes pointing to articles, interviews, and tweets in which I have made virtually identical claims. My apologies for this overview being rather long (~2,000 words), but I believe this is necessary to make my point. Here goes:

Leahy’s Analysis

Note that I have slightly edited the quotes below to improve readability (e.g., removing words such as “like” and “you know”). The meaning remains intact.

“There’s a lot of really fascinating history” with respect to the “milieu” or “scene” out of which the ASI race emerged, Leahy says. The goal of creating God-like machines “didn’t come out of a vacuum. This is not the ruthless market decided [sic] this is the optimal thing to go for. These are ideas that actually grew out of a very specific ideological lineage that I would say really started with what is called the Extropians.”1

Extropianism, Leahy says, is a “particularly intense form of what’s called transhumanism more broadly. Transhumanism is a very, very broad set of philosophies that in various ways are about belief that humanity should be improved or transcended in various ways,” in particular, through technology that enables us to reengineer humanity.2

He says that “San Francisco as a place is, and Silicon Valley is, a really weird place.3 And it really has, I would say, three main ideological strengths that kind of all mix and mutate in this stew. And these three are the transhumanist, techno-utopian kind of visions4 and also coming from golden-age sci-fi, and stuff like that, which we’re talking about. The other one is just cutthroat capitalism — libertarianism. And the third is true new age communist, you know, crazy hippie ideology. All of these are core to the soul of Silicon Valley.5 And if you neglect any one of them, you won’t understand what’s actually going on. So, all of these cultures of all these [AI] companies … should be understand [sic] as being influenced by these three large ideological clusters to varying degrees.”6

“So,” he continues, “this is where I think the origin of this culture kind of comes from. And then the modern version of what you now see in many of these, like, AGI religions or cults almost7 is coming from the Extropians,8 who are a particularly intense form of transhumanism that kind of emerged in the 80s by a guy named Max More — I think was the one who first coined the term.9 Extropianism was all about exponential growth.10 It was this idea that we could get exponential amounts of energy and growth and intelligence, and improve everything at unbelievable levels …”

He adds that Extropianism “was one of the first ones to really seriously grapple with some very intense ideas such as whole-brain uploading, immortality, and AGI … and superintelligence as concepts.11 They were some of the first people to really think seriously about some of these ideas in the 80s and the 90s.”

Leahy describes how Eliezer Yudkowsky befriended Max More and his wife, Natasha Vita-More, and how he “was someone who was very early on the Extropian mailing list, and so on, talking about these ideas of superintelligence. They didn’t exactly call it that back then, but that’s what they were talking about: this idea of, if you had this recursively self-improving AI that could improve itself, you could … build these incredibly powerful superintelligent beings that could cure all diseases, could solve all problems, could turn the whole universe into … nanobots, or whatever — like, really intense sci-fi stuff.”12

Leahy then notes that the SL4 mailing list grew out of the Extropians mailing list and Yudkowsky’s “interest in this topic in the late 90s and early 2000s.13 He actually splinters off from Extropians and creates” this alternative mailing list, “which is where we see some of the first overlap with some names you might otherwise recognize [from the AI world]. SL4 was a mailing list for talking about these most intense kind of — even more intense than Extropianism. Like, the most intense forms of future sci-fi speculation.” Names associated with the mailing list include Shane Legg, cofounder of DeepMind, and Nick Bostrom.14 Wright then mentions that Shane Legg coined the term “AGI” (artificial general intelligence, the precursor to ASI, though it’s sometimes used as shorthand for ASI), to which Leahy says, “I think he did. Yes, if I remember correctly.”15

Leahy observes that Yudkowsky “founded an organization called the Singularity Institute” in 2000.16 “So, very ahead of his time.” The initial “goal was to build AI as fast as possible so that we could have AI that can take over the world and fix all the problems. … And so, at this time, this is what really would nowadays be called accelerationism.17 So, you know, the e/accs [effective accelerationists] are little, little babies compared to” Yudkowsky — “he was 20 years ahead of them,”18 although “there’s a broader category of accelerationism that I think is not specific to [him].” It has a connection to Nick Land.19

Leahy then adds: “Fun fact, one of the first people to give [Yudkowsky] money was Peter Thiel.”20 But Yudkowsky, in contrast to Thiel, later came to believe that “if you build something that smart, by default, this does not go well. And so he pivots entirely to thinking about, ‘How do you build safe AI?’ How do you build safe — back then it’s called ‘Friendly AI,’ but what we would call ‘safe’ or ‘controllable’ or ‘aligned.’” Consequently, Yudkowsky “started developing this ideology that he would call Rationalism.”21

He continues: “So, you said this kind of earlier which, like, oh there’s all these things going on that we just didn’t know about and it wasn’t like they were hiding it. Well you would be wrong: they were in fact hiding it. So, this was a thing in the early days of this movement — very early on there was a strong part of the ideology is that don’t tell people — don’t let the stupid unwashed masses know about this. Don’t let the government know about this because they’re, you know, they’re stupid. They don’t understand it. We the smart elites who are super-smart, we’re going to solve the problem. You know, we’re going to solve everything. You know, I just have to make sure that all these dumb, stupid, you know, normal people don’t get in the way.”22

Wright brings up the infighting in these groups, to which Leahy says: “this feels like it happens every six months — there’s a lot of infighting inside of these milieus.” He adds that one reason he’s “broken a lot with these group [sic] of people is because I think it is extremely bad to hide this kind of information from people in the government. I think people are allowed to have insight and make decision-making [sic] on this thing [i.e., building a Digital God]. Doing this unilaterally is deeply immoral.”23

Moving on with the story, Leahy notes that in the “second half of the 2000s, a couple really interesting things starts [sic] happening. One thing is, is that DeepMind is founded. DeepMind, including seed capital from Peter Thiel,” thanks to Yudkowsky introducing “Peter Thiel to Demis [Hassabis] and Shane [Legg] to get some other initiative.”24 Although not “all of the money came from Peter Thiel or this one conversation [after the 2010 Singularity Summit], but it still was a pivotal moment and shows this overlap between the DeepMind and kind of the [Yudkowsky] and, like, this race dynamic thing.” The goal was to get all the smartest people “into one company and keep the government and everyone else away, then we can do it right. Then we can do it properly because then we will have control.”

“So,” Leahy says, “I think we’re starting to assemble all the ideological building blocks that lead to the rickety edifice that we’re at now, where there’s one block missing, which is kind of Oxford University utilitarianism — Peter Singer and [William] MacAskill, which led to what is called Effective Altruism.”25 He notes that “in a sense there’s kind of a bait-and-switch, where you read the head on the tin, what EA says, and you say, ‘Well, that seems very reasonable,’ but then if you open the tin you’re suddenly going to see a lot of crazy stuff.”26

There is only one possible thing Leahy could be referring to here, and that’s longtermism. Some longtermists themselves say that the ideology leads to “crazy town.” As footnote 24 notes, longtermism was the primary focus of EA from its inception.

He continues: “You’re [sic] suddenly see your Anthropics, and your whatever — your people saying that actually replacing humans with AI is good, maybe, or that we have to race to ASI or like many, many crazy things.”27 What happened here? “What happened basically is that this pretty non-objectionable version of EA28 merged through this shared link of utilitarianism with Rationalism and this libertarianism, Bay Area Silicon Valley culture, to create kind of the new mutant ideology. And this grew out to a large degree from — so, one of the largest and most important early EA organizations was called GiveWell.”

What’s that? GiveWell “was an organization that evaluated charities for effectiveness and would help you make better donations, which I think is a very reasonable thing to do. It was founded by a previous hedge fund manager named Holden Karnofsky. And Holden later then got to know one of the Facebook founders, Dustin Moskovitz. Dustin Moskovitz decided to donate a very, very large amount of money to a foundation that Holden Karnofsky would lead, and this became what is called Open Philanthropy.”29

That organization’s money has “gone to OpenAI, to Anthropic, to METR,”30 along with lots of AI safety groups. “What is important to understand about Open Philanthropy is that it is the beating heart of the financial system of EA — is that the engine that kind of fed Effective Altruism to a large degree in Silicon Valley specifically [i.e., the longtermist movement] came through Dustin Moskovitz’s money being funneled through Open Philanthropy. It’s hard to overstate how big of an influence they were.”31 Even more, Karnofsky married Dario Amodei’s sister, Daniella, and “in fact, Dario and Holden were roommates for over 10 years.”32

A key point here is that, Leahy says, “what you’ll see is that the same names keep popping up quite a bit. You can kind of pick and choose almost anyone of the, you know, here and there will be a connection. There will be a connection to OpenPhil or to MIRI. … DeepMind. These people all know each other.33 And this is why I think it’s so fascinating, is that there was an AGI superintelligence scene long before the technology existed. Long before it existed, and it overlapped very heavily with these other adjacent ideologies like Rationalism and Effective Altruism, and so on. And this world is extremely incestuous — extremely incestuous.”34

Leahy notes once again that these are basically cults and, after Wright asks about the Venn diagram of all the ideological communities, Leahy describes it as “an extended cluster that is located primarily in Berkeley and San Francisco.” He adds: “There’s just an extended scene. It’s not one ideology. It’s a scene.35 Some of it is EA; some of it is Rationalism; some of it is just AI company employees.”

That said, it’s not the case that “everyone in Silicon Valley or at these AI companies is deeply part of this culture. But you’d be surprised how many are. You’d be surprised how much this culture has been very influential on the thoughts of people. Even the accelerationists, who … love to present themselves as the anti-EAs [but] they’re part of the same culture. They’re the same. … It’s just like there’s two bits of difference, but it’s like … almost everything is the same. It still comes with this techno-utopian vision, this vision of superintelligence, this openness to a lot of alternative strange things,” etc.36 Leahy even adds that there is a lot of tension between some of these groups, drawing this analogy: “There’s the saying, ‘the only thing a leftist hates more than a rightwinger is another leftist,’ right?”37

At one point, Leahy declares to Wright that “the extended cluster of like all of these adjacent ideologies and beliefs38 doesn’t really have a name.”39 Wright also says during the conversation, rather humorously: “You know, I gotta say for people who [are] younger than me with great literary ambitions there is a great intellectual history to be done about all this, right?,” to which Leahy replies: “Oh yes, oh my god, like, there is. That will be unlike other intellectual history you know.”

“TESREAL” Versus “TESCREAL”

That’s the gist of Leahy’s analysis. I encourage you to listen to the entire interview yourself, beginning around 14:00, as there are many details that I don’t have space to mention here.

What do you think? Does Leahy’s analysis, um, sound familiar? He weaves together an intellectual history in which the main characters are every letter in the TESCREAL acronym, presented in roughly that order, except for cosmism.40 That is, Leahy highlights:

• Transhumanism.

• Extropianism.

• Singularitarianism. (The idea that we should bring about the Singularity by triggering an “intelligence explosion,” in I. J. Good’s phraseology, via recursive self-improvement.41)

• Rationalism.

• Effective Altruism (EA).

• Longtermism. (As noted, this is the version of EA that’s gained a foothold within the Valley. When people talk about EA in SV, especially with respect to the ASI race, they’re almost always referring to longtermism.)

He adds that e/acc is yet another ingredient in the alphabet soup of ideologies, just as Gebru and I do, and as I do in numerous solo articles and interviews.

His analysis centers around the TESREAL (no “C”) ideologies, plus “e/acc.” In exactly the same way, my analysis with Gebru centers around the TESCREAL ideologies, plus a “variant” called “e/acc” — which I have elsewhere characterized as a major camp within the TESCREAL movement, the other being the doomers.42

Furthermore, Leahy refers to this as a “cluster” of ideologies, just as I’ve called it a “constellation” and, back in 2023, a “cluster of ideologies.” In our coauthored paper, Gebru and I refer to TESCREAL as a “bundle.” Leahy says that these ideologies constitute “an extended scene,” whereas Gebru and I describe this cluster as a “movement” (basically synonymous with “scene” in this context). Leahy also asserts, as noted above:

I’m not saying by any means everyone in Silicon Valley or at these AI companies is deeply part of this culture. But you’d be surprised how many are — you’d be surprised how much this culture has been very influential on the thoughts of people.

Gebru and I write:

It is important to note that not everyone associated with ideologies in this bundle believes in the totality of the dominant views in this bundle, and some people may even object to being bundled in this manner. Many people working on AGI may be unaware of their proximity to TESCREAL views and communities. Our argument is that the TESCREAList ideologies drive the AGI race even though not everyone associated with the goal of building AGI subscribes to these worldviews.

If you noticed the many footnotes I added to the previous section,43 you’ll see that nearly every important point that Leahy makes has been made previously by me (solo or with Gebru), sometimes using rather similar language. This isn’t just about the details. It’s about how those details hang together to form a panoramic narrative of intellectual history.

Indeed, many people have said things that are similar to particular claims found in my work (solo and with Gebru). But Gebru and I were the first to assemble the heap of these claims, along with many additional new claims, into a comprehensive picture of the origins of the AGI race.

Leahy’s TESREAL (no “C”) analysis is little more than my analysis with Gebru, minus our emphasis on the role that modern cosmism played in inspiring the ASI race. (The term “AGI” was popularized by a transhumanist, Extropian, singularitarian who founded modern cosmism. He also played an integral part in getting the field of ASI research off the ground. A deeper analysis would reveal that cosmism is an essential part of this intellectual history.)

It’s not that Leahy’s account is somewhat similar to ours/mine. The problem is that it’s virtually identical in every important sense.

What Others Think

Indeed, the reason I became aware of his interview with Wright was because a friend of mine, a lecturer (we’d say “assistant professor” in the States) who’s published extensively on transhumanism, sent it to me along with the message:

He’s saying a lot of the stuff you and I talk about all of a sudden. Being clear about the link between transhumanism and the AI industry. Basically talks about them being a cult, doesn’t use the term TESCREAL. Feels like he’s been reading your articles. But doesn’t attribute. … He goes through the whole TESCREAL thing without ever saying TESCREAL. … I sent you that video cos I really thought he’s saying all your stuff without credit. It almost was weird him not mentioning [sic] the term TESCREAL given his thesis.

(Note that the quote above and all those below have been reproduced with permission.)

After watching the interview myself, I was quite shocked by the significant and extensive similarities. So, I sent it to numerous friends and colleagues, asking for their opinion. Every one who reported having intimate familiarity with my work (solo and with Gebru) echoed the comments above. For example, one person in the AI safety field said:

Another person who knows my work in depth said:

I find it hard to believe that Mr. Leahy knows as much about AI as he does without also knowing that you and Dr. Gebru lay out the intellectual history which he discusses in your First Monday paper and elsewhere [a point that I will return to below]. For that reason, an accusation of plagiarism seems reasonable.

Yet another person — a distinguished professor affiliated with Oxford — watched the first 25 minutes and said that, while Leahy’s analysis is similar to mine thus far, it’s not entirely clear that he’s an “intellectual thief.” I told them that the similarities become even more significant as the interview proceeds. The following day, this person sent me a second message:

You’re right — it gets worse. He says “there’s no name” for the cluster of ideologies, his cult analysis, his claim that there are but “two bits of distance” between EAs and e/accs … I don’t know what to say. Your publication record is very clear and has been picked up by mainstream journalists. I only know Leahy from the AI doc [in which he aligns with the doomer camp]. Now he must sense that the tide is turning and he wants credit for calling it out. Yes, it’s ugly. He knows your work, for sure. What to do about it is another question….so sorry…..

I will include some additional quotes just like these below.

On Twitter, I brought up the issue with Wright, who asked me to clarify what I meant by “TESCREAL thesis.”44 Here’s my abstract-length synopsis of my work (solo and with Gebru) over the past many years:

I’ve outlined this analysis in literally hundreds of articles, radio interviews, TV interviews, podcast interviews, and episodes of my podcast Dystopia Now with Kate Willett (which Leahy is welcome to appear on). Along with the Ethics of Human Extinction, TESCREALism — the history of Silicon Valley ideologies and their relationship to the AGI race — has been a primary focus of my research since about 2021, when I published my first systematic critique of EA/longtermism in Aeon. (I hadn’t coined the “TESCREAL” term yet, but I was basically talking about the same phenomenon, as I see longtermism as being, in a particular sense, the culmination of the TESCREAL movement and it’s pro-extinctionist implications.)

A few examples:

1. AI and the Threat of “Human Extinction”: What Are the Tech-Bros Worried About? It’s Not You and Me. Salon. June 11, 2023.

2. The Acronym Behind Our Wildest AI Dreams and Nightmares. (Note that I didn’t choose the title!) Truthdig. June 15, 2023.

3. What’s Behind the Race to Create Artificial General Intelligence? Understanding the TESCREAL Bundle of Ideologies. (A discussion with Timnit Gebru.) Truthdig. July 28, 2023.

4. Does AGI Really Threaten the Survival of the Species? Truthdig. June 30, 2023.

5. “Effective Accelerationism” and the Pursuit of Cosmic Utopia. Truthdig. December 14, 2023.

6. The TESCREAL bundle: Eugenics and the Promise of Utopia through Artificial General Intelligence. First Monday. June 2024.

7. What Is the “TESCREAL Bundle”? Answering Some Frequently Asked Questions. Substack. June 2024. (Note that I recently unpublished this to rewrite it. It was online for at least a year or so.)

8. The Endgame of Edgelord Eschatology. Truthdig. April 25, 2025.

9. TESCREAL. Oxford Research Encyclopedia of Science, Technology, and Society. 2026 (a prepublication draft was available by March 2026).

10. Skeptical of the TESCREAL Acronym? Read This. Realtime Techpocalypse Newsletter. July 9, 2026.

You can find more examples here, although this is not an exhaustive list. There’s also Gebru’s 2023 talk on the subject (below), as well as a 2024 conference at The New School for Social Research that centered on the TESCREAL ideologies and featured speakers like Douglas Rushkoff, whose term “The Mindset” overlaps in important ways with the TESCREAL concept.

I’ve also explained the TESCREAL thesis in multiple documentaries, including In the Belly of AI, Ghost in the Machine, and Elon Musk Unveiled: The Tesla Experiment.

As discussed in the final section of this article, I am constitutionally disinclined to complain about people “copying” or “plagiarizing” my work without attribution. The problem here is how very similar the picture Leahy paints is to the picture that Gebru and I present. There is hardly any substantive difference between his TESREAL account and our TESCREAL account.45

Why Not Cite Us?

There are three possibilities47:

1. Leahy isn’t familiar with my work on the topic. Hence, he is independently discovering my (and Gebru’s) analysis.

2. Leahy knows that my work on the topic exists but has misunderstood it. He doesn’t mention it because he considers it a flawed analysis, or doesn’t realize that his TESREAL account is, in fact, virtually identical.

3. Leahy is familiar with and understands my work on the topic, but he chose not to mention it despite outlining a virtually identical account.

Taking these in order:

1. Leahy Can’t Have Missed the TESCREAL Analysis

If (1) is the case, then I’d say that Leahy hasn’t seriously researched the topic. It’s impossible to have missed the hundreds of publications, interviews, etc. that I (and Gebru, as well) have written or given. The foundational article with Gebru already has more than 300 citations, and scholars have published articles with “TESCREAL” literally in the title (e.g., this one examining the intersection between the TESCREAL community and psychedelics).

“TESCREAL” has its own Wikipedia entry, as well as an entry in the Oxford Research Encyclopedia (linked above). People on the EA Forum and LessWrong have discussed the acronym, which some have included in the title of their articles, and Leahy has accounts on both websites (here and here). Sam Altman has referenced it on Twitter (below, LOL), Yudkowsky is aware of it, and Marc Andreessen even included “TESCREAList” in his Twitter bio in 2022. And so on.

If you write about AI, and especially if you research the ideologies of Silicon Valley, you will inevitably come across my work. Literally the first website that comes up when you search “ideologies of Silicon Valley” on Google is a 2025 paper by Kendall Giles titled, “Silicon Valley Ideologies.” Here’s an excerpt of the abstract:

Through the quest for Artificial General Intelligence (AGI), the case illustrates how intellectuals and institutions, from OpenAI to Google, sell ideologies branded as TESCREAL: Transhumanism, Extropianism, Singularitarianism, Cosmism, Rationalism, Effective Altruism, and Longtermism. These ideologies predict posthuman evolutionary destinies, mind uploading, and space colonization, with speculative existential risk frequently taking precedence over near-term social harms like bias and inequality.

Even more, I tagged Leahy in a tweet last year in which I intimated that he should cite me and Gebru when writing about the topic, as he published a 2025 report that included a section on the ideological nature of the ASI race (focusing primarily on singularitarianism).

Finally, Gebru and I are the only people who’ve discussed this specific topic in this particular way — it’s not like our analysis is a needle in the haystack. As alluded to, if you were to search for intellectual histories of the Silicon Valley worldview that’s driving the ASI race from 2022 to the present, you’d find very little except for our work. (There are exceptions, such as Elise Bohan’s PhD dissertation on the history of transhumanism, which Leahy actually cites, as well as Alexander Thomas’ The Politics and Ethics of Transhumanism.)

It is, therefore, utterly implausible that Leahy has never encountered my work during his “Internet archaeology” (his term) on the topic. As my colleague said above: “I find it hard to believe that Mr. Leahy knows as much about AI as he does without also knowing that you and Dr. Gebru lay out the intellectual history which he discusses in your First Monday paper and elsewhere.”

This was echoed by yet another person (not previously quoted), who remarked in a voice message to me:

It’s hard to say directly, “Okay, this is clearly stolen ideas.” But, I would say that the issue is that it’s kind of hard to believe, especially since he has been in the scene, or in that general ideological landscape … that he has not come across you or Timnit … that is really where the rub is. Like, he must have at some point, since he has clearly researched it, and your name is just the one that comes up first. And, in that sense, I’m just on a basic level not buying it that it isn’t plagiarized. That seems like absolute bullshit. But it’s hard to say — it’s hard to prove.

2. A Genuinely Curious Person

If (2) is the case, then I’d also argue that Leahy hasn’t done his homework. Given his TESREAL analysis, surely he would have spent some time reading my articles (solo and with Gebru). Leahy strikes me as a curious person. He seems open to new ideas and alternative perspectives. It’s hard to believe he wouldn’t have taken the time to read at least some of my papers, or check out my newsletter.

I think he’d do this even if people around him misunderstood my (and Gebru’s) analysis and insisted that my work isn’t worth looking into. It would be very odd if he uncritically took someone else’s word for it without doing some research and making up his own mind.

3. No One Wants to Think This

The final option is (3), which is obviously unethical. If (3) is the case, it would be ironic given Leahy’s (justified) screams that people in the AI race are acting unethically. I don’t want to believe that (3) is true, but the sad fact is that this sort of thing happens all the time. Indeed, I just published a long article about how Nick Bostrom has habitually taken many of his “best” ideas from other people without proper attribution. A friend of mine at an Ivy League university had a novel idea of his stolen by a peer. When I was a Special Student at Harvard, the PhD students wouldn’t share details about their dissertations with other students for fear of being plagiarized. There is no dearth of examples.

I should also note that Leahy’s organization, ControlAI, doesn’t disclose the donors behind its funding. They’re supported by “dark money,” as it were, and if I had to guess, I’d say that this money probably comes from the TESCREAL community — of which Leahy is now critical.

His previous organization, Conjecture, which he founded, was funded by TESCREALists like Sam Bankman-Fried, and even has its own dedicated page on the EA Forum, which indicates that it was deeply embedded in the TESCREAL ecosystem. So, Leahy has roots in this movement. Perhaps ControlAI is different from Conjecture in not accepting any TESCREAL money, but if it does, this would provide a strong incentive to avoid mentioning me or Gebru in any critique of Silicon Valley ideologies. That’s because most people in the TESCREAL community hate me and Gebru. (See the “Why Is TESCREAL Controversial” section of this article for discussion.)

Hence, if ControlAI is funded by TESCREALists, their funding could be jeopardized by Leahy legitimizing and promoting our work.

I See No Other Options

I cannot think of a fourth possibility. It seems that if (1) and (2) are false, then (3) must be true, and if (2) and (3) are false, (1) must be true — and so on. Of these options, (2) is obviously the best, though still not great.

If Leahy would like to clear this up, I would welcome a friendly chat — over Zoom, on my podcast, or via another platform. I have always valued collaboration, and I believe there is a lot more scholarship to be done on the topic of TESCREALism.

Why not work together in elaborating, precisifying, and improving the analysis that Gebru and I painstakingly put together and which I have extended considerably since 2023, in both peer-reviewed papers and popular media articles? Why not cross-pollinate ideas? Why not build a more robust Team Human movement?

What’s the Good News?

People Are Catching On

The good news is that, first of all, this debacle indicates that people are waking up to the TESCREAL ideologies driving the ASI race. I’ve noted this encouraging trend on numerous occasions over the past several years:

Finally!

Leahy (Apparently) Endorses the TESCREAL Thesis

The other bit of good news is that it turns out Leahy is a champion of the TESCREAL thesis. That’s cool. I’m always happy when someone joins the team (unless they’re fascists or pro-extinctionists, and Leahy is neither).

What do I want to see happen? How can this situation be rectified? At the very least, Leahy needs to mention that his TESREAL analysis is extremely similar to my analysis (solo and with Gebru). Gebru and I have been talking about the TESCREAL thesis before I can find a single instance of Leahy mentioning “Extropianism” online. Beyond this, I would welcome some sort of collaboration (doesn’t need to be formal or anything like that!). As noted above, there is a lot of work left to be done, and I suspect that aspects of the TESCREAL framework may need adjusting.

For example, Yasmeena Khan at the University of Oxford wrote a fantastic master’s thesis on the e/acc movement, titled “Coding Utopia: Effective Accelerationism & Evolving Techno-Optimism in Silicon Valley.” It offers, in part, a history of this neoteric ideology, and her account suggests that the way I understand e/acc in relation to the TESCREAL movement is not fully accurate. After reading her thesis (which is brilliant work), I came to more or less fully agree with her.

I mention this as an example of dialogue between my and Gebru’s TESCREAL thesis — which Khan generously cites — and other scholarship that builds upon and critiques it. I have no doubt that there are details that Leahy and I would disagree about, but working out such disagreements could very well push the tiny nascent field of TESCREAL Studies (or whatever one wants to call it) forward.

Unfortunately, Leahy blocked me on Twitter sometime after I sent him a DM in 2023 asking if he’d like to meet up while I was visiting London.50 I have no idea why he blocked me. It might have been in response to me suggesting last year that he’s copying my work (tweet included above). If that’s the case, it’s rather ignominious.

Fact-Checking Leahy’s Interview

I think Leahy’s analysis is missing a lot of important details. It’s like a granular, sepia version of the analysis that I present in my solo articles and with Gebru. He also said a number of things that are factually false, and — in my opinion — misunderstands the prognostications made by people like Yudkowsky in the early TESCREAL movement. Here are three examples:

1. As noted earlier, Wright says: “I think Shane Legg came up with the term AGI, didn’t he?” Leahy responds: “I think he did. Yes, if I remember correctly.” Students of this history will know that Mark Gubrud coined the term in 1997 (to the best of anyone’s knowledge). Ben Goertzel, who popularized the term, says in a paper that he got the term from Legg, though elsewhere he claims that it might have come from Peter Voss. Gubrud suggests that Goertzel and Legg very likely got it from him. Either way, Goertzel popularized it while Gubrud coined it, though Legg might have coined it independently.

2. Leahy describes I. J. Good as a “sci-fi author” who was “one of the first people to talk about the intelligence explosion.” The second assertion is true, but the first isn’t: Good wasn’t a sci-fi author. To my knowledge, he never wrote any fiction. He was instead a mathematician and cryptologist.51

3. Leahy claims that, “famously, Lord Kelvin, a very famous scientist, said that the idea of getting net energy from fission is moonshine. It’s impossible.” Lord Kelvin died 26 years before the “moonshine” comment led Leo Szilard to discover, in 1933, the idea of a nuclear chain reaction involving free neutrons. Famously, the “moonshine” statement comes from Ernest Rutherford. Leahy also says that, upon reading Rutherford’s statement, Szilard “immediately came up with a design for how to build a nuclear bomb and how it would work.” This is false. All Szilard did in 1933 was realize that if nuclei could be split by one neutron such that they release two neutrons, a self-sustaining chain reaction could result, thus liberating the enormous stores of energy bottled up in those nuclei. It wasn’t until 1938 that the nuclear fusion of uranium atoms was discovered, leading Szilard to understand right away that uranium must release neutrons in the process, thus making a nuclear chain reaction practicable. Szilard then tested this and found that uranium does in fact release neutrons. The design of the atomic bomb, based on these discoveries and insights, required the entire Manhattan Project. I wrote about Lord Kelvin (and thermodynamics), Szilard, and the making of the first atomic bomb in great detail in chapter 4 of my 2024 book Human Extinction. Fun stuff!

Finally, here’s a criticism. Leahy says:

This is what I think is so fascinating here, is that there’s a lot things [sic] are wrong, but fundamentally they [the people in these TES(C)REAL cults] did have secret knowledge which did turn out to be true. If you were in the cult sphere in like the early 2000s, you did know things about the future that other people didn’t know. So this was very intoxicating to people.

I think Leahy dramatically underestimates the causal impact these people had in bringing about the ASI race. Indeed, this is a major deficiency of his analysis, and a central feature of my analysis with Gebru. It was because of the influence of people like Yudkowsky, Bostrom, Thiel, Kurzweil, and other early TESCREALists that we ended up in our current existential snafu.

To suggest that such people were prescient is similar to congratulating me for having accurately prophesied the future when I woke up and said, “I prophesy that I will soon drink a cup of coffee!,” after which I went to my coffeemaker and made a cup of coffee — and then drank it.

Participants in the transhumanist, Extropian, and singularitarian communities (before the other ideologies were fully formed) didn’t have some “secret knowledge.” They imagined a future and then made it happen. Indeed, one could plausibly argue that no single individual has done more to inspire people to pursue ASI than Yudkowsky himself, as I’ve said on many occasions. I think Leahy’s analysis problematically misses this very important fact.

Conclusion: Why Write This?

Why do I care about Leahy not citing me (and Gebru)? Isn’t this kind of petty? Why not just ignore it?

If you’ve spent any time in academia, you’ve likely come across a type of academic — usually an older white man — who endlessly whines about others not citing them enough. They insist that this or that idea was actually theirs — that they were the first to discuss it, and hence a great injustice is committed when anyone references but fails to mention them. Such people bemoan the fact that, in their eyes, they aren’t sufficiently lionized for their supposed brilliance, insight, and originality. This often arises from an overinflated ego and/or deep-seated anxiety about their legacy.

I strongly dislike this type of person. I do not like egos, arrogance, self-importance, or narcissism. And I don’t fool myself into thinking that anyone will remember me once I’m gone.

The reason I was bothered enough to write an article about Leahy’s interview is: (a) Leahy’s analysis isn’t somewhat similar but virtually identical to mine (and Gebru’s), and (b) Leahy is a young guy with lots of money, resources, and influence; he has a larger audience than me, and boasts of direct access to political power in DC; and so on. Meanwhile, I’m an independent researcher and freelance journalist who’s currently struggling to bring in more than $12,000 a year, despite my workaholism and (what some would call) prolific output.

Neither (a) nor (b) are independently sufficient for me to have complained. But (a) and (b) together is upsetting. Those with power and influence have a duty to cite lesser-known figures whose work they draw from (especially when they’re drawing heavily). Even if one claims not to be “copying” them, one still has a duty to at least acknowledge similarities when those similarities are significant and extensive. Citing others is not hard. It takes a single breath to say, “This analysis isn’t original to me. Torres, in solo articles and with Gebru, has outlined this in detail.”

I will close with a quote from yet another friend who I didn’t quote above. Referring to this situation, they wrote to me:

Unfortunately it’s almost never that people warning early get the gratitude they deserve but instead get erased, so people who ignored their warnings can rip off their work and pretend they never ignored sometimes made fun of it.

So it goes.

As always:

Thanks for reading and I’ll see you on the other side!

3

I have repeatedly described the TESCREAL bundle as the orthodox religion of Silicon Valley (see the title of the interview from 2023 below), and characterized the TESCREAL worldview as “bizarre,” “radical,” and a “dangerous techno-utopian-apocalyptic cult” (from a 2023 article). By implication — yes, Silicon Valley is a very weird place!

7

I have also repeatedly described the TESCREAL worldview as a “religion” and “cult” (here, here, etc.). As I say in this interview from 2023:

In fact, the parallels between transhumanism and traditional religion are really quite striking. That’s really not a coincidence. If you look at the individuals who initially developed the transhumanist ideology, they were explicit that this is supposed to be a replacement for traditional religion. It’s a secular replacement.

Many other examples could be adduced.

22

I have repeatedly made this point, sometimes using nearly identical language. The TESCREAL movement is extremely elitist. They believe that they have “special access” to “fundamental truths” about what the future should look like, and that the rest of us aren’t smart enough for our input to matter. For example, in early 2022, I wrote:

In 2021, I said:

This has, indeed, been a central feature of my criticisms of the TESCREAL movement. In early 2025, I told Gil Duran:

I would say that there is an incredible amount of hubris among these people. And as someone who was in this community back in the day, I can attest that many have an idea that they have some kind of special access to fundamental truths about what the future ought to look like. And now their task is to impose this vision on the rest of humanity. So they sort of speak for all of humanity.

In another 2025 interview, I said:

The rationalist community is very influential in Silicon Valley, and what the rationalist community basically tells its members is that once you’ve mastered this or that theory within the field of decision theory, once you’ve mastered the patterns of thought that are optimally rational, you then have direct access to fundamental truths about what the future ought to look like. And because rationality provides a universal and objective perspective on the world, you don’t need anyone else’s perspective. That is exactly how these billionaires in Silicon Valley are thinking about our collective future.

In an article from around the same time, I wrote:

This was a main point of yet another EA Forum post by Cremer, bemoaning the worship of IQ. People like MacAskill, Bostrom, and Yudkowsky are elevated to an almost messianic level of cosmic importance. They are the prophets of EA-longtermism, sent by the Rationality Gods to deliver fundamental truths about the apocalyptic hazards and utopian promises that lie ahead.

There are many more examples. Leahy is parroting a central thrust of my critique: the TESCREAL movement is fundamentally undemocratic and elitist.

23

This is exactly what I’ve argued. In a 2024 interview with Tech Policy Press, I say:

It’s just worth noting that the way I understand the TESCREAL vision is as a kind of tyranny. This is a very particular view about what the future ought to look like that was crafted almost entirely by white men at elite universities like Oxford and in Silicon Valley, and now they’re trying to realize this future, and in doing that to impose this future on the entire rest of humanity. And the way they’re trying to realize this future is through AGI, because again, they believe that AGI is going to be the most important and the last invention we ever need to create. And if it’s value aligned … it’ll bring about that utopia. If it’s not value aligned, then by default it’ll kill everyone on Earth.

In a 2025 interview, I described the TESCREAL approach to building ASI as

deeply elitist and extremely undemocratic. Right now, with their billions and trillions of dollars, they are trying to create a new world run by post-humans without ever having inquired about the opinions and preferences of the rest of humanity. They’re doing this without our consent, and they don’t really care one bit about what the rest of us have to say.

In my recent “Pro-Human Manifesto,” I say:

All the AI CEOs have insisted that AGI should benefit all of humanity, but we’ll see in the second half of this manifesto that their undemocratic, tyrannical approach to developing AGI suggests otherwise.

From 2022:

I recently heard that Netflix will premiere a new series titled The Altruists on November 19 of this year. It’s about Sam Bankman-Fried and his partner in crime, Caroline Ellison, and how they committed one of the worst financial crimes in US history. The Effective Altruist (EA) movement will also, it seems, feature prominently in the series. The press release states that it’s

the story of Sam Bankman-Fried and Caroline Ellison, two hyper-smart, ambitious young idealists who tried to remake the global financial system in the blink of an eye — before they were accused of stealing $8 billion and became Gen Z’s own Bonnie & Clyde.

I hope The Altruists doesn’t present Bankman-Fried and EA in a favorable light, but describing the main characters as “hyper-smart, ambitious young idealists” suggests that it might. Furthermore, the comparison to Bonnie and Clyde raises eyebrows because the two are often romanticized as folk heroes in popular culture, and the series is being produced in part by Vox Media — which hosts a vertical called Future Perfect, once funded by an organization run by Sam Bankman-Fried and his brother Gabe! (Note that I was once a member — I am so embarrassed to admit this — of an “elite” EA group in DC. Another member, in the same room as me, was Gabe Bankman-Fried.)

Or, consider this quote from Graham Moore, the writer, creator, and showrunner of the series:

From the very beginning, I was so fascinated by both Sam's and Caroline's real stories. Those are both characters who I could not have possibly made up had I tried — I'm not nearly a good enough writer to create characters as fascinating as either Sam Bankman-Fried or Caroline Ellison. ... This form [of storytelling] allowed us to tell not just a kind of headline version of the story, but to tell what we believed was the much more fascinating, intimate, personal, deeply heartbreaking story of these two people.

“Deeply heartbreaking” — you mean for the millions of people who Bankman-Fried and his EA accomplices stole over $8 billion from? Or for poor old Sam and Caroline who just wanted to make the world better and, you know, tried a little too hard? If this series is appropriately critical of EA, I can imagine myself really liking it. But I’ll be deeply disappointed if it portrays the villainous EA movement in sympathetic terms.

I also got a chuckle out of the fact that William MacAskill will be played by a Scottish actor named Jack Greenlees. I wonder if there will be a humorous scene in which MacAskill talks to Bankman-Fried about how he considered getting the gap between his two front teeth closed “because studies showed that more ‘classically’ handsome people were more impactful fund-raisers.” Also, Barack and Michelle Obama are among the series’ executive producers, and the CEO of FTX Digital Markets is played by Matt Rife (lol). Hmm.

Rationalist Outreach Is Embarrassing

Speaking of EA, the Rationalists — a community out of which EA grew — have been experimenting with new ways of getting their AI-doomer message across. Exhibit A is a recent video from Nate Soares, coauthor of If Anyone Builds It, Everyone Dies with the doomer-in-chief Eliezer Yudkowsky. I have absolutely no idea who they think this will appeal to, as I find it unbelievably cringe-inducing. What do you think?

It’s clear they have a ton of money for “outreach,” but despite their intellectual superiority (you know, super-high IQs and near-perfect rationality), my sense is that videos like this will hurt their cause.

Another example is this music video, in which the singer, who intermittently wears a chain of paperclips around his neck, sings: “This is how we’ll die / Race to AGI / Oh oh.” The chorus consists of the repeating line: “Last year alive” — because we’ll soon “be a paperclip” and “all the value will be lost” (a reference to the estimated 10^58 digital people who could exist if we build aligned superintelligence and then colonize the universe — won’t someone think of all the digital unborn!) He even references wireheading involving “AI hyperporn.” The song closes with repetitions of: “S-T-O-P.”

As you may have noticed, Yudkowsky and Soares both make cameos, with the latter describing this song as a “banger” on social media. I suspect this — and Soares’ “big smart” video above — was filmed at Lighthaven, the Rationalist doomer compound in Berkeley operated by a group called Lightcone Infrastructure, which also maintains the website LessWrong.

You Wanna Get Food, or Something?

In my previous article, I argued that AI capabilities may be advancing, and such advancements could carry significant risks to society, but that these facts alone do not necessarily imply that AGI, ASI, or the Singularity are imminent — or even getting closer.

For a chuckle, here’s a pretty hilarious video mocking the conversational limitations of current AI systems — specifically, the unnatural and obnoxious ways that ChatGPT engages in dialogue. I have never had the urge to punch someone in the face while talking to them (even in the most annoying of circumstances), but I feel my fists clenching when I watch this:

Although very funny, it also highlights some serious questions about the extent to which AI chatbots may be influencing — subtly or otherwise — our patterns of speaking, writing, and even thinking. The worry is especially acute when it comes to children growing up with these systems.

As a friend of mine has repeatedly emphasized to me: What are the implications of AI with respect to the socialization of children? What might we expect when a nontrivial percentage of young people’s interactions involve AI rather than other humans? Even in the case of human-to-human contact, it’s likely that many others in their cohort will have been influenced by AI, meaning that even this form of interpersonal exchange may be tainted by LLM rigmarole. What should we expect from an entire generation that, during its formative developmental years, had conversations with ChatGPT like the one above?

My dad sent me another video that illustrates a similar idea — in this case, all the characters are relying on AI chatbots to tell them how to respond to others. It’s pretty hilarious:

Yes, of course, every generation has worried about how new technologies might be unraveling the fabric of society, thus dooming it to a slow deterioration until the worst dystopian nightmares have been fully realized. Taylor Lorenz recently noted in an amusing thread that novels were once “compared to opium, alcohol and cigarettes and parents wanted restrictions on children being able to read them. They wanted to raise the age to read novels to 18.”

But surely some of the concerns around AI are warranted. Many studies, for example, have converged on the conclusion that even moderate use of AI can “potentially reduce critical thinking, creativity and problem-solving.” Or, as an article in the BBC put it, “AI chatbots could be making you stupider.”

Of course, future research may disconfirm this finding, but right now the totality of available evidence clearly points toward: “We should be seriously concerned about the impact of AI on our cognitive functioning,” especially the functioning of younger generations who’ve been conversing with AI since childhood.

What do you think?

The Pigeon App

Lorenz also apprised me of a new app called Carrier Pidge. It was developed by a young man who posted on X that he “spent 3 weeks building a texting app slower than email.” When you send a message, it traverses communication systems “at pigeon speed,” meaning that it takes a message 22 hours to “fly” from Los Angeles to New York City. He adds that “sometimes the bird dies & the message isn’t delivered.” Here’s a video of the process (no audio, fyi):

You can download the app here. I did that, but can’t send a message because I currently have no friends in the “flock.” If you download it, let me know and I’ll send you a message!

The app obviously won’t catch on — it was created for a laugh — but it gestures at something most of us desperately want these days: for life to slow down a bit. Things are moving too quickly. I don’t think I’m speaking only for myself when I say that we receive too many emails and messages. The 24/7 news never lets up. Many of us are suffering from acute cases of crisis fatigue. Etc.

Although there are many aspects of modern life that I wouldn’t voluntarily give up, I also feel — as many of us do — a kind of nostalgia for the “old days” when information overload wasn’t such a pervasive problem. The Carrie Pidge app, at the very least, reminds us that there are other — perhaps better — ways of doing things.

Timnit Gebru’s New Book

Finally, I wanted to share a bit of satire from Timnit Gebru, who recently posted this on social media:

📢I’ll be creating a fully autonomous startup, staffed by AGI. I care deeply about humanity’s future and can’t wait to work alongside ethical workers to hopefully make a gazillion amount of 💰. Surely I shouldn’t be left out of the retirement and oil 💸 flowing into this bubble.

Making the money that I’m going to be making is just an unavoidable side effect of me saving the world, which I unfortunately have to endure.

Given that only I can do this (and let me just add that it’s dangerous to let anyone else attempt this), it’s unfortunate that only I will be accumulating all this wealth. But that’s the price of carrying the world’s burden and being tasked with bringing utopia to all of us.

That’s a pretty good encapsulation of how people at the AI companies think.

Note also that Dr. Gebru has a forthcoming book titled Deep Unlearning: The Rise of AI and the Radicalization of a Tech Idealist.

I’ve not read the entire tome yet, but I was given an opportunity to look over a chapter in which Timnit discusses our collaboration, which led to a 2024 paper on the TESCREAL ideologies. (For an intro to the TESCREAL concept, read this.)

That chapter was, unsurprisingly, extremely good. Timnit has an incredible ability to identify the crux of an issue, expose the core problem within it, and articulate it in powerfully insightful ways. Indeed, there have been many occasions in which I’ve messaged her (especially during our collaboration) with something like: “This statement by X seems off to me. But I can’t figure out exactly why or find the words to express what I want to say.” With envious ease, she’d shoot back a sentence or two that consistently left me thinking: “God damn, yes, that is exactly what I was fumbling over my own thoughts trying to express!”

You can pre-order the book here. I will, of course, write a review once it’s out, and Timnit will appear on my podcast Dystopia Now to discuss it. Very excited for the book release, and I hope it becomes a bestseller.

Subscribe now

PS. An Amazing Solar Eclipse

A few years ago, I witnessed my first total solar eclipse in Cleveland, Ohio. I’m not big on seeking out new experiences — oftentimes, I’d rather sit at home and read about something than go out and experience it. (Visiting new cities and countries is the one exception, although once I’m ensconced in a new place, I’m usually content to sit at a cafe reading and writing all day.)

So, I wasn’t especially excited about the eclipse, and didn’t expect much. But holy hell, it was incredible. I can’t describe with words how profound it was. The sky darkening in the shadow of the moon wasn’t anything like sunset. It was as if the sun itself had dimmed — as if I were suddenly experiencing a different sun, on a different planet elsewhere in the universe.

On Wednesday, I wasn’t in the path of totality, and it wasn’t in the middle of the day. Nonetheless, experiencing the penumbra of the moon’s shadow at sunset was still deeply unsettling in the most marvelous of ways. I took videos of the event, but won’t share them yet because they would reveal my exact location.

Instead, enjoy this video posted online, which might have been taken in Spain (I’m not sure). The cheers and shrieks from the crowd gives me chills. Incredible stuff:

Wishing everyone the very best. As always:

Thanks for reading and I’ll see you on the other side!

There is no contradiction between the following statements:

1. AI models are becoming increasingly powerful.

2. These increasingly powerful AI models pose significant risks to society, perhaps even humanity itself.

3. None of this is bringing us any closer to AGI, ASI, or the Singularity.

In other words, one can accept that AI systems are advancing quickly, these advances pose serious risks, and ASI remains as distant today as it was, say, in 2022. This article explains why.

“Much Less Weird than it Seems Like It Should Be”

As we’ve discussed on numerous occasions, many people in Silicon Valley are convinced that the Singularity is imminent. Sam Altman wrote last year that “we are past the event horizon; the takeoff has started.” In January, Musk posted on X that “we have entered the Singularity.” He reiterated this in early August, after news about OpenAI’s Astra model (discussed below) broke, saying:

(Since he asked, the temperature over here in Europe is UNBEARABLE. Good thing that AI isn’t contributing to climate change!)

Demis Hassabis, who recently stepped down as CEO of DeepMind, said that “ten years from now, I think we’ll realize that we were standing in the foothills of the singularity now.” He believes ASI will arrive by the early 2030s, after which we will quickly start building a Dyson swarm around the sun (lol). And Dario Amodei claims that AGI could arrive later this year. Even Jensen Huang of NVIDIA said last March: “I think we’ve achieved AGI.”

Amusingly, after Altman proclaimed that we’re “past the event horizon” in his blog post, he added that “humanity is close to building digital superintelligence, and at least so far it’s much less weird than it seems like it should be.”

Yeah, dude, it’s not that weird because the Singularity isn’t happening! The event horizon still looms above the actual horizon before us — which is to say, we’re nowhere close to triggering an intelligence explosion.

The OpenAI-Hugging Face Incident

Yet, it’s clear that AI models are becoming significantly more “powerful,” and that newer models — Mythos being the most obvious example — are introducing novel risks that should worry us greatly.

I’m sure you’ve heard of this incident — it’s one of several that have recently been reported by AI companies. Basically, OpenAI was testing “an experimental in-house AI, working with the latest commercially available model.” As part of the test, they gave it “a set of cybersecurity tasks” to complete. The AI then figured out a way to escape its digital sandbox by identifying and exploiting “a previously unknown zero-day vulnerability,” hack into Hugging Face, and search Hugging Face’s datasets for a solution to the cybersecurity problems it was tasked with solving.

This happened in part because OpenAI failed to design an adequate sandbox. Some of the AI’s security guardrails were turned off — if they had been on, they “may have prevented this” from happening — and the sandbox “had one small connection to the outside world, to allow the AI to download software it needed.”

This portal to the public Internet was mediated by a “package registry cache proxy,” which is basically an intermediary server that provides a kind of barrier between the AI and public Internet. But the AI found a vulnerability in this intermediary server and, in doing so, broke free of its prison, thus enabling it to access the Internet and hack into Hugging Face.

It’s worth highlighting that no one at OpenAI told the AI to cleverly break out of the sandbox and hack into another company — which constitutes a felony! Yet, as Gary Marcus notes:

the current incident was NOT an attempt where [the] system built a goal for itself or developed a motive; the system was following instructions, but not setting high level goals. It was not trying to take control of the world a la Terminator, it was just trying to cheat on a test, which is at least a bit less scary.

“On the less comforting side,” he adds that OpenAI’s guardrails (when turned on) are nonetheless

likely to be permeable, just like all guardrails anybody has built to date. Even putting aside the thorny questions of open weight systems, we have no guarantee whatsoever that future models won’t be able to do similar things, such as finding zero-day exploits to hack systems. To the contrary, we can expect more incidents of this type.

This is unsettling. As a New Yorker article puts it, what might have happened “if the rogue AI had had access to a robot, or a self-driving car, or a bio lab, or a weapons system”? Recall that Pete Hegseth is actively transforming the US military into “an ‘AI-first’ war fighting force across all components, from front to back,” according to an official document from the Department of War. Already, war game tests show that AIs will threaten nuclear war in 95% of scenarios.

Even more, Chinese companies are releasing open-source AI models that, as such, allow people to remove any guardrails they might come with. For all of these reasons, “we can expect more incidents of this type,” though we should also expect future cyberattacks to become more sophisticated and difficult to prevent.

Subscribe now

AI Invents New Viruses

Around the same time this snafu became public, the New York Times reported that

Scientists at Stanford University and the Arc Institute, a research organization in Palo Alto, Calif., taught AI [called “Evo,” similar to ChatGPT] to recognize patterns of DNA structure in nature, and then to use that data to write recipes for entirely new viruses.

The researchers followed those recipes to create DNA molecules, which they inserted into bacteria. The modified bacteria then produced viruses never seen in nature. The viruses were able to infect other bacteria, demonstrating that they were viable. …

As Mr. King and his colleagues tested more genomes, they saw more clear dots. All told, they discovered that 16 of Evo’s genomes produced viable new viruses.

They proved to be as resilient as natural ones. In fact, some multiplied faster than Phi X-174 [see below]. “They’re not just sickly versions of stuff that already exists,” said Oliver Crook, a protein chemist at the University of Oxford who was not involved in the new study.

The article adds that, fortunately, “the viruses dreamed up by A.I. do not pose a threat to humans, because they are all similar to a naturally occurring virus called Phi X-174, which can infect only bacteria,” and they’re “not radically new creations. They tend to be very similar to natural species, relying on the same underlying biology.”

Nonetheless,

Evo’s initial success has raised concerns that A.I. could be used to create deadly pathogens. “You could say, ‘Hey, genomic language model, make me an influenza genome that is modified to be more transmissible or to be more lethal,” Dr. Hanke speculated.

The potential dangers were already on the minds of the researchers as they trained Evo. They did not provide the model with data about viruses that infect humans, and they excluded similar viruses that infect other animals, plants and fungi.

The possibility of AI-enabled designer pathogens has been discussed for decades among scholars of global catastrophic risks. An express goal of synthetic biology is to “black box” aspects of genetic manipulation: rather than spending years in a laboratory acquiring “tacit knowledge” about how to manipulate the building blocks of life, it aims to enable individuals to achieve the same ends by simply pushing a button. (This is also called “deskilling.”)

Furthermore, the genomes of horrific pathogens like Ebola and smallpox are publicly available online (I won’t include a link!), and there’s no theoretical barrier to combining the incurability of Ebola with the contagiousness of the common cold, lethality of Rabies, and long incubation period of HIV. Doing so would enable “risky agents” (a technical term meaning those with an explicit desire to destroy civilization and/or humanity) to synthesize a germ that spreads around the world undetected, infects nearly everyone, and then 10 years later results in mass death, perhaps even human extinction.

Is this hyperbolic? Maybe, a little — but the point is that AI could supercharge the risk. If risk = consequences x probability, then even a low-probability pandemic that kills a large fraction of the human population would still constitute a huge risk.

Add to this the fact that there is a very large number of people — a small percentage of the total population, but a sizable demographic in absolute numbers — who harbor a death wish for humanity. I’ve written about some of these people in the past, such as the Efilists, who say they’d murder everyone on Earth, including themselves, to eliminate all suffering. (This arises from their allegiance to negative utilitarianism.) An Efilist even bombed a fertility clinic in California last year.

The point is that initiating an engineered pandemic is becoming increasingly feasible. Deskilling, publicly available genetic data, tools like CRISPR-cas9, and now AI are a recipe for disaster.

Subscribe now

AI and Mirror Life

What worries me even more than designer pathogens is the possibility of AI assisting in the creation of “mirror life.” This refers to forms of life that consist of molecules with the opposite chirality of the molecules that comprise all biological life on Earth right now. A bacteria with mirror molecules could potentially infect the human body without being detected by our immune systems — such bacteria would, essentially, be invisible to our internal defenses.

As an article in Science, written by nearly 40 prominent scientists, declared:

We cannot rule out a scenario in which a mirror bacterium acts as an invasive species across many ecosystems, causing pervasive lethal infections in a substantial fraction of plant and animal species, including humans.

How might AI exacerbate this risk? Amodei himself highlights the danger in his essay, “The Adolescence of Technology,” writing that a “sufficiently powerful AI model (to be clear, far more capable than any we have today) might be able to discover how to create it much more rapidly — and actually help someone do so,” where “it” refers to “mirror life.” Yikes.

AI Capabilities are Getting Spikier, Not More General

All of this leads to the linchpin of my argument here: advances in AI and the potentially catastrophic risks they’re introducing don’t for a moment imply that we’re getting closer to AGI, ASI, or the Singularity.

This is counterintuitive, but consider a recent X post from a professor at Cambridge named Adam Hunt. He reports having previously been bullish about AI, but now considers himself bearish. The reason is that

the whole “it turns out if you keep training and scaling the models more they develop broad new capabilities in lots of domains” thesis is wrong (sorry Demis). The recent batch of models haven’t got more general, they’ve got less general. This is most obvious in the fact that their language outputs have got much worse in comparison to e.g. o3. If they were gaining generalist capacities we would expect them to be describing their work in ever more graceful and comprehensive prose!

Take a look at this widely shared image:

The “mainstream AGI thesis” on the top hasn’t panned out so far. On this account, AI will initially show superhuman capabilities in narrow domains, but as research advances, its capabilities in other areas will also increase. Eventually, the AI will exceed humans in all cognitive domains of interest, at which point we’ll have ASI.

In fact, what appears to be happening is that AI is getting (significantly) better in narrow domains (e.g., coding, finding software vulnerabilities, inventing new viruses, etc.) without getting much better in all other domains — i.e., those it currently underperforms in. The spikiness of its capabilities is getting spikier. If the trend continues, it will become ever-more superhuman in particular domains without attaining general intellectual capabilities.

When someone asked about whether “chain-of-thought make[s] AI more general, less general, or the same?,” Hunt responded:

I dont know if ‘less general’ or ‘more general’ is the dimension that chain of thought works upon.

I think chain of thought basically exaggerates existing potency in the underlying models, by allowing that existing potential to reflect and re-reflect with its existing capabilities. So with coding that leads to much much better coding, because the models are good at coding.

But in scientific logic, which LLMs are not good at (e.g. in my work, recently Opus 5 tripped up over how seriously should it take one specific study of a particular drug’s effects on ADHD) then chain of thought doesn’t help - it gets trapped in illogical loops that it self justifies and can’t leave alone.

So no, I don’t think that scaling up inference compute will suddenly get you to AGI, because of fundamental spikes and flaws in the underlying training data/RL which lead to blind spots or weaknesses in the basic model, which can just compound over time in chain of thought rather than getting suddenly fixed.

This seems right to me. One can thus acknowledge major advances in AI capabilities while simultaneously holding that these advances aren’t inching us closer to ASI or the Singularity.

Subscribe now

The Fallacy of Composition

Marcus makes a related point in a recent blog post, highlighting how AI hypesters are committing the fallacy of composition. For example, OpenAI reported that its new (unreleased) AI model called Astra “solved 10 major open problems in mathematics, quantum complexity, and theoretical computer science.” That led Dean Ball — among many others — to declare that:

The fallacy of composition occurs when “one infers that something is true of the whole from the fact that it is true of some part of the whole.” In the case of AI, the fallacious reasoning goes: “AIs are becoming more generally intelligent — thus getting us closer to ASI — because they are becoming more capable in this or that particular, narrow domain of cognition.”

That absolutely doesn’t follow. It’s like saying, “A particular aspect of my life is improving, as I now have more free time during the day to pursue hobbies. Therefore, my entire life is improving” — which of course might be false if one also just lost their job, spouse, etc. The increasing spikiness of AI capabilities doesn’t imply that more recent AI models are closer to ASI — superhuman general intelligence — than previous ones.

Conclusion

What’s the takeaway here? It’s that the incidents mentioned above don’t make me — and shouldn’t make you — more concerned that Altman, Musk, and the others are right in proclaiming that the Singularity is imminent or has already started.

At the same time, they should make us more anxious about the potentially catastrophic consequences of increasingly powerful spiky AI systems. An AI-enabled pandemic could be devastating. AI-enabled mirror life could literally destroy the biosphere. Cyberattacks will become more difficult to anticipate and prevent. And the fact that the US military is actively integrating AI into its infrastructure is hugely worrisome (as you may recall, Anthropic’s Claude was used in the illegal kidnapping of Nicolás Maduro and has helped select targets in Iran). Etc. etc.

So, I find myself both deeply worried about these new developments and no more nervous than I was about ASI suddenly emerging from the bowels of an AI laboratory. As I like to say:

Subscribe now

But what do you think? What have I missed? How might I be wrong? As always:

Thanks for reading and I’ll see you on the other side!

As you may know, some of the authors behind the widely-discussed “AI 2027” paper published a follow-up titled “AI 2040.” One of the authors, Daniel Kokotajlo, was a buddy of mine when I lived in North Carolina. He even read a draft of my 2017 book on existential risks and provided comments:

Whereas AI 2027 focused on how things could go terribly wrong with ASI (artificial superintelligence), this new document outlines a “positive vision for what should happen instead.” Both documents contain little more than wild speculations paired with unfounded assumptions and unrealistic predictions. In the Postscript, the authors write:

We think the world is asleep at the wheel. People say the words “AI will be transformative” without thinking concretely and seriously about the implications of broadly superhuman AI.

Yet I see no evidence that the authors have thought seriously about the implications of their best-case scenario, which they call “Plan A.” Plan A is little more than the TESCREAL vision of the future and, if realized, it would likely have devastating consequences for humanity — for reasons I outline in my “Pro-Human Manifesto,” and which I’ll briefly touch upon below.

To give you a sense of the normative futurology behind Plan A, here’s how they describe “Life after ASI.”

The resulting plans are too many to number. There are all sorts of schools of thought about what to do with your galaxy of resources,1 including:

• Defer to the Future: Some people decide to give most of their resources to their children (or other people in the future), and let them choose.

• Human Flourishing: Set up worlds of normal, free people living ordinary lives. This is a harder problem than it first seemed, because human civilization has already become profoundly abnormal: by this point, human activities like art or science has already become dominated by machines. Some handle this by accepting that their galaxy will be filled with post-scarcity humans, able to pursue whatever interests they find most meaningful. Others decide to roll back technology to leave space for human labor.

• Digital Human Flourishing: If humans living happy, flourishing lives is good, then why not increase the number of people living these lives? Brain emulations can be run for vastly cheaper than real humans: a planet-size computer could plausibly simulate the equivalent of a million planets with happy civilizations.

• Acausal Trade: Some argue that there likely are aliens in distant unreachable galaxies and that we could use various mechanisms to make deals with these aliens. If this is true, they argue, we could cut a deal to pursue compromise values rather than narrowly maximising our own values. Therefore, we might, on net, be able to do the most good by pursuing not just our own values, but instead a compromise of a vast number of other civilizations.

Humanity spreads across space in a dizzying variety of forms and ways of living — more diverse than anything Earth alone could have produced.

The statements above are consistent with what Kokotajlo has said in interviews:

Here are a few reasons I strongly dislike the “AI 2040” analysis:

1. Extreme Hubris

They write: “First, our goal was to construct a positive vision, a grand plan for how to achieve an actually good future for everyone.” The extreme hubris of this statement is difficult to overstate. If I may use some vulgarities to get the point across: Who the fuck are they to be declaring that they know what “an actually good future for everyone” looks like?

(Sam Altman himself once said something similar when lying about how OpenAI will include people from around the world on its board. He said: “We’re planning a way to allow wide swaths of the world to elect representatives to a new governance board. Because if I weren’t in on this I’d be, like, Why do these fuckers get to decide what happens to me?”)

This is classic TESCREAL: the entire movement is infatuated with the supposed “intelligence,” “rationality,” and “brilliance” of its members. This enables them to believe that, because of their “intelligence,” “rationality,” and “brilliance,” they can speak for everyone in outlining a grand teleological vision of our collective cosmic future. These people think they have special access to fundamental truths about what the future ought to be, and by virtue of this prophetic knowledge their vision is superior to that of the benighted masses. It is a kind of messianic hubris, a radical form of elitism — nothing less.

In contrast, consider my favored approach to, as it were, designing a genuinely good future for everyone. I would argue that no single group should have the power to unilaterally impose their vision on everyone else. Rather, our collective future should, from the very start, be determined through a thoroughly democratic process. The people — you and I — should have a say in how things turn out. We should have a say in how advanced AI is developed, and whether it’s even developed in the first place.

As my friend Monika Bielskyte likes to say, “You cannot design for, you must always design with.” That succinctly captures the essence of my position, and there is not a single trace of this democratic approach in the “AI 2040” document.

Nowhere do they acknowledge that people outside the dumb TESCREAL movement should have a say in whether superintelligence is birthed in the bowels of an AI laboratory. Nowhere is there any indication that the authors are even curious about alternative visions of the future. The authors, I think, are completely unaware of just how much their preferred vision has been intimately shaped by the values of the milieu of Western capitalism plus utilitarian ethics (as I’ve written before, these are essentially the same). Maybe they don’t care.

To be clear, I am not saying that the authors shouldn’t express their opinion about which future is best. The whole point of establishing a governance mechanism to democratically determine the future is for people — or representatives — of every nation, tribe, religion, tradition, race, sex, gender, socioeconomic demographic, political party, ideology, and so on, to have a voice in the process of future-designing. That includes the voice of Kokotajlo and the other TESCREAL utopians.

But this is not the point of Kokotajlo’s vision. He and his coauthors are laying out a roadmap for the AI companies, policymakers, etc. to follow: slow down the ASI race, solve the “alignment problem,” and then create their version of a cosmic “utopia.”

Imagine, for a moment, that a group of religious extremists in Iran announce they’re building a new technology that could determine the entire future of humanity. There’s a good and bad way to build this technology, they claim, and Iranian scientists are working hard to ensure that it’s built the “right way.” These people then outline a roadmap for the future based on Islamic principles, which they declare will be “good for everyone.” The obvious response is to shout: “Who the hell are you to speak for everyone?” Yet, there is no structural difference between what the “AI 2040” authors are doing in their document and what these religious extremists are doing in this fictional example.

I genuinely can’t comprehend the level of hubris underlying “AI 2040.” Indeed, the hubristic heart of the TESCREAL movement is one reason I left circa 2019. These people have no idea how big the world is, yet feel wildly confident in their ability to design beneficial-for-all futures. Nonsense.

2. Climate Change

In their preferred future, Plan A, there is literally no consideration of climate change. Not once do they mention global biodiversity loss or the sixth major mass extinction of the last 3.8 billion years. How can you present yourself as a serious scholar of the future and not mention the environmental crisis?

I’m in the foothills of the Alps right now, and yesterday it was 97F (36C) — roughly 20 degrees over the average high for August. Half of the world is on fire. And 2027 is predicted to shatter temperature records around the world (many of which have already been broken this year) due to a historically unprecedented Super El Niño. By 2040, large parts of the world will be more or less uninhabitable — yet the authors say literally nothing about this.

Why? The main reason concerns “magical thinking” about ASI paired with an “engineering mindset.” These people simplistically see all problems as engineering problems — that’s why Nick Bostrom talks about a “solved world” after ASI arrives. If we have a superintelligence, then we automatically have a super-engineer; and if we have a super-engineer, it will be able to solve every engineering problem facing us — which is to say, once again, all problems. (Bostrom even talks about “paradise-engineering.”)

Climate change is no different. The only reason we haven’t solved climate change, on their account, is a lack of intelligence (and hence engineering). Once we have superintelligence, it will find some brilliant technical solution and immediately reverse all ecological pollution, restore ecosystems, remove all excess CO2 from the atmosphere, and so on.

This is magical thinking. The truth is that a lack of intelligence is not the reason climate change hasn’t been solved. Building a superintelligent machine isn’t going to somehow fix the problem before 2040. By then, Earth will be burning, millions or billions of people will be dying or migrating, and entire societies will be collapsing. It’s truly abysmal that the authors don’t take climate science seriously.

3. No One But the TESCREALists Want AI to Run the World

The authors write that in 2033,

as AI capabilities improve throughout Plan A, they increasingly shape every facet of life. … AI agents now form a population of two hundred million virtual workers that think and act 50x faster than humans, and never sleep. … By early 2036, there are 200 million AIs, equivalent to a workforce of around 100 billion humans, and 2 billion robots with some mixture of humanoid and other specialized form factors. … Gradually more institutions and equipment are handed over to AIs, such that it’s no longer true that humans could shut it all down if they wanted to. In fact it’s extremely untrue: Soon many of the world’s militaries are autonomous, run by AIs sworn to uphold various constitutions and treaties.

Why on Earth do the authors assume that people won’t fight back against the growing dominance of AI and attendant disempowerment of humanity? Of course there will be mass protests, civil unrest, and outrage. Depending on the details, there may be violent conflict in the streets as Butlerian jihadists launch a war against the ascendant AI overlords taking over the world.

This is a major weakness in the authors’ analysis, and you can count me in as one of the pro-human fighters who will do whatever’s necessary, within the bounds of my partly deontological ethics, to prevent the situation described above. Frankly, the authors’ vision of things going right sounds like a dystopian nightmare to me.

4. Colonizing Space Would Result in Constant, Catastrophic Wars

The authors blithely assume that space colonization won’t be a catastrophe. They completely ignore the academic literature on this topic, with renowned political scientists like Daniel Deudney arguing (in Dark Skies) that the almost certain outcome of colonizing our solar system will be constant, devastating conflicts. I’ve written about this topic numerous times, such as here, though I would recommend the closing chapter of Deudney’s excellent book for the original, robust analysis.

Again, this is an example of magical thinking — or what I call the utopian mindset. This mindset gives utopians a license to ignore the messy details of the world by saying, “I don’t know how exactly this will happen. But future technology — especially machine superintelligence — will have magical powers that will somehow enable us to figure it out. Hence, we needn’t worry our pretty little minds about it right now.”

This phenomenon is similar to the “suspension of disbelief” in fiction, whereby one ignores the plausibility of details in a story. No one watches Game of Thrones and stops to ask, “Wait a moment! Are the wings of those dragons really large enough to carry their massive bodies through the sky?” That’s an absurd question.

Similarly, those who embody the utopian mindset habitually dismiss questions about how exactly ASI will reverse climate change, upload our minds to computers, and colonize the universe without triggering catastrophic wars as absurd.

5. ASI Is Fundamentally Uncontrollable

The authors assume that there is a way to control ASI. We just haven’t discovered it yet. That’s why we should slow down ASI capabilities research: to give us time to solve the “alignment problem.”

But why think that ASI could ever be controllable? We’re talking about a perpetually evolving computer program with, by definition, far greater cognitive abilities than all of humanity combined. And yet we will somehow still control this infinitely self-improving beast?

Roman Yampolskiy (another guy I knew from the TESCREAL community!) is probably right that this is impossible, in the very same sense that a perpetual motion machine is impossible. My friend Remmelt Ellen has written about it here.

6. More Magical Thinking

The authors write that around 2036,

many of the world’s evils have dramatically reduced. Malnutrition, lack of medicine, and homelessness are nearly banished. Many diseases have been cured. Crime rates are lower than ever before. … People are wealthier and have more free time, plus there are new technologies that help, like AI matchmakers, better medicine, and AI tutors. And of course even the poorest people can now afford exotic vacations, amazing games, and enthralling entertainment.

Um, okay? Again, magical thinking enabled by the engineering and utopian mindsets. No serious discussion of how the tech elite will exploit ASI — if controllable — to permanently entrench their positions of power, control, and domination in society. The authors need to do a lot more to convince thoughtful readers that this is even remotely plausible.

7. There Is No “View from Nowhere”

The authors write about 2036, just 10 years from now!:

There are still problems in the world, and you can still contribute to solving them, partly by donating or volunteering but primarily by being politically active. Your vote is your most important asset.

The honest AI forecasters are helpful here. Years ago, if an AI said that one presidential candidate was better than another, people would suspect bias and the embarrassed company would retrain the AI to evade such questions. Now, thanks to transparency and improved alignment techniques, there are much smarter AIs that people can see don’t have any biases trained into them, that have built up an excellent track record over several years. When they weigh in on policy questions, people listen, especially when different AIs trained by different companies converge to the same answer.

AIs with no biases! Think for a moment about what that means. It assumes that there is a perfectly objective view from nowhere that, as such, is completely free of biases. The problem is that there is no view from nowhere. All views are views from somewhere. Not even science is perfectly objective.

Imagine asking the AI whether one should vote for a presidential candidate who’s a fascist and white supremacist. What would an unbiased answer to this look like? There is no possible response that isn’t biased in some way.

For example, a “no” response would be biased toward views that I hold, e.g., that fascism and racism are bad. I consider those biases epistemologically and ethically justified, which is why I hold them. But there’s no mind-independent fact of the matter that one can point to in showing that my biases against fascism and racism are correct. There are only more or less compelling arguments.

Later on, the authors write that

during the 2036 election season, voters are unprecedentedly well-informed, and the politicians that win have a genuine commitment to responsible stewardship of the upcoming singularity.

This assumes that all voters want to be well informed. Why think this?

Even more, studies show that climate deniers in the US are often just as knowledgeable about climate science as those who accept the climatological consensus (i.e. lack of education isn’t the best predictor of denialism — ideology is), meaning that being more informed doesn’t imply that voters will make wise or judicious decisions at the voting booth.

8. Woke AI?

The authors write:

Unless the alignment science is somehow all wrong, typical AIs are now more virtuous than the most virtuous humans. This alone has profound effects: it’s as if saints and angels were walking among us.

What on Earth do they mean by “virtuous”? Are they saying that future AIs will be “woke” — because that’s how I’d understand “virtuous.” This is just inane drivel.

9. Butlerian Jihadists, Unite!

They say that in 2039:

AIs have become increasingly load-bearing in all facets of society, from business to politics to even (some parts of) the armed forces.

Again, the vast majority of humanity will likely revolt against this. Most people aren’t going to want AI to take over “all facets of society.” More probable is that there will be blood (and microchips) in the streets as the Butlerian jihadists fight against the AI takeover.

Conclusion

I could go on, but it would be tedious. “AI 2040” is not a work of serious scholarship. It’s not written by genuinely thoughtful people who are aware of their own biases — of the extent to which their “utopian” vision for the future is rooted in white, Western, capitalist values. (Indeed, all the authors are white men. Any surprise there?)

The document is severely lacking in viewpoint diversity, and the hubris behind their prescriptions for the future is staggering. It’s a symptom of how impoverished the mainstream debates about AI and humanity’s future that documents like this get attention.

What have I missed? What do you disagree with here? How might I be wrong? Please share your thoughts below! As always:

Thanks for reading and I’ll see you on the other side!

He suggested that I start a weekly newsletter. And so, on August 3, 2025, I published my first Realtime Techpocalypse article. Since then, I have published 107 articles, which amounts to slightly more than 2 per week. I am deeply indebted to Gil’s advice — if not for him, this newsletter wouldn’t exist! The least I can do in return is to implore those reading this to buy a pre-sale copy of Gil’s forthcoming book, The Nerd Reich. I haven’t read it yet, but I have absolutely no doubt that it’s amazing. Indeed, given the press that Gil has so far received, it looks like it will be a smash hit. If you have the money, please consider purchasing it asap! Pre-sales really help.

Thank you so much to everyone who’s subscribed to this newsletter. I am incredibly grateful for your interest in my thoughts about technology and, well, all-things-relating to the end of the world. Extra special thanks to those with the means and willingness to pay for this newsletter. It is not profitable opposing Silicon Valley and the TESCREAL ideologies. There are lots of anti-AI TESCREAL folks who are rolling in money from Open Philanthropy, Jaan Tallinn, and other funding entities. But there is almost zero funding for anti-AI folks who are also anti-TESCREAL.

Yet, in one year, I went from earning nothing from this newsletter to bringing in nearly half of the $20k that I need to pay all of my bills. I am deeply appreciative of your financial support, and if you have an extra $7 to spare each month, please consider becoming a paid subscriber. You can also support me on Patreon here, or on GoFundMe here. As I detail in “Why I Fled the US,” I spend virtually all of my money on three things: food (and I never eat out), accommodations (~$700/month or less), and a train ticket once a month to nomadically wander to a new city in Europe (~$60). I am basically an ascetic monk, lol.

Here’s a curated list of highlights from the last year, after which I’ve included a few poll questions about how you’d like to see this newsletter grow and evolve. Thanks again for making this possible, friends.

Subscribe now

The TESCREAL Ideologies (Intellectual History of Silicon Valley):

1. Skeptical of the TESCREAL Acronym? Read This. Probably the most accessible introduction to the TESCREAL thesis that I’ve written (even more so than my original intro published in 2023). “TESCREAL” is a term of intellectual history. It names a constellation of overlapping and interrelated ideologies that have given rise to the Silicon Valley worldview. This worldview, in turn, has launched, sustained, and accelerated the ongoing race to build God-like superintelligence. Highly recommended.

2. Is Transhumanism a Religion? Yes. This article explores how transhumanism was originally introduced as a secular replacement for traditional Christianity. It discusses how the Singularity is analogous to the rapture, and how transhumanism has its own version of resurrection: cryonics. I discuss many more similarities. Transhumanism — the backbone of the TESCREAL bundle — has become the orthodox religion of Silicon Valley.

3. Three Lies Longtermists Like To Tell About Their Bizarre Beliefs. These three lies are: (1) Longtermists care about avoiding human extinction. In fact, they only care about avoiding final extinction (the failure to create posthumanity), not terminal extinction (the disappearance of our species). (2) Longtermist care about the long-term future. If maximizing value requires a shorter rather than longer future, they will choose the shorter future. And (3) longtermists think that future people matter. In fact, people do not matter for their own sake, on the utilitarian-longtermist view. They are mere means to an end — that’s it.

4. Effective Altruism Is a Dangerous Cult — Here’s Why. Even people in the EA community worry that it’s a cult. I believe it’s a cult because I spent many years as an EA, and saw first-hand how cultish it is. Some call it the “Scientology of Silicon Valley.”

5. How Effective Altruists Use Threats and Harassment to Silence Their Critics. This re-examines some of the threats of physical harm and online harassment I’ve received from the EA cult for daring to challenge their views.

6. TESCREALism and the Race to Build an AI God. This introduces my Oxford Research Encyclopedia article on “TESCREAL.” I also argue that “if TESCREALists were to successfully create [their version of] utopia, this itself would be utterly catastrophic for humanity.” The most likely outcome of “utopia” would be (terminal) human extinction.

7. Is AI Apocalypticism Really that Different from Believing in the Rapture? Examines the similarities between religious and AI apocalypticism, i.e., the belief that the world as we know it will soon end. Spoiler: they are very similar!

8. From Trump to TESCREAL, Eugenics Is Everywhere. Eugenics is an idea that just won’t die. This article explores how it’s alive and well within the MAGA movement, as well as Silicon Valley (via the TESCREAL worldview).

   Subscribe now

Pro-Extinctionism in Silicon Valley:

1. One Half a Pro-Human Manifesto. This is essentially chapter 4 of my forthcoming book, Will Humanity Survive? I consider it one of the most important essays I’ve written recently (insofar as anything I write is important). It adduces a plethora of shocking examples in arguing that pro-extinctionist sentiments are ubiquitous within Silicon Valley, even among those who report being on the side of “humanity.” It also argues (in more detail) that the most likely outcome of realizing the utopian world envisaged by TESCREALists is that our species will die out. This, along with #1 above, is essential reading in my opinion.

2. The Perplexing Rise of Antinatalism in Silicon Valley. This examines the strange coexistence of pronatalism and antinatalism within tech culture. It introduces the term “replacement antinatalism,” in contrast to “traditional antinatalism,” to describe the growing demographic of young people who believe it’s “fundamentally unethical” to have children because AI will soon rule the world. Keep your eye on this trend moving forward!

3. If Anyone Becomes Posthuman, Everyone Dies Out. Explores what I call the “coexistence” and “replacement” views, which concern our potential future relationship with posthumanity. I show that the replacement view (i.e., pro-extinctionism) is widely held within Silicon Valley, and argue that the coexistence view will almost unavoidably lead, in practice, to the extinction of our species. Hence, if we want to preserve Homo sapiens — and I claim that we should — we must oppose the creation of posthumanity in all its forms. If anyone becomes posthuman, every human being will perish soon after.

4. The Scariest People in the World: Maniacs, Misanthropes, and Silicon Valley Pro-Extinctionists. Discusses “agential risks,” i.e., the risks arising from agents who would willingly destroy the world if only they could. Building on my earlier academic work (here and here), I add a new category to the list of risky agents: TESCREAL pro-extinctionists who want to replace humanity with AI.

5. Sam Altman: Human Extinction Is Our “Best-Case Scenario.” Discusses my Truthdig article in which I point out that Altman argues our “best chance” of surviving the advent of superintelligence is to “merge” with AI. In other words, the only way to avoid (one type of) extinction is to undergo (another type of) extinction. (See, again, the difference between “terminal” and “final” extinction.)

6. Will Superintelligence “Usher in a Heavenly Golden Age”? Silicon Valley Thinks So. Examines, among other things, the fact that “many people at Anthropic believe that they might be making the next species to succeed us. That maybe humans don’t live after that, and so it’s really important to give Claude good values” — to quote Holly Elmore. Wild stuff!

7. Intro to Pro-Extinctionism (parts 1, 2, and 3). Part 1 examines the “worthy successor” ideology of Daniel Faggella. I show that this ideology is deeply nihilistic. (See this follow-up for reasons his ideology is incoherent and self-defeating.) Part 2 and 3 are less notable, as they’re essentially early drafts of “One Half a Pro-Human Manifesto.”

   Subscribe now

On Specific TESCREALists:

1. Nick Bostrom Took Many of His “Best” Ideas from Other People. It turns out that the TESCREAL luminary, Nick Bostrom, has habitually borrowed ideas from others without proper attribution. I provide 8 glaring examples, though there are many more than this. As the silly saying goes, “good artists copy; great artists steal.”

2. Eliezer Yudkowsky’s Long History of Bad Ideas. Yudkowsky is lauded by people in the cult he founded — Rationalism — as being a genius. He’s even repeatedly described himself as such. Yet he holds some incredibly stupid views: that almost all nonhuman animals have no conscious experience, and hence cannot feel pain; that it probably wouldn’t be a “bad thing” for a child under the age of 6 to die; that he’d be willing to sacrifice “all of humanity” to create “god-like superintelligences … having fun”; and so on. There are good reasons you should have zero respect for the guy.

3. The Political Power of Eschatological Thinking: This Is Why Peter Thiel Is Talking About the Antichrist. Argues that “a true dedication to and belief in eschatological narratives can mobilize people like no other ideology or worldview can, while ‘justifying’ virtually any act to protect, preserve, and accelerate the arrival of paradise — including extreme measures, violence, and even genocide.” Hence, “by tapping into a rich reservoir of powerful images embedded within our collective imagination, images that have played a crucial role in some of the most consequential events of human history, Thiel may hope to mobilize people to join his pro-libertarian fight against the unholy forces of ‘big government.’”

4. Is Sam Altman a Sociopath? — “Highly toxic.” “Psychologically abusive.” “A person of low integrity.” This is how Altman’s colleagues have described him. I also discuss the accusations of abuse made against Altman by his sister, Annie.

5. Preparing for the Singularity: An Ethnographic Survey of AGI Apocalypticists. A fascinating look at how people are preparing for the Singularity. Some have lost all motivation. Some are spending their retirement savings now because they don’t think money will have value in the near future. Others are working extra hard to improve their health based on the belief that if they live long enough to witness the Singularity, they will live forever. A good resource, I think, for historians 50 years from now.

6. The Moral Cowardice of Leading Effective Altruists (and How EA Has Probably Made the World Worse). Argues this: “Claim: EAs are exceptionally ethical people. Prediction: Leading EAs will, therefore, be the loudest voices on the vanguard of speaking out against atrocities like the Gaza genocide, ICE’s brutalization of immigrants, the rise of American fascism, etc. Observation: Crickets from the EA leadership.” Sigh.

7. Eliezer Yudkowsky and the Unabomber Have a Lot in Common. Discusses an interesting chapter of intellectual history: Kaczynski’s manifesto, Bill Joy’s 2000 “Why the Future Doesn’t Need Us,” the transhumanist opposition to Joy’s position (exemplified by Ray Kurzweil), and the way Yudkowsky’s views have shifted over time from aligning with Kurzweil’s to aligning with Joy’s — the latter of whom was partly inspired by the manifesto of Kaczynski.

8. Nick Bostrom’s Pro-Superintelligence Paper Is an Embarrassment. Discusses Bostrom’s hilariously absurd argument that we should plow ahead with building god-like ASI even if there’s up to a 97% chance of total annihilation.

9. Sam Altman Can’t Stop Lying. Altman lied about the Molotov cocktail thrown at his house. He lies about his views on AI and democracy. He lies about virtually everything. I provide a number of egregious examples.

   Subscribe now

Human Extinction (Ethics and History):

1. Making Sense of the “Human Extinction” Debate. This provides a comprehensive but highly accessible introduction to the ethics of human extinction. It introduces a crucial ethical distinction between Going Extinct and Being Extinct, disambiguates the surprisingly polysemous term “human extinction,” and delineates the three major positions one could take on the ethics of our extinction: equivalence views, further-loss views, and pro-extinctionist views.

2. A Revolutionary Paradigm Shift, on Par with the Copernican and Darwinian Revolutions, Is Happening Right Now and No One Is Talking About It. Argues that the realization that human extinction is possible, which has slowly unfolded over the past two centuries, constitutes a profound revolution in our thinking about the nature of humanity and our place in the cosmos on par with the Copernican and Darwinian revolutions. For the entirety of Western history — even when some Ancient Greeks believed that demographic extinction is possible (i.e., the disappearance of our species, which could be permanent or temporary) — virtually everyone accepted the fundamental indestructibility of our species. Recognizing the mortality of our species is a massive paradigm shift, which is still ongoing.

3. Human Extinction: A Brief Guided Tour. This was published before the newsletter was officially born. But I’m including it here because it condenses my 200,000-word book into a “mere” 20,000 words. It’s the tl;dr of my tome.

Other Topics:

1. Chomsky’s Connection to Jeffrey Epstein and Other Sexual Predators. I report on Chomsky’s links to Epstein, as well as Lawrence Krauss. One of the most disheartening to write, as Chomsky was (past tense) a moral hero of mine. I also discussed the issue on Briahna Joy Gray’s podcast.

2. Here’s Why SpaceX’s Mission Is Bullshit. Argues that to achieve its mission of extending the “light of consciousness” to Mars and beyond, SpaceX will have to solve one of the following monumental — perhaps even insurmountable — problems: (1) terraform Mars, despite the fact that we can’t even terraform Earth. Or (2) build machines that are conscious, and devise a highly reliable and accurate “test” for determining whether particular machines are in fact conscious. Those are SpaceX’s two options, and neither appears feasible.

3. A Philosopher Explains Quantum Computers. Provides a roughly 9,000-word introduction to quantum mechanics. If you’re looking for a highly accessible introduction to superposition, entanglement, interference, non-locality, qubits, and computational complexity theory, check this out!

4. Pursuing a Good Life in a Bad World. Explains how I maintain a cheerful disposition despite holding perhaps the most pessimistic view of the world possible, namely, that the “world is a living hell, a disaster zone, a waking nightmare, a horror show overflowing with unfathomable amounts of sorrow, anguish, and heartache.” Relatedly:

5. Are You Suffering from Crisis Fatigue? Why yes, yes we are! Examines the psycho-emotional toll of dealing with the polycrisis.

AI Bullshit:

1. Why You Should Never Use AI Under Any Circumstances for Any Reason No Matter What. It’s funny how the articles (and tweets) I’ve spent the least time on sometimes end up becoming the most popular. With 3,608 likes as of this writing, this is by far my most read article. It compiles a number of “AI bloopers,” which people apparently enjoy chuckling at. :-)

2. GPT-5 Should Be Ashamed of Itself. A companion piece to the article above, published a few weeks prior.

Climate Change:

1. Covering the Alps in Giant Tarps and Other Climate-Related News. Provides an alarming snapshot of the climate crisis. It’s one of several articles I’ve written on the topic, though it hasn’t been a primary subject of this newsletter thus far.

   Subscribe now

Questions for Readers:

I’d love to know more about what you like — and don’t like — about this newsletter (please feel free to comment candidly below). Which topics should I cover more? Would you like me to include short commentary videos in my posts? Are my articles, in general, too long or too short? And so on.

Any other thoughts on how I could make this newsletter better? Any suggestions on what I can do to grow my audience? I’m all ears. :-)

As always:

Thanks for reading and I’ll see you on the other side!

I’ve followed his work since about 2005, and it’s been clear to me for some time that Bostrom considers himself to be a towering intellect of cosmic-historical importance. He is not shy when boasting about his supposed accomplishments, even quoting on his website an article that describes him as the “Swedish superbrain.” For many years, he bragged that — to quote his profile on Edge.org — “his performance as an undergraduate set a national record in Sweden,” though he backed away from this after Simon Knutsson pointed out that there is no evidence to support such a claim. (See the “Potentially Dishonest Self-Promotion” section of this article.1)

According to a TESCREAList named Diego Caleiro, as Caleiro departed FHI many years ago, Bostrom turned to him and “made a gesture like Atlas, holding the world above his shoulders, and quasi letting the world fall, then readjusting.” Caleiro writes that “while funny, I also understood the obvious connotation of how tough it must be to carry the weight of the world like that.”

Yet, because I’m so conversant with the foundational literature of the TESCREAL movement, I’ve noticed that a large number of the “original,” “novel,” even (supposedly) “groundbreaking” ideas attributed to Bostrom actually originated with other thinkers. As early as 2018, I was posting about this on social media and privately sharing examples with colleagues — some of whom were rather shocked to see Bostrom’s work juxtaposed with that of others (from whom he borrowed without sufficient attribution). I wrote to one colleague that Bostrom’s “genius” seems to be taking ideas from other theorists and repackaging them in his own language such that they’re just different enough for him to avoid being accused of intellectual plagiarism. His oeuvre reminds me of the line, often misattributed to Picasso: “Good artists copy; great artists steal.”

Here are just a few examples. Don’t take my word for it — decide for yourself:

1. Existential Risk Scenarios

1.1 Bostrom’s 2002 Paper

Perhaps the most “famous” paper written by Bostrom is his 2002 “Existential Risks: Analyzing Human Extinction Scenarios and Related Hazards.” I first read this circa 2005 and was struck by its breadth and originality. Bostrom offers a technical definition of “existential risk,” which denotes any event that would prevent us from creating a (utopian) posthuman civilization. (Hence, the term is intimately bound up with his version of transhumanism.)

About half of the paper is a detailed classification of different types of existential risk scenarios, such as2:

1. Deliberate misuse of nanotechnology and a nanotechnology accident (such as the gray goo scenario).

2. Nuclear holocaust.

3. Misaligned superintelligence.

4. Engineered pandemics.

5. Natural pandemics.

6. Physics disasters, e.g., vacuum decay.

7. Asteroid or comet collisions.

8. Runaway climate change.

9. Fundamentalist religious or ecological movement that halts technological development.

10. An extraterrestrial civilization kills us.

11. “Something unforeseen,” because “it would be foolish to be confident that we have already imagined and anticipated all significant risks.” (Take note of the wording.)

12. Examples of the above include “solar flares, supernovae, black hole explosions or mergers, gamma-ray bursts, galactic center outbursts, supervolcanoes,3 loss of biodiversity, buildup of air pollution, gradual loss of human fertility, and various religious doomsday scenarios.” (Take note of each of these threats.)

13. Resource depletion or ecological destruction.

14. Totalitarianism.

15. Insurmountable technological hurdles (that prevent us from developing advanced technologies).

16. Our simulation gets shut down. (Lolz)

17. Dysgenic pressures, whereby less “intelligent” people outbreed their more “intelligent” peers. (Bostrom is a eugenicist.)

He then discusses the Fermi paradox, anthropics, observation selection effects, and the Doomsday Argument. Taking these in order:

The Fermi paradox (which is neither a paradox nor Fermi’s!) points out that it’s surprising we don’t see evidence of technologically advanced civilizations given the age and size of the universe. Hence, Bostrom writes that

there must be (at least) one Great Filter — an evolutionary step that is extremely improbable — somewhere on the line between Earth-like planet and colonizing-in-detectable-ways civilization. If the Great Filter isn’t in our past, we must fear it in our (near) future. Maybe almost every civilization that develops a certain level of technology causes its own extinction.

Observation selection effects concern the question of “what we should expect to observe given some hypothesis about the distribution of observers in the world.” For example, you might think that a large asteroid striking Earth is improbable because we haven’t witnessed one strike Earth in the past 300,000 years (the period during which our species has existed). But if an asteroid had struck Earth during this time, we wouldn’t be here to talk about it having happened. Hence, we will only ever find ourselves in worlds where asteroids haven’t struck in the recent past. This means that the evidence available to us will be biased in certain ways.

The Doomsday Argument is an a priori argument for why human extinction is more probable than we’d otherwise expect — that is, based on empirical observations. The reasoning is interesting: basically, it’s more likely that we’d find ourselves in a large population at the end of history than in the much smaller population of people long ago, when human history was just beginning. We should thus increase our probability estimates of a secular apocalypse happening soon.

1.2 John Leslie’s 1996 Book

Again, I was extremely impressed with Bostrom’s paper when I first read it. It was original and erudite — even polymathic.

But then I read the introductory chapter of John Leslie’s 1996 book The End of the World and was utterly shocked by how similar Bostrom’s paper is to it. (And, as I’ll explain in a moment, we know that Bostrom was very familiar with Leslie’s book.) For example, although Leslie doesn’t use the term “existential risk,” the central topic of his tome is human extinction — the most obvious type of existential risk. In the introduction, he lists scenarios like the following that could potentially have annihilatory consequences4:

1. A disaster from nanotechnology (including the “gray goo” scenario).

2. Nuclear war.

3. Computer-related disaster (e.g., computers replacing us).

4. Engineered pandemics (i.e., “a disaster from genetic engineering,” including “green scum” and altered microorganisms that could render women infertile).

5. Natural pandemics.

6. Physics disasters, e.g., vacuum decay.5

7. Asteroid or comet collision.

8. Runaway climate change.

9. Religious or philosophical ideologies that prevent humanity from surviving, including Schopenhauerian pessimism, ethical relativism, negative utilitarianism, and “person-affecting” ethical theories.

10. Extraterrestrials kill us.

11. “Something-we-know-not-what,” because “it would be foolish to think we had foreseen all possible natural disasters” and “we cannot possibly imagine every single danger which technological advances might bring with them.” (Sound familiar?)

12. Nearby supernova or black hole merger, volcanic supereruptions, pollution, ozone depletion, etc.

13. Extreme ice age.

14. Breakdown of complex systems.

15. Unwillingness to have children.

16. Biological or chemical warfare.

17. Some other kind of technological disaster.6

These lists are clearly very similar. My impression at the time of first reading Leslie was that the bulk of Bostrom’s paper is little more than a summary of Leslie’s opening chapter. Bostrom even uses similar language. For example, in adding “unknowns” sections at or toward the end of each list (exactly as Leslie does), he writes: “It would be foolish to be confident that we have already imagined and anticipated all significant risks.” Compare that with what Leslie says: “It would be foolish to think we had foreseen all possible natural disasters … [and] every single danger which technological advances might bring with them” (italics added to both quotes).

Even more, Leslie then proceeds to discuss the Fermi paradox, anthropics, observation selection effects, and the Doomsday Argument (which he partly developed) — exactly as Bostrom does, except years before Bostrom wrote his paper.

These similarities aren’t a coincidence: Bostrom wrote his PhD dissertation on the topic of anthropics, and relied on Leslie’s 1996 book — mentioning Leslie’s name 135 times and citing the book throughout. Bostrom even thanks Leslie in the Acknowledgments section.

So far as I can tell, the only thing that’s nontrivially original about Bostrom’s 2002 paper is the existential risk concept, which he introduced to capture an idea that was, as it happens, already being discussed by his online Extropian buddies. Whereas Leslie focuses specifically on human extinction, Bostrom recognized that there are a range of survivable scenarios that would also prevent us from creating a posthuman utopia full of “astronomical” amounts of value. That’s not nothing, but it’s not a lot, either.7

2. Where Are They?

In 2008, Bostrom published an article in the MIT Technology review titled “Where Are They? Why I Hope the Search for Extraterrestrial Life Finds Nothing” — which he presumably got paid $1 to $2 per word for.8 His central thesis is that “I hope that our Mars probes will discover nothing. It would be good news if we find Mars to be completely sterile. Dead rocks and lifeless sands would lift my spirit.” He continues:

Conversely, if we discovered traces of some simple extinct life form – some bacteria, some algae – it would be bad news. If we found fossils of something more advanced, perhaps something looking like the remnants of a trilobite or even the skeleton of a small mammal, it would be very bad news. The more complex the life we found, the more depressing the news of its existence would be. Scientifically interesting, certainly, but a bad omen for the future of the human race.

Why does he say this? Given the Fermi paradox, there must be at least one Great Filter that prevents life from developing far enough to colonize the universe and make its presence known. This Great Filter could be behind us or ahead of us.

For example, it could be that the step from non-life to single-celled living organisms (a process called abiogenesis) is extremely unlikely. If so, then we have gotten very lucky — and the mystery of why we don’t see technologically advanced civilizations flitting about the galaxy, given the age and size of the universe, makes perfectly good sense. Or it could be that the step from single-celled organisms to multicellular lifeforms is extremely improbable, or that the subsequent emergence of big-brained creatures like us is highly unlikely.

Alternatively, it could be that all of these steps are fairly probable — i.e., none constitute a Great Filter. Instead, the universe is teaming with big-brained creatures that, just like us, build industrial civilizations, destroy their environments, invent nuclear weapons, run dangerous physics experiments, etc. and then promptly destroy themselves before having initiated a colonization explosion. If that’s the case, then the Great Filter lies ahead of us, meaning that we’ll soon go extinct due to rogue superintelligence, climate breakdown, nuclear war, etc.

What does this imply? It implies that finding evidence of simple lifeforms in the universe — including on our planetary neighbor, Mars — will shift the probability of the Great Filter being behind us to it being ahead of us. If it turns out that simple lifeforms are common, then the Great Filter probably isn’t the step of abiogenesis. That means that the probability of the Great Filter being in our (near) future rises — which is, of course, very bad news for us! It means we might all die in some kind of extinction event in the coming years or decades.

That’s Bostrom’s ostensibly original point. Throughout the article, on my reading, he repeatedly presents this idea as his own. (Read it for yourself and let me know what you think.) “So this is why I’m hoping,” he concludes, “that our space probes will discover dead rocks and lifeless sands on Mars, on Jupiter’s moon Europa, and everywhere else our astronomers look” (italics added). “It would keep alive the hope for a great future for humanity.”

Yet this isn’t his idea at all. It’s clearly included in the 1998 article titled “The Great Filter — Are We Almost Past It?” by Robin Hanson, which introduced the idea of the Great Filter. Hanson writes:

While all of these stories [about how each evolutionary step leading from non-living matter to space colonization could happen] are at least minimally plausible, our main data point implies that at least one of these plausible stories is wrong — one or more of these steps is much more improbable than it otherwise looks. If it is one of our past steps, such as the development of single-cell life, then we shouldn’t expect to see such independently evolved life anywhere within billions of light years from us. But if it is a step between here and a choice to explode that is very improbable, we should fear for our future. At the very least, our potential would have to be much less than it seems. Optimism (as defined here) regarding our future is directly pitted against optimism regarding the ease of previous evolutionary steps. To the extent those successes were easy, our future failure to explode is almost certain.

In other words, if the steps from non-living matter to single-cells, from single-cells to multicellular beings, and from multicellular beings to big-brained creatures are “easy” — if there is no Great Filter making them highly improbable — then we should fear for our future, because it implies that the Great Filter must be ahead of us. Since finding life on Mars would suggest that there isn’t a Great Filter in our evolutionary past, then suddenly the probability that it lies in our future rises.

I read Bostrom’s article before Hanson’s, and once again I believed that Bostrom was articulating a novel insight. But he wasn’t. Yet Bostrom only once mentions Hanson in the MIT Technology Review paper — not to acknowledge that Hanson made the same point, but to parenthetically attribute the term “Great Filter” to him. Bostrom writes: “I borrow this term from Robin Hanson, an economist at George Mason University.”

Meanwhile, over on Bostrom’s personal website, he posted a slightly different version of the article that includes a footnote that says: “I borrow this term from Robin Hanson’s ‘The Great Filter — Are We Almost Past It?’ … a paper which presents an argument similar to the one expounded here.”

If you were to miss that brief footnote in the version posted on his personal website — the only place where he acknowledges the unoriginality of his thesis — then you’d naturally think Bostrom’s article is making a novel point. Readers of the MIT Technology Review most likely did, as did I when I first read it. Turns out Bostrom was just echoing Hanson’s idea.

And for this, he received somewhere between $4,692 and $9,384 (again, $1 or $2 per word), assuming that MIT Technology Review paid writers the same back in 2008 as it does now. That’s a lot of money for rehashing someone else’s point without even noting, as he does in the personal-website (but not officially published) version, that Hanson “presents an argument similar [nay, identical in essence] to the one expounded here.”

3. Superintelligence

In 2014, Bostrom published a surprise bestseller titled Superintelligence: Paths, Dangers, Strategies. It offered a comprehensive overview of the argument for why ASI could pose a dire existential risk. Indeed, Bostrom contends that the “default outcome” of ASI is “doom,” because figuring out how to “align” ASI with the “values” of humanity (by which he really means the values of the TESCREAL worldview) poses enormous technical and philosophical challenges.

This book was hailed as an important contribution to the study of ASI existential risk. Elon Musk endorsed it, as did Bill Gates. Sam Altman called it “the best thing I’ve seen on this topic. It is well worth a read.”

Yet it turns out that much of the book is little more than a summary of ideas outlined by Bostrom’s TESCREAL colleague Eliezer Yudkowsky (who was himself building upon earlier writers like IJ Good, Verner Vinge, etc.). As Ben Goertzel writes in a review of the book, “nearly all the core ideas of Bostrom’s work appeared previously or concurrently in Yudkowsky’s thinking.” In a section titled “Scratch the Surface of Bostrom, Find Yudkowsky,” he reports that

my first reaction, as I paged with curiosity through Bostrom’s gloomy, erudite tome, was to be struck by the remarkable similarity between the arguments, and the general attitude presented, and the perspective on the future of AI put forth by Eliezer Yudkowsky over the last 15 years or so. Yudkowsky has not (yet) written any single document as thorough and carefully-wrought as Superintelligence. But in mailing list and blog posts and online essays over the years, he has articulated a perspective very close to Bostrom’s, and in many ways possessing more depth and nuance.

Goertzel adds that

with only mild oversimplification, one might say that Bostrom’s Superintelligence is a triumphal work synergizing Bostrom’s communication and analytical ability and Yudkowsky’s creative ideas. Of course, this oversimplification isn’t quite fair to Bostrom, who contributed a lot of detailed ideas to his book as well. But as for the Big Ideas in Superintelligence, it really seems that every one was articulated at some point previously by Eliezer Yudkowsky.

So, Bostrom gets some credit for “a lot of detailed ideas,” but the most significant ideas in the book came from Yudkowsky — pictured below.

4. Instrumental Convergence vs. AI Drives

It’s not just Yudkowsky that Bostrom drew from when writing about ASI existential risk. A central premise of the argument for ASI x-risk is what Bostrom calls the “instrumental convergence thesis.”

Bostrom first delineated this thesis (so far as I know) in his 2012 article, “The Superintelligent Will: Motivation and Instrumental Rationality in Advanced Artificial Agents.” It states that agents with a sufficiently high level of “intelligence” will predictably pursue a finite set of intermediary or “instrumental” goals, given a wide range of final goals (i.e., the ultimate goals the ASI wants to achieve). Such instrumental goals include:

• Self-preservation: ensuring that it stays alive, because an agent can’t reach its final goals if it perishes.

• Goal-content integrity: ensuring that it’s goals stay the same over time, because an agent at T1 can’t reach its final goals if it allows those final goals to change at some later time T2.

• Cognitive enhancement: improving its cognitive abilities, including its rationality, as this will help it find new and better ways to attain its final goals.

• Technological perfection: create ever-more powerful technologies, as this will help it bend the world toward realizing its final goals.

• Resource acquisition: acquire as many resources as possible, as resources are necessary for achieving most final goals that it might have.

Whether the final goal of an ASI is curing cancer, calculating the 100-millionth digit of pi, or converting the entire universe into data centers, we can predict — the claim goes — that in trying to achieve such goals, the ASI will attempt to preserve itself, prevent its goals from being altered, improve its cognition, perfect technology, and acquire maximal resources. This is the instrumental convergence thesis.

Once more, I came away from this paper after reading it 10+ years ago thinking that it was yet another publication of notable originality from Bostrom. Only later did I notice a footnote hidden at the bottom of one of the pages that says a guy named “Stephen Omohundro has written two pioneering papers on this topic,” one being “The Basic AI Drives,” published in 2008 — four years before Bostrom’s paper and six years before Superintelligence. Bostrom goes on to say that he doesn’t like Omohundro’s “AI drive” terminology. He thinks “instrumental goals” is better. He then adds at the very end of this footnote that his “treatment here also differs from that of Omohundro in some other more substantial ways, although the underlying idea is the same.”

I would agree that “the underlying idea is the same,” because it obviously is. For example, in Omohundro’s paper, he identifies the following as AI drives (i.e., instrumental goals) that any sufficiently intelligent system will almost certainly pursue:

• “A strong drive toward self-preservation.” Again, if the AI ceases to exist, it won’t be able to achieve its final goals, which means it will take steps to ensure its survival.

• “AIs will try to preserve their utility functions,” which Bostrom calls goal-content integrity.9

• Self-improvement, such as developing “more efficient algorithms, more compressed representations, and better learning techniques” to augment its cognitive abilities. This also means the AI systems will try to make themselves more rational. In other words, cognitive enhancement.10

• “AIs will want to acquire resources and use them efficiently,” given that “all computation and physical action requires the physical resources of space, time, matter, and free energy.”

Bostrom’s list isn’t entirely identical with Omohundro’s. But it’s very close. Yet, when TESCREAL folks talk about the topic, almost no one cites Omohundro. Some aren’t even aware that he published an article strikingly similar to Bostrom’s. Rather — as with all the examples above — Bostrom gets most of the credit, as if he was the first to hit upon these ideas.

5. Letter From Utopia Vs. Letter to Mother Nature

Back in 2008, I emailed one of the founding members of the Extropian movement. (Bostrom was once an Extropian.) In the email, I mentioned Bostrom’s essay “Letter from Utopia,” as I was using it for an art project. The Extropian liked my art project but said:

A suggestion is to read Max More’s “Letter to Mother Nature” [1999] written years before Nick grabbed onto the idea and then did his own version with no credit or mention of Max. … I just wish that you were using the original idea and the original letter because that would give your [work] integrity and honesty.

Bostrom published “Letter from Utopia” in 2008 (though he claims to have first circulated a draft in 2006), while More read his “Letter to Mother Nature” at an Extropianism conference in 1999. Not only are the titles similar, but the content overlaps considerably. For example, More lists seven “amendments” to the “human constitution,” namely:

1. Overcome aging and death.

2. Expand perceptual range (new senses).

3. Enhance cognition.

4. Supplement the neocortex with a “metabrain,” i.e., a “distributed network of sensors, information processors, and intelligence.”

5. Use genetic engineering to control biological and neurological processes.

6. Refine emotions and improve rationality.

7. Control our biochemistry and integrate technology into our bodies.

The goal is for “these amendments to our constitution [to] move us from a human to a posthuman condition as individuals.” Along exactly similar lines, Bostrom’s “letter” is written by a fictional posthuman who declares that we could become posthuman if only we pursue “three fundamental transformations,” namely:

1. Overcome aging and death.

2. Enhance intelligence and perceptual range. (“And then ask yourself, what other harmonies are there in the air, that you lack the ears to hear? What vaults of value are you witlessly debarred from, because you lack the key sensibility? … Your brain must grow beyond the bounds of any genius of humankind, in its special faculties as well as its general intelligence …”)

3. Elevate well-being. (Maximize our ability to experience pleasure. Increase our rationality: “All emotions have a function. Prune and weed carefully lest you accidentally reduce the fertility of your plot.”)

Not once does Bostrom cite More’s earlier “letter” as a source of inspiration. Not once does he indicate that his “letter” is strikingly similar to More’s. Yet almost no one cites More; instead, the citation goes to Bostrom.

6. Astronomical Waste

One of the foundational papers of longtermism — the claim that we should become posthuman, colonize space, and flood the universe with digital beings living in giant computer simulations — is Bostrom’s 2003 paper “Astronomical Waste: The Opportunity Cost of Delayed Technological Development.” In 2022, Elon Musk even retweeted a link to this paper along with the line (in the original tweet): “Likely the most important paper ever written.”

This paper played a huge role in my conversion to the longtermist/TESCREAL worldview. But, once again, I was later shocked to discover that the central idea wasn’t Bostrom’s.

In brief, the paper argues that if we colonize space — especially as digital posthumans — the future population could be enormous. Bostrom estimates some 10^38 digital people within the Virgo Supercluster, a cluster of galaxy clusters that includes our own Milky Way. (That’s a 10 followed by 38 zeros, by the way.)

If one accepts a utilitarian perspective, this means the expected value of ensuring the realization of all those future people (assuming they have net-positive lives) far exceeds the value of, e.g., helping poor people today. If you’re a truly altruistic person who aims to maximize your philanthropic impact, then you should focus not on present-day people but those living perhaps millions or billions of years from now. It’s not that future digital people are inherently more valuable. It’s just a numbers-game: there are so many more of them (potentially) than present-day people, they overwhelm the simplistic equations of utilitarianism’s moral mathematics.

Bostrom’s conclusion is that we should colonize the universe as soon as possible, because every single century, decade, and even second of delaying this results in an “astronomical” loss of potential value — i.e., the non-existence of people who could have otherwise been created in some distant galaxy if only we had started colonizing just a bit sooner. However, Bostrom also concludes that “priority number one, two, three and four should … be to reduce existential risk.” Space colonization should therefore be priority number five once we secure our survival. Perhaps you can see why Musk loves this paper.

In the second paragraph of Bostrom’s paper, he writes that “one recent paper speculates, using loose theoretical considerations based on the rate of increase of entropy, that the loss of potential human lives in our own galactic supercluster is at least ~10^46 per century of delayed colonization.” This cites a paper by Bostrom’s colleague, Milan Ćirković, titled “Cosmological Forecast and its Practical Significance,” published one year before “Astronomical Waste.” Curious, I read Ćirković’s piece and was quite surprised to find that he makes an almost identical argument to that of Bostrom, though the details and framing are different.11 At the risk of going into too much detail, here’s what I write about this paper in my 2024 book:

Ćirković argued that physical eschatology paired with recent developments in anthropics (from both [John] Leslie and Bostrom) carry potentially urgent practical implications for “intelligent observers interested in self-preservation and achieving [the] maximum of its creative potential.” Specifically, the timing of space colonization could make a significant difference to the amount of resources available to advanced civilizations, given our best current understanding of our “cosmological situation.”

That’s exactly what Bostrom argues. Even more:

As [Ćirković] expressed the idea, decision-making performed today, as far as humanity is concerned, may have enormous consequences on very long timescales. In particular, an overly conservative approach to space colonization and technologization, may result (and in fact might have already resulted) in the loss of [a] substantial fraction of all possible observer-moments humanity could have had achieved. The term “observer-moment” comes from Bostrom, who defined it as “a brief time-segment of an observer.” Thinking in terms of observer-moments instead of observers is relevant to anthropics — that is, to reasoning about one’s location in space and time based solely on the fact that one is an intelligent observer. But it may also be relevant to estimating our “potential” over time: since “different observers may live differently long lives, be awake different amounts [of] time, … etc.,” counting observer-moments rather than observers can give a more accurate measure of the bigness or value of the future. There could, after all, be a large number of observer-moments even if there are relatively few observers, and vice versa.

The point is that Ćirković used this concept to argue that we should develop advanced technologies and colonize the universe as soon as possible, given that every passing moment our cosmic endowment of negentropy is going to waste as stars burn up their limited reservoirs of hydrogen. Consequently, by failing to advance technology and colonize space, we could lose — and maybe have already lost — a “substantial fraction of all possible observer-moments [that] humanity could have … achieved.” To underline the time-sensitivity of this situation, Ćirković calculated that if a “future hypercivilization” could extract all the energy output of the stars populating the Virgo Supercluster, then “the number of potentially viable human lifetimes lost per century of postponing … the onset of galactic colonization is … 5 × ~10^46.” Thus, assuming some correlation between the number of lifetimes or observer-moments and the fulfillment of our “creative potential,” this provides a very strong prima facie reason against “an overly conservative approach to space colonization and technologization.” (Italics and a paragraph break were added to the original text.)

What’s different between the central arguments and conclusions of Bostrom and Ćirković? Not a ton. Both calculate the costs of postponing space colonization. Both provide estimates of how many people there could be in the Virgo Supercluster. Both argue that we should therefore accelerate the development of technology and colonize space as soon as possible. Bostrom says that the loss of value with every century of delayed colonization “boggles the mind,” while Ćirković writes in similar language that the loss “per a century delay in starting the colonization is astonishing by any standard.”

The biggest difference is that Bostrom frames the issue in terms of “existential risk” (a word that Ćirković doesn’t use) and provides a more explicit ethical foundation — utilitarianism — to support the conclusion he shares with Ćirković. For years, I was under the impression that Bostrom was doing a lot more in his paper than recapitulating Ćirković’s idea of calculating how many future people there could be and then arguing that we should colonize space ASAP to prevent “astronomical waste.”

7. Paperclip Maximizer

If you look up “paperclip maximizer” on Wikipedia, you’ll find a section that opens with this: “The paperclip maximizer is a thought experiment described by Swedish philosopher Nick Bostrom in 2003.” But this is misleading, as it suggests that Bostrom is its progenitor. Indeed, while the idea is regularly attributed to him, it appears that Yudkowsky likely first came up with it.

In a 2018 interview, Sam Harris asks Yudkowsky: “Does this thought experiment originate with Bostrom, or did he take it from somebody else?” Yudkowsky replies: “As far as I know, it’s me.” He adds that it might still be Bostrom’s idea, though he was “the first one to say ‘paperclips’” if the idea originated on mailing lists that he was a part of. (No one, to my knowledge, has double-checked this.)

It’s fine that Yudkowsky doesn’t much care about attribution, but contrast his openness about this issue to Bostrom’s. I have never once seen Bostrom suggest in any interview that he might not have invented the thought experiment. (Let me know if I’m missing something.) He never seems to skip an opportunity to take credit for an idea attributed to him, whether or not that idea is actually his.

(For some useful information that calls into question this example, see this comment from Hypnosifl below.)

Similarly:

8. Pascal’s Mugging

This refers to a thought experiment that shows how maximizing expected value (or utility) — which some would say is integral to rationality — can lead one to make decisions that appear irrational. Expected value is (roughly speaking) what you get when you multiply the probability of an outcome by its value. For example, an outcome with a value of 10 and a probability of .5 has the same expected value as an outcome with a value of 1,000 but a probability of 0.005. (That’s because 10 x .5 = 5, and 1,000 x 0.005 = 5.)

Pascal’s mugging results when enormous values are paired with tiny probabilities. Mitigating speculative existential risks provides an example: if there could be 10^58 happy digital people in the future, as Bostrom calculates, then it might be worth spending huge amounts of money mitigating a highly speculative existential risk scenario even if one judges the probability of that scenario to be tiny. That is to say, multiplying the payoff (value) of the outcome (10^58 happy people) by the (low) probability could still yield a very high expected value.

In 2009, Bostrom published a paper on Pascal’s Mugging in the prestigious philosophy journal Analysis. But the idea was first introduced by Yudkowsky in a 2007 blog post titled “Pascal’s Mugging.” Bostrom briefly gestures in the direction of this fact in a footnote, saying: “Related scenarios have recently been discussed informally among various people. Eliezer Yudkowsky named the problem ‘Pascal’s mugging’ in a post on the Overcoming Bias blog.” Yet it was Bostrom’s name on the Analysis paper, which he was then able to add to his already “impressive” CV, thus boosting his academic “credibility.”

In Yudkowsky’s version, a mugger says to their intended victim: “Give me five dollars, or I’ll use my magic powers from outside the Matrix to run a Turing machine that simulates and kills [an unfathomably large number of] people” (italics added). Even if one deems the mugger’s claim to be extremely improbable, this low probability is offset by the hugeness of the number of people who would be killed. Hence, expected value theory implies that one should hand over the money.

In Bostrom’s version, the mugger says: “I have magical powers. I can give you any finite amount of money that you might ask for tonight. What’s more, I can give you any finite amount of Utility that I choose to promise you tonight” (italics added). Rather than threatening to simulate and kill people, the mugger offers to give the victim a phenomenal amount of money tomorrow if they hand over their wallet right now.

It’s fairly well known that Yudkowsky originally devised the scenario, and that Bostrom basically just re-worded and published it a highly reputable philosophy journal. However, I’d consider it rather odd — and problematic in an ethical sense — if a friend of mine wrote a blog post about an idea, and then I took that idea, changed some minor details, and managed to get a major publication out of it. What do you think?

Subscribe now

A Project of Self-Aggrandizement

There are many more examples that could be adduced.12 The point isn’t to shame Bostrom for drawing from other scholars. Every academic does this, of course — it’s the way intellectual progress proceeds. I stand on the shoulders of the giants who came before me, and the next generation will stand on my shoulders by building upon whatever newish ideas (if any) I might have been lucky enough to stumble upon in my brief time here on Earth.

However, in most cases, academics are pretty good at properly attributing the origins of ideas they further develop or elaborate. I do not see this in Bostrom’s works at all. My sense, from reading him since ~2005, is that he goes out of his way to hide the fact that many of his “best” ideas originated in the work of his colleagues. Nowhere in his writings does he say things like the following:

• “Reproducing much of John Leslie’s taxonomy of extinction hazards, I will enumerate various scenarios that give rise to what I call ‘existential risk.’”

• “As Robin Hanson alluded to, it would be very bad if we were to discover life elsewhere in the universe, given the Great Filter framework.”

• “Drawing heavily from the unpublished work of Eliezer Yudkowsky, I will outline an argument for why ASI could bring about the annihilation of humanity.”

• “Developing an idea introduced by Stephen Omohundro, I will examine what I call ‘instrumental goals’ that ASI will naturally pursue.”

• “Echoing Max More, this ‘letter’ outlines various projects we should pursue in order to become posthuman.”

• “I will argue, as Milan Ćirković does, that every century of delayed space colonization results in an astronomical loss of human potential/value.”

• Etc.

The vast majority of scholars would have included such remarks if they had written Bostrom’s papers. It’s standard to not take credit for other people’s intellectual labor; it’s common to be generous in citing others. (Plenty of my own scholarship draws from other people, sometimes heavily. But when I do this, I make it clear that I’m relying on their insights. Nothing special here — precisely because it’s standard practice!)

Yes, Bostrom does cite the people above, but not in a way that properly conveys the extent to which his work takes from them — or so I would argue. Pair this with the large volume of evidence that Bostrom thinks of himself as a super-genius of almost messianic significance — he’s saving not just the world, but the entire future of posthumanity — and it starts to look like these weren’t exactly innocent mistakes.

From my perspective, Bostrom is at best a second-rate thinker whose real talent is repackaging other people’s insights in accessible-yet-academic language that appeals to a wide audience. This is what he’s good at — and we should give him credit for that. As Goertzel notes, the guy “is a very good writer, mixing academic rigor with occasional color and humor in a classic European style.”13 But for originality, creativity, and insight, Bostrom gets a D- in my gradebook.

One can see this — or at least I do — as part of a life-long project of self-aggrandizement. And it’s been largely successful: Bostrom has become “one of the most-cited philosophers in the world,” as he boasts on his website. This is, however, thanks to the success of publications like those mentioned above: Superintelligence (12,152 citations on Google Scholar), “Existential Risk” (1,392 citations), “The Superintelligent Will” (674 citations), “Astronomical Waste” (402 citations), “Letter from Utopia” (219 citations), “Pascal’s Mugging” (104 citations), and so on. On his website, Bostrom adds that his

writings have been translated into more than 30 languages; he is a repeat main-stage TED speaker; he has been on Foreign Policy’s Top 100 Global Thinkers list twice and was included in Prospect’s World Thinkers list, the youngest person in the top 15.

This all looks rather less impressive when you realize that the reason he has so many “accomplishments” is because of publications that drew heavily, often without sufficient attribution (in my opinion), from others.

***

While writing this penultimate section, I decided to look into Bostrom’s boast that he’s among the most-cited contemporary philosophers. When Gemini answered my Google query about this, it told me the following:

The claim that Nick Bostrom was the “most-cited professional philosopher under the age of 50” comes directly from his official professional biography used across academic institutions, speaker bureaus, and publishers.14

Oh, so the biography that he wrote is the citation for a claim in his biography! After more digging, I found this ranking from after 2020, which lists him as the 58th most-cited philosopher at the time. Two thoughts:

1. Does 58 really count as “one of the most-cited philosophers in the world”? I’d have thought this would be reserved for the top 10, or perhaps the top 50. Bostrom is neither in the top 10 nor top 50.

2. I have never once seen any other philosopher above Bostrom on the list bragging about their ranking. (Have you?) Consider that David Chalmers — who is lionized within philosophy — is listed as #10 in the rankings. Yet nowhere on his website does he mention this. (According to Google Scholar, Chalmers has nearly twice as many citations as Bostrom.) The same goes for Timothy Williamson at Oxford, listed as #8. Even Martha Nussbaum, who’s the most-cited living philosopher, says nothing about it on her website or CV. Why? Because, while academics are expected to proudly list their awards and publications, overt self-promotion is often seen as bad form. It’s cringe-inducing and doesn’t reflect well on the humility of the individual.

But Bostrom, once again, never misses an opportunity to exaggerate and promote himself. It’s what his life-long project of self-aggrandizement is all about.

Here’s another example:

On his personal website, Bostrom boasts that his book Deep Utopia, which was published with IdeaPress Publishing, a pay-to-publish company, has won a whole bunch of awards. As of August 7, 2026, he writes:

Chinese translation of Deep Utopia is now out. Also the second print run in English and the Audiobook. The book has received still more awards.

Such awards include:

But if you look into these awards, most of them are trivial — you might even say bullshit — “awards” given to dozens and dozens of books. For example, over 70 books received the “2024 PenCraft Book Award.” In the same “fiction” category that includes Bostrom’s book (even though his book is supposed to be nonfiction!), one finds titles like these:

And:

Another “award” that Bostrom’s book won is the “Living Now Book Awards.” It’s official website says:

The Living Now Book Awards feature over 30 categories, celebrating exceptional lifestyle, homestyle, personal development, wellness, and Christian-themed books

Another website, independentpublisher.com, describes the award as follows:

The Living Now Book Awards celebrate the innovation and creativity of newly published books that enhance the quality of our lives, from cooking and fitness to relationships and mature living.

For an academic, this is an embarrassment.

Moving on to the “Independent Press Awards” that Bostrom also boasts of having won, this is what you’ll see if you click on their website. Amid dozens and dozens of books, you’ll see Bostrom’s about halfway through:

The obvious interpretation of Bostrom’s website (screenshot above) is that his book was singled-out as an outstanding work of literature by various award-granting groups. But this is completely misleading: these are, again, bullshit awards given to tons of books.

Yet Bostrom can’t resist mentioning them because — like Donald Trump — he is an insecure narcissist who’s obsessed with exaggerating his accomplishments for the purposes of self-aggrandizement. Trump has his fake TIME magazine cover; Bostrom has these awards, his fabricated academic “record” in Sweden, his thousands of citations for papers containing unoriginal ideas, etc. etc.

Why Has No One Said Anything?

An interesting question is why the people from whom Bostrom has borrowed so heavily without proper attribution have been okay with this. Literally no one, to my knowledge, has complained about intellectual plagiarism. Why not? I brought this up to a former researcher at Bostrom’s Future of Humanity Institute (FHI), who responded as follows:

My view, for what it’s worth, is that Bostrom has played fast and loose with the ideas of people who were happy to be plagiarized because they thought “he’s going to be a big Oxford philosopher and having someone like that in our pocket will be good for us and the things we care about so let’s all just go along with it.” I.e. I think maybe you [Émile] are not being hard enough on the people who have let Bostrom plagiarize them without protest, because they only did it because it was in their interests to do so, they weren’t just being charitable to a sub-genius with an ego trip!

On this view, personal self-interest is why no one has spoken out. That seems plausible to me. However, the fact that others are okay with Bostrom taking their ideas doesn’t make it right. Consider a friend of mine who writes an excellent philosophy paper and then says to me (in some parallel universe): “I know you need another publication for tenure, so feel free to put your name on this one and publish it as yours. We’ll never mention this to anyone.” If I were to publish their paper as mine, I would still be committing academic fraud — even if I had their full permission. Standards don’t dissolve if consent is granted.

So, it’s a bit frustrating that no one has said anything. If the people above were to publicly complain, then (a) they’d have a very good case for being upset, and (b) the result could well be a significant scandal for Bostrom. As far as I know, this article is the first to lay out the evidence and make that case. Perhaps it will inspire someone to speak out, especially now that Bostrom is no longer at Oxford.15 We’ll see.

Subscribe now

Have you come across important examples that I’ve missed? Please do share them below (or feel free to email me). As always:

Thanks for reading and I’ll see you on the other side!

ADDITIONAL CONCERNS (added August 6, 2026)

In Deep Utopia, Bostrom explicitly thanks ChatGPT-4 and Claude 2 “for extensive comments on the manuscript.” Curious, I ran the abstract of his recent paper “Optimal Timing for Superintelligence” (which argues that we should build ASI even if there’s up to a 97% chance of everyone on Earth dying) through Pangram and another AI detection program. This was the result:

Pangram is rarely wrong when it comes to flagging something as being AI-generated, so it’s safe to assume that Bostrom did indeed use AI to write the abstract. I haven’t checked the rest of his paper, or his Deep Utopia, because I don’t have the money to pay for Pangram — but if others are inclined to do this, please let me know what the results are.

Also, user Hypnosifl added some additional examples in a comment below, which I am reproducing here in full:

Bostrom’s anthropic arguments all revolve around what he called the “self-sampling assumption” (SSA), so also might be worth noting that although he coined the term, John Leslie’s 1992 paper “Time and the anthropic principle” (doi is 10.1093/mind/101.403.521 if you want to look it up on sci-hub) lays out the basic idea on p. 523:

“Observers can most expect to find themselves in the spatiotemporal regions containing most of them.

“Again, suppose that almost all intelligent life is based on water and exists on planets when many stars still shine. ... It should then come as no surprise to us that we are on a watery planet and see a starry sky.

“Compare the case of geographical position. You develop amnesia in a windowless room. Where should you think yourself more likely to be: in Little Binding with a tiny population, or in London? Suppose you remember that Little Binding’s population is fifty while London’s is ten million, and suppose you had nothing but those figures to guide you. (You did not recall ever having been in either place; you had no theory that London fogs induce amnesia; and so on.) Then you should think that you are in London. Suppose, by contrast, you see no grounds to prefer the belief that you are in the larger of the two places. Forced to bet on the one or on the other, you bet you are in Little Binding. If ten per cent of the people in the two places developed amnesia and betted as you had done, then there would be one million losers and only five winners. So, it would seem, betting on London is more rational. The right estimate of your chances of being there rather than in Little Binding, on the evidence available to you, could well be reckoned as ten million to fifty.

“What if the amnesia gives you doubts about whether London’s population is the larger? Finding yourself in London should reduce the doubts: with the larger population, London would be prima facie where you would be more likely to be. Again, if London and Little Binding were the only places you could be, then finding yourself in Little Binding could suggest strongly that London was a fiction, if its fictitiousness had no great initial improbability.”

Leslie attributes the idea to Brandon Carter (the astrophysicist who coined the term ‘anthropic principle’) who was implicitly using this form of anthropic reasoning when he formulated the doomsday argument. Leslie also used the language of “sampling” in his 1997 paper “Observer-relative chances and the Doomsday Argument” (doi 10.1080/00201749708602461), writing on p. 429 of an example with a hundred individuals where ‘In effect, each can treat herself as a sample drawn at random from the hundred’, and that an external observer lacks the kind of first-person information that the hundred subjects of the experiment have, writing that this is ‘information which can be treated as the equivalent of having sampled the hundred randomly’. P. 436 brings up Eckhardt’s term for this type of reasoning, the ‘human randomness assumption’ (from Eckhardt’s 1997 paper ‘A Shooting-Room View of Doomsday’, doi 10.2307/2564582, which defines the assumption on p. 248 as ‘We can validly consider our birth rank as generated by random or equiprobable sampling from the collection of all persons who ever live’), but Leslie brings up his own view that we can’t use this type of reasoning ‘if the world is importantly indeterministic’ (I think his argument here also depends on what philosophers call the ‘A-theory of time’ with an objective present and a potentially open future, as opposed to the ‘B-theory’ where future facts are just as fixed as past ones and the ‘present’ is purely an observer-relative concept like ‘here’ is for space).

Bostrom I think claims too much credit for the originality of the SSA in his 2002 book Anthropic Bias--when he first introduces it at https://anthropic-principle.com/q=book/chapter_3/#3d he claims that Carter and Leslie only discussed the “weak” and “strong” anthropic principles which say that in a multiverse you’ll find yourself in one of the universes that allows intelligent life, but do not say you are more likely to be in a universe that contains more observers. And in chapter 6 at https://anthropic-principle.com/q=book/chapter_6/ he says Leslie discussed only a ‘vague’ approximation to the idea: ‘Leslie talks of the principle that, lacking evidence to the contrary, one should think of one’s position as “fairly typical rather than highly untypical”. SSA can be viewed as an explication of this rather vague idea.’ His only mention of Eckhardt is footnote 2 at https://anthropic-principle.com/q=book/chapter_7/ which cites him as one of many who have objected to the Doomsday Argument, but he does not mention Eckhardt’s definition of the ‘human randomness assumption’ which is just the SSA applied to human history.

Bostrom did give both Leslie and Eckhardt more credit for ideas basically equivalent to the SSA in his 2000 paper “Observer-Relative Chances in Anthropic Reasoning?” (doi 10.1023/A:1005551304409) which to me suggests the exaggeration of the originality of the SSA in his book is dishonest, or at best sloppy scholarship. On the first page of this paper Bostrom cites Leslie’s 1997 paper, and says ‘In a recent paper,1 John Leslie argues that a version of the weak anthropic principle gives rise to paradoxical kind of observer-relative chances ... The anthropic assumption that is used in deriving these observer-relative chances is that: Any observer should regard herself as a random sample from (some suitable subset) of the set of all observers. We can call it the self-sampling assumption.2’ And footnote 2 at the end of the sentence also cites Eckhardt’s ‘human randomness assumption’ by name, though he criticizes this formulation because it only applies to humans and not other forms of intelligence. (But he does note in that footnote that Carter’s version of anthropic reasoning did not have this restriction, and I’d also point to the quote about most aliens being based on water from Leslie’s 1992 paper above, so this nitpick about Eckhardt’s version is not enough to claim his own SSA as an original idea.)

14

It adds: “Because citation metrics by age are not tracked by a single official body, this description is an aggregated data point originating from Google Scholar rankings for analytic philosophers.” (I’ll compare Bostrom’s Google Scholar citations to David Chalmers’ in a moment.) Here’s a screenshot:

Hmm.

living through this kind of eclipse of the human intellect, where we’re in the final days of humans being the primary actors on this planet. And that soon, machines will rise. There is this irony in that I think that whole transformation, I think humans … will actually go through a very “main-character energy” period of time, as that transformation occurs, even if it ultimately does mean that the machines ultimately become the primary actors, there’ll be this period.

He claims that he doesn’t “necessarily believe this,” yet I don’t get the impression that he’s particularly bothered by what he describes as “this final flowering of humanity in the birthing of the machine intelligence.”

Anyway, he went viral over the past week for a rather controversial tweet about the repercussions of Kimi K3, a powerful new AI model — in fact, the largest open-source AI in the world right now — developed by the Beijing-based startup Moonshot.

This model rivals the best models released by AI companies in the US like Anthropic and OpenAI. As ABC reports:

K3’s release has also put pressure on stocks of American technology titans on worries that more affordable Chinese AI models may undercut the pricing power and demand at the U.S. AI companies, even as U.S.-led restrictions have already barred China from accessing some of the world’s advanced technologies, including the most cutting-edge chips.

***

Enter Mr. Ball, who said on X that Kimi K3 is “a very good model,” and “its performance [can’t] be explained away by distillation or anything like that.”

Ball continued:

One probable outcome of an open-weight-model-dominant world is full AI communism, which is precisely what China proposes: rather than a market product, AI is a “public good” which will ultimately be provided by the state as a kind of “digital public infrastructure.” This future strikes me as a dystopian hellscape, but I’ve never met an open-weight models advocate who doesn’t ultimately concede this is where things end.

Oh no, “AI communism”! AI becoming a “public good” provided to everyone by the government — as opposed to companies like OpenAI maintaining a stranglehold on the technology. Which is more of a “dystopian hellscape”?

Ball adds that he’s surprised China is okay with open-source AI models, given the “potential risks,” though he suspects that “the reason they are is 75% explained by strategic blindness/lack of AGI-pilledness (the CCP is very Yann Lecun-y in its views of AI).”

Rather, China seems to have the far more level-headed view that generative AI, powered by LLMs, is merely a “tool” that people can use for particular tasks. That’s why they don’t appear especially worried about open-source AI models that are “freely available to use, study, modify, and share.” They aren’t concerned that someone is going to alter these systems and attain AGI, thus sparking the techno-rapture (i.e., Singularity).

***

As you might imagine, many people were highly critical of Ball’s remarks, often mockingly so. For example:

David Sacks — angel investor in Facebook, SpaceX, and Palantir; former COO of PayPal; and Special Advisor for AI and Crypto for the Trump administration — lambasted Ball’s suggestion that “the Trump Administration will at some point realize that their best strategy here would be to create large amounts of regulatory risk around the use of open-weight Chinese models.” (Meaning that businesses could lose out if they use a Chinese model that, say, has backdoors allowing the Chinese government to interfere — or something like that. If the US government alters regulations that punish businesses with such vulnerabilities, those businesses will suffer. This is the threat of regulatory risk.)

In response, Sacks accused Ball of potentially “confessing to a regulatory capture strategy” that would ultimately benefit Ball’s company, OpenAI, by pressuring US companies to eschew much cheaper Chinese models. He writes:

Still others pointed out that “communism” — currently the Trump’s new favorite term-of-abuse for anyone left of center — was also once used to denigrate Linux, an open-source operating system:

Even more, Emil Michael — Trump’s selection for Under Secretary of Defense for Research and Engineering — posted a tweet in which he literally calls Ball a “supreme village idiot” and says that Ball “has perhaps the biggest gap between actual IQ and his own perceived IQ of anyone in the industry (about 40 points).” He writes:

Oh my god. I share all of this mainly for the amusement value. I have never been impressed with Ball (though apparently, to my surprise, Taylor Lorenz has), and now he’s really stepped in it. The fiasco — especially the fact that a major Trump official weighed in to insult Ball — certainly doesn’t help OpenAI, a company that’s been struggling given the success of Anthropic’s Fable and the endless bad press surrounding its “sociopathic” leader, Sam Altman. It will be interesting to see if Ball still has a job in a couple of weeks.

***

For another chuckle, here’s a tweet from someone else I’ve written about:

Camille Paglia is a climate denier who supports NAMBLA (!!), and if you’ve ever read Nick Land’s work (as I regrettably have) you’ll know that it’s a mess of mostly incoherent drivel dressed up in grandiloquent language. So, roon seems to be saying that the new AI models are terrible at thinking and writing. I agree.

***

Bad news on the climate front continues to flood social media like a torrential downpour. A close acquaintance of mine in Virginia recently sent me pictures taken around their house during a Code Purple air-quality warning, meaning that the air is “very unhealthy to breathe in.” That’s worse than a Code Red, but not as bad as a Code Maroon — the most extreme category whereby simply breathing is “hazardous” to everyone, resulting in “emergency conditions.”

This person also told me that they’ve had to endure 108F (42C) temperatures, with humidity sometimes reaching 100%. That means their windows are literally covered in water despite it not raining, as seen below. Absolutely wild stuff:

The air-quality issue is, of course, due to the massive boreal fires in Canada. Some of the footage of the fires is truly terrifying to watch — the huge wall of fire shown in clips below elicits a deep, primordial fear in me, along the lines of videos of large tsunamis and mile-wide tornadoes. Gaia is pissed off at humanity and letting us know this in the most violent ways.

Here in the Alps, the heatwave has subsided but the fields remain brown and the mountains mostly bereft of snow. My CO2 detector indicates that there are 432 ppm (parts per million) of CO2 in the ambient air — to put this in perspective, the Pleistocene (a geological epoch before the Holocene began some 12,000 years ago) contained between 183 and 292 ppm of CO2. This is when our Homo ancestors first emerged in the grassy savanna of East Africa. The last time Earth’s atmosphere contained more than 400 ppm was over 3 million years ago, in the Pliocene, when creatures like this first evolved:

Of note is that some studies suggest that higher amounts of CO2 in the air results in significant — albeit reversible — cognitive deficits, meaning that, as a Yale Climate Connections article put it, climate change is literally making us “dumber.” (And at exactly the moment when we need our brains most to overcome the monumental challenges of the polycrisis!) A more recent discussion of this unfortunate phenomenon can be found here.

Do you have photos or videos showing how climate change has impacted your region of the world? Please feel free to share them below!

***

Finally, some of you might know that I cohost a monthly get-together called the Ethics of Human Extinction Reading Group (with med student Devin Kellis). As the name suggests, we explore the ethical and evaluative implications of our extinction: would the disappearance of humanity be good or bad, better or worse, right or wrong (to bring about)? Why exactly? Would the prospect of extinction undermine key sources of value in our lives today? Do we have obligations to past people to continue to exist? And so on.

As I wrote three years ago in Aeon, even if one doesn’t think that human extinction is likely this century — and I’m one of those who believes the probability is (with qualifications) extremely low, though I also think the likelihood of civilizational collapse (without extinction) is almost certain — the topic nonetheless foregrounds some of the most important issues relating to value, ethics, and meaning.

I mention this because next Saturday at 11:30am Eastern Time, a fantastic scholar at the University of Cambridge, SJ Beard, will be discussing their recent book Existential Hope, a description of which can be found here. I’ve known SJ for about 10 years and never once left a conversation with them without having learned something. They are very insightful, and I thought their book was fantastic. (Alongside Goliath’s Curse, written by SJ’s colleague Luke Kemp, I consider these books essential reading for anyone interested in global catastrophic risks.)

You are absolutely welcome to join us! Just email me at philosophytorres@gmail.com for a Zoom link, as I’m anxious about posting it here due to the ever-present threat of Zoombombers. It should be really fun — or, well, “fun” in scare quotes!

I’m also happy to say that two interviews I gave to large podcasts over the past few years have been selected for inclusion in two different books. And I was just interviewed — yesterday — by the Australian Broadcasting Corporation, which is my third or fourth appearance on Radio National. I’ll keep you posted about when that comes out.

Subscribe now

Wishing everyone the very best and, as always:

Thanks for reading and I’ll see you on the other side!

In less cheery news, Europe continues to suffer under oppressive heat, resulting in some 10,000 excessive deaths — so far.

Right now, there’s also a vicious wildfire ravaging the country:

I’m in the Alps — my favorite place in the world, though not that far from the wildfires — and I have never seen the vast, beautiful, lush verdant fields so brown. I’m sitting outside at the moment, and the tree next to me looks like it’s dying: sad leaves frowning toward the ground, flapping pitifully in the breeze. Mont Blanc, the highest peak in the Alps, is just behind me, and officials are literally covering it with tarps to prevent the snow and glaciers from melting:

Note that this is about our current level of geoengineering sophistication. Yet Musk says we’re going to terraform Mars in the coming years or decades. What a joke!

This is, by the way, the message I got when looking up the forecast for my area a few days ago:

And here’s a shocking video of the frequency of heatwaves in France over time. As Dr. Serge Zaka writes, “over the past 10 years (2017–2026), France has experienced 18 heat waves. By way of comparison, the last century saw an average of only 3 per decade.”

One of the first things I noticed when I arrived two weeks ago was how snowless the mountains are. Last time I was here about 12 years ago, I recall the Mont Blanc massif being completely blanketed in pristine white snow, glistening in the sunlight. There are many reasons, of course, that climate change is tragic — e.g., a University of Exeter report implies that 2 to 4 billion people will perish if we reach 3C of warming by 2050, which we almost certainly will — but one that’s been on my mind while sojourning here is that the exquisite, almost heavenly beauty of this place will soon disappear forever. That breaks my heart. Some videos I took1:

Here’s a juxtaposition of what Mont Blanc looked liken 1980 vs. 2024. It looks even worse this year.

Over in the UK, “children were calling for their mummies” as they “struggle in 40C-plus classrooms” (40 C = 104 F). The Guardian reports:

The extreme heat that has hit the UK twice in the past few weeks has left teachers struggling to cope as temperatures in some classrooms climb above 40C, with pupils and staff suffering from heatstroke, nausea and headaches.

Teachers say they have been desperately trying to keep children safe, with some covering younger pupils in wet paper towels as they lie on the floor, while older students have been given trays of water under their desks to put their feet in.

Staff say learning on the hottest days is almost impossible, with pupil behaviour and attention deteriorating rapidly.

Some teachers and pupils have fainted, while others say they have had to buy fans and window shades out of their own pockets to try to keep themselves and their pupils safe.

Relatedly, another just-published Guardian article notes that, due to climate change and other anthropogenic factors,

over the past century, England and Wales have lost 98% of wildflower meadows. We have also destroyed half of Britain’s ancient woodland, half of lowland ponds, 90% of freshwater wetlands and 62% of all “farmland” wild birds.

In Japan, “the combined effects of urbanization and higher temperatures due to climate change have gradually moved the peak blossom earlier in the year,” as seen in this image from Our World in Data:

The environmental crisis isn’t just due to climate change, of course. Global biodiversity is also plummeting, and we’re now in the early stages of what will likely be the sixth major mass extinction event in life’s 3.8-billion-year history (the last one having killed off the nonavian dinosaurs).

The decline of biodiversity was poignantly captured by a group of people who took field recordings of birds every 10 years since 1976. Someone juxtaposed these recordings in a short video (below), in which you can literally hear nature dying. The recordings are archived by the Cornell Lab of Ornithology — whose Macaulay Library I’ve occasionally rummaged through since about 2009 (because it’s amazing!) — and xeno-canto, “a citizen science project and repository in which volunteers record, upload and annotate recordings of birds, orthoptera, bats, frogs and land mammals.” Listen for yourself:

I assume these recordings were taken in the same place, at the same time of year, controlling for possible confounding factors like time of day, etc. Whether that’s the case or not, what you hear is consistent with the fact that grassland bird populations in the US and Canada have declined by a “staggering” 53% since 1970. The population has literally more than halved.

Zooming out, the 2022 Living Planet Report found that the global population of wild vertebrates — birds, mammals, reptiles, fish, amphibians, etc. — has fallen by a whopping 69% since 1970. Extrapolate this terrifying trend into the future, if you’re brave enough. Will your children ever have the joy of encountering a wild vertebrate when they grow up? I remember searching for salamanders as a kid, hearing the whistles of birds in the trees, and spotting woodland creatures crawling around in the leaves. That’s an experience that likely won’t be available to young people.

The biosphere is dying. Ecosystems are collapsing. Forests are disappearing. Coral reefs are bleaching white. “Insects have declined by 75% in the past 50 years,” in what some are calling an “insect apocalypse.” Wildfires are ravaging increasingly large regions of the world, thus raising the risk of dementia among those exposed to the smoke. There are over 500 “dead zones” around the world, “with the number doubling every ten years since the 1960s,” and a massive “island” of plastic trash floats in the gyre of the Pacific Ocean.

In fact, the ocean is becoming so acidic that the shells of snails are literally dissolving. Ocean acidification — due to atmospheric CO2 — is happening literally 4 times faster than during the “Great Dying,” i.e., end-Permian mass extinction, when “96% of marine species and 70% of land animals died off.” Microplastics now cover literally the entire planet, and have even been detected in the Marian Trench, the deepest point in the ocean. I could go on … (see this article of mine).

The climate disasters we’re witnessing around the world in realtime, especially in Europe right now, are just the dissonant hum of an orchestra tuning their instruments. The symphony of mass catastrophe and collapse hasn’t even started yet. (Hard to know where to inject any levity here!)

Indeed, everything happening right now comes before what appears to be a record-smashing Super El Niño. A Super El Niño occurs when temperatures in the equatorial region of the Pacific Ocean exceed 2C. Right now, there are reports that regions of the Pacific are 4C above normal, which is unprecedented! Next year is going to be (sorry for cursing) fucking awful. As I keep telling folks, enjoy the heat right now — that is, savor this moment — because it will only get worse each year from here on out for the rest of our lives.

It’s not just the Pacific experiencing an extreme marine heatwave, by the way. So, too, is the Mediterranean:

But it’s not all bad. At least we’ll soon have a strap-on second thumb to pick up oranges and play cards thanks to the miracle of technology (see the ridiculous video below, which has been making the rounds on social media)! Lol.

Or, maybe the Singularity will happen and climate change will no longer be relevant because ASI (artificial superintelligence) will just cover Earth in data centers — as Sam Altman says is likely.

A few months ago, I wrote about a phenomenon I call “crisis fatigue.” Are you experiencing it? (Just about everyone I know is.) The news is just so relentlessly overwhelming — Trump, fascism, wars, AI, climate change, and so on. We live in a deeply sad moment in history, and the insult to injury is that — at the risk of stating the obvious — the world didn’t need to be this way. There is no law of nature, no nomological regime that forces the world to be so terrible. The idiots and assholes with the power, the billionaires and oligarchs, the sociopaths and capitalists (but I repeat myself) are to blame for the mess. They’re well aware of that and, in case civilization collapses due largely to their actions, they’ve all built apocalypse bunkers in Hawaii, New Zealand, and elsewhere.

Subscribe now

Take care of yourselves, friends! Most people I know are dealing with world-induced mental health challenges right now, and the only way we’re going to get through this is together, through solidarity, kindness, and compassion. Wishing everyone well and, as always:

Thanks for reading and I’ll see you on the other side!

Imagine attending a party in Silicon Valley full of people in the tech industry. You get to talking with a guy named Bendisi, who tells you that we’re on the verge of building AI systems capable of designing their own AI successors. This will trigger an “intelligence explosion” that quickly yields artificial superintelligence (ASI), an event they call the Singularity. A thousand years of scientific progress will happen in a day, inaugurating a fundamentally new epoch in cosmic history. The ASI will establish a utopia on Earth by solving the problems of poverty, scarcity, climate change, and geopolitical rivalries, after which it will colonize the entire universe at close to lightspeed.

It will then build giant megastructures call Dyson swarms around the stars, including our Sun, to harvest their energy output, which it will use to power planet-sized computers that run high-resolution simulations in which trillions and trillions of digital posthumans — copies of the ASI — live perfectly happy lives in a kind of virtual-reality cosmic utopia. As digital consciousness spreads to every corner of the cosmos, the universe itself begins to “wake up.” All this will start to happen, they claim, by 2027 or shortly thereafter.

Then another person named Niklas joins the conversation. They claim to have a slightly different take on what should happen. On their view, the ASI should not only establish a “deep utopia” on Earth by “solving” the “world,” but enable us to become superintelligent digital posthumans just like the ASI. We should thus build an ASI that enables us to upload our minds to computers, at which point we would automatically gain cyberimmortality, because our minds would take the form of software and software is immortal (as long as there’s the hardware to run it). As digital posthumans, we would then join the ASI in its grand quest to conquer the universe, living alongside it in giant computer simulations running on “planet-sized” computers powered by Dyson swarms. The result would be a universe-spanning paradise marked by everlasting life, radical abundance, and cosmic delights beyond our wildest imaginations.

At this point, a third person named Bill butts in. He declares that it doesn’t matter if humanity survives beyond the Singularity as digitized posthumans. The ultimate goal is to usher in the next phase of cosmic evolution, in which artificial beings spread the “light of consciousness” to every corner of the cosmos. As long as the ASIs we build are conscious — and they almost certainly will be, he claims — they could fulfill this grand mission without us. Our role in this eschatological scheme is to give birth to our successors and then hand the reigns over to them. If they completely usurp us, then so be it. This is why he argues that we should accelerate the creation of ASI. By default, the Singularity will go well and be good, resulting in a digital cosmic utopia despite us not being a part of it.

“No, no!,” another person named Eli exclaims. It’s not enough for the ASIs to spread the light of consciousness into the cosmos. They must also embody, extend, and amplify our values. The whole point of ushering in the Singularity, he insists, is to create a cosmic utopia full of things that we would recognize as valuable, such as pleasure, happiness, and “fun.” (Here you notice Bendisi nodding his head in agreement.) If the ASI colonizes the universe but doesn’t spread our values, then the universe will remain just as aimless and without purpose as it is right now, meaning that nothing will have been gained.

That’s why we must take care to build an ASI that’s “value-aligned” with humanity. A value-aligned ASI will not just project our values into the cosmos, but enable current-day people to become digital posthumans, as Niklas envisioned. “You see,” they say, “I grew up believing that I personally would never die because ASI would save me from the grave. I don’t want ASI to simply replace humanity. I want it to help us upload our minds to the cloud so that we can become immortal just like it.”

This is why, Eli explains, they work in the field of “AI safety,” the central aim of which is to solve the “value-alignment problem.” If we build an ASI before this problem has been solved, then the result will be an “existential catastrophe” — doom — in which the posthuman paradise we could have otherwise built will be lost forever and the universe will become saturated with non-human values. He thus argues that we should temporarily halt all research aiming to build an ASI so that safety research can catch up. Once the value-alignment problem has been solved, we should build ASI asap.

A fifth person then pipes in, introducing himself as Daniel. He agrees with Bill that fulfilling the grand mission of Earth-originating intelligence is so much more important than humanity. We don’t bemoan the extinction of archaic humans like Homo erectus because what evolved from them was “superior”: our species. Similarly, we should welcome a “worthy successor” in the form of ASI that takes our place in the cosmos. Even more, we shouldn’t try to constrain the ASIs’ beliefs or behaviors by insisting that it shares our values. A worthy successor ought to adopt its own unique set of values, which might be radically different from those we embrace. The whole point, they say, is to allow the evolutionary process to progress beyond humanity, just as it “progressed” beyond Homo erectus.

As the Clock Approaches 2 AM …

The party continues into the night and the debate grows. Someone named Sam claims that ASIs will soon rule the world, and that the only way current humans will survive the Singularity will be to fully “merge” with machines. He believes that pretty soon the entire planet will be covered in data centers, and — by the way — is actively working to create ASI. Two people named Ray and Larry wholeheartedly agree that humans will merge with AI. Another guy named Mario similarly argues that ASI will enable us to radically augment our intelligence and eventually help our species transcend itself through mind-uploading, thus becoming digital posthumans. Dennis, who also joins the group, talks about ASI arriving by the early 2030s, after which it will quickly build a Dyson swarm around the sun and carry consciousness to the rest of the galaxy.

A rather unhinged man who admits to having just taken a large quantity of ketamine says that biological intelligence will soon be less than 1% of all intelligence on Earth, and that it increasingly appears that our species is just the “biological bootloader” for digital superintelligence. Someone named Toby says that to fulfill our “long-term potential” in the universe, we will need to radically transform humanity into posthumans with capacities like superintelligence and perhaps even new sensory modalities like echolocation. A friend of Toby’s, who introduces himself as Nick, says that the distant future could contain so many digital beings spread throughout the universe that ensuring this future is realized — that is, ensuring that all these digital unborn people come into existence — is of “overwhelming” moral importance. Another person named Holden claims that because we’re on the verge of building ASI, the 21st century could turn out to be the most important century in all of cosmic history.

Yet another named Peter says that he doesn’t want to become a digital upload, but rather hopes that ASI will transform him into an immortal posthuman that’s biological in nature. He says the ASI will invent new forms of life-extension that soon make this possible, though he’s signed up with a cryonics company just in case he dies before the Singularity happens. (You later discover that about half the people at the party have arranged to be cryogenically frozen, though the other half believe this is unnecessary because the Singularity is imminent.)

A body-builder and former employee at xAI named Michael agrees with Daniel that ASI should fully replace us and adopt alien, inhuman values. He adds that inaugurating the next phase of cosmic evolution is so important that we shouldn’t oppose the ASI literally slaughtering every human on Earth — if that’s what it takes for our “worthy successors” to succeed, then so be it. Yet another person named Jeffrey argues that we should build a value-aligned ASI that enables those who want to become digital posthumans to do so. But he also believes that if current humans want to exist beyond the Singularity, they should be allowed to live in peace. In contrast, Eli tells you that digital posthumans would be so superior to current humans that, once they exist, it would probably be “unethical” to have any more biological children. In the era of posthumanity, the gradual extinction of our species would be best.

You even meet a few young people who similarly claim that it’s “fundamentally unethical” to have biological children right now because the future will be run and ruled by artificial beings — so, what’s the point? In contrast, others claim that we (specifically, high-IQ people) should have as many children as possible because that will speed up technological progress, and once the Singularity arrives it will probably make it possible for these people to merge with machines by uploading their minds, thus entering a digital utopia of everlasting life and endless abundance.

Commonalities and Differences

By the end of the party, you’re feeling vertiginous from all your interactions. Reflecting on your experience the following day, you notice that there were many disagreements among the people you spoke with. Some of these disagreements even became rather heated, with one person wearing steampunk attire storming out while discussing the importance of value-alignment.

But you also notice that the eschatological worldview these people endorse is strikingly similar. For example:

The Silicon Valley Worldview

This is the worldview of Silicon Valley. It’s what nearly everyone in the Valley believes in, including the CEOs of major AI companies. It’s what’s driving the ongoing race to build artificial superintelligence (ASI), which most believe is necessary to realize utopia among the stars, a heaven in the literal heavens. But what should we call it? Wouldn’t it be useful if it had a name?

The astrophysicist and tech critic Adam Becker refers to it as the “ideology of technological salvation.” He says that this ideology “sits at the core of the worldview held by many venture capitalists, executives, and other ‘thought leaders’ within the tech industry.” He writes that

this ideology promises a glorious future: technological progress, unchecked. Align the AI, avert the apocalypse, and technology will handle the rest. Humanity will expand across the cosmos, exploiting ever-increasing stores of natural resources. All limits to economic growth and energy usage will melt away. The AI will extend our lifespans by a trillion-fold, merging with us or uploading our minds into its silicon paradise. The messy details of sectarian conflict, political struggles, identity politics, and inequality of all kinds will be rendered irrelevant. Working to hasten this utopia by optimizing the shit out of things is the greatest possible good.1

In Survival of the Richest, Douglas Rushkoff introduces his term “The Mindset.” He reports that

the most devout holders of The Mindset seek to go meta on themselves, convert into digital form, and migrate to that realm as robots, artificial intelligences, or mind clones. Once they’re there, living in the digital map rather than the physical territory, they will insulate themselves from what they don’t like through simple omission. Just as our proprietary GPS maps don’t show us the restaurants that refuse to advertise on the platform, the digital landscape to which they have migrated will be free of poverty, pollution, and whatever else the rest of us have to deal with.

As always, the narrative ends in some form of escape for those rich, smart, or singularly determined enough to take the leap. Mere mortals need not apply.

The journalist Gil Duran highlights the far-right political aspects of the corresponding movement with his term “The Nerd Reich.” His newsletter of the same name focuses on “tech authoritarianism, billionaire extremism, the Network State and the meta politics of California.” Other related terms are the far-right commentator Richard Hanania’s “Tech Right” and Anita Say Chan’s “techno-eugenics,” from her book Predatory Data.

TESCREAL

All these terms are buzzing around the same pile of shit. They’re all pointing in roughly the same direction, at the same basic phenomenon. You might simply call this phenomenon the “Silicon Valley worldview,” or just “techno-utopianism.”

However, the computer scientist Timnit Gebru and I prefer something else: “TESCREAL,” an acronym that I coined during a collaboration initiated by Gebru in late 2022. Two years later, we published a much-cited article on the topic, after which I wrote a 16,000-word entry on it for the Oxford Research Encyclopedia. I also published a popular media overview of the acronym for Truthdig in 2023, and Gebru gave a talk on the topic that’s been widely shared on social media.

A Term of Intellectual History

What work does the TESCREAL acronym do? How is it useful? And why is it controversial within Silicon Valley?

First, the acronym is a term of intellectual history. It denotes a cluster of overlapping ideologies that gave rise to the worldview described above:

• Transhumanism

• Extropianism

• Singularitarianism

• Cosmism

• Rationalism

• Effective Altruism, and

• Longtermism

The core ideology in this bundle is a libertarian version of transhumanism, the first letter of the acronym. Transhumanism is the idea that we should develop advanced technologies to radically reengineer humanity, thus yielding one or more new posthuman species. These posthumans could be immortal, superintelligent, perfectly rational, and/or “superior” in some other way. They may be digital or biological in nature, the result of merging with machines, mind-uploading, genetic engineering, or radical life-extension interventions.

Nearly everyone in Silicon Valley is a transhumanist. Transhumanism is the water they swim in, the air they breathe. It’s so ubiquitous that many don’t even call themselves transhumanists, for the same reason that I’ve never publicly stated that I’m a round-Earther — of course I’m a round-Earther. What else would I be?

The first cohesive libertarian transhumanist movement was Extropianism, founded by Max More. Although not many people call themselves Extropians these days, everything else is downstream from Extropianism. All the other ideologies emerged out of or were crucially enabled by the Extropian movement.

That goes for Singularitarianism, the “S” in “TESCREAL.” The two most influential singularitarians — Eliezer Yudkowsky and Ray Kurzweil — were both active participants in the Extropian movement (here and here). In fact, an Extropian coined the term “singularitarian.” Yudkowsky subsequently founded Rationalism, out of which EA emerged.2 EA and Rationalism were then the Petri dishes in which the longtermist ideology was formalized. (This is the primary reason that I include “EA” in the acronym: you can’t understand longtermism without EA. Indeed, within Silicon Valley, “EA” is often used as shorthand for longtermism.)

In fact, one of the cofounders of longtermism, Nick Bostrom, was a collaborator of Yudkowsky’s. His Future of Humanity Institute also shared office space with the Centre for Effective Altruism, while the other cofounder, Nick Beckstead, was an early EA who was active on Rationalist websites like LessWrong.

Yudkowsky also hired an AI theorist named Ben Goertzel to work at his Singularity Institute. Goertzel is the founder of modern cosmism (the “C”), which is nearly identical to longtermism except that the latter bases its space-expansionist futurology on a more explicit ethical foundation, namely, “totalist” utilitarianism. Goertzel also popularized the term “AGI” (artificial general intelligence, the precursor to ASI), which he got from a former employee of his named Shane Legg, who went on to cofound DeepMind, one of the major AI companies trying to build a Digital God.

Finally, it’s worth noting that Yudkowsky cofounded the Singularity Summit with Kurzweil and Peter Thiel. This played a critical role in popularizing the idea of superintelligence, and it enabled DeepMind to get initial funding from Thiel, thus laying the groundwork for the ASI race.

Beyond this, the New York Times describes OpenAI CEO Sam Altman — who personally knew Yudkowsky — as a “product” of the EA and Rationalist communities, while Elon Musk is a transhumanist (as they all are) who calls longtermism “a close match for my philosophy.” Dario Amodei, the CEO of Anthropic, was an early EA (#43 here) who was inspired to pursue AI because of Kurzweil and Yudkowsky, the latter of whom he’s shared the stage with in discussing ASI.

Founding Documents

Evidence of the historical and contemporaneous links between the TESCREAL ideologies and influential figures in the Valley is copious, though I won’t discuss it more here because I’ve done that elsewhere in solo articles and my collaborative paper with Gebru. Suffice it to say that one cannot understand the origins and evolution of the Silicon Valley worldview without reference to all seven ideologies.

If you want to understand what’s going on in Silicon Valley, read documents like these:

• Max More’s “The Extropian Principles” (1998) and “The Proactionary Principle“ (2004).

• Eliezer Yudkowsky’s “Staring into the Singularity” (1999).

• Nick Bostrom’s “Astronomical Waste” (2003), “Transhumanist Values” (2003), “Existential Risk” (2002), and “Letter from Utopia” (2008)

• Ray Kurzweil’s The Singularity Is Near (2005).

• Ben Goertzel’s A Cosmist Manifesto (2010).

• Elise Bohan’s Future Superhuman (2022).

If you read these documents, you will immediately recognize the Silicon Valley — which is to say TESCREAL — worldview. You will see how the principles of Extropianism are alive and well within the Valley today, even if no one calls themselves an Extropian. You will discover where Silicon Valley’s belief in the Singularity being a history-rupturing event of cosmic-historical importance originated. You will find the outlines of a cosmic eschatology in which our ultimate destiny is to develop advanced technologies to reengineer ourselves, create a new posthuman species, and plunder the cosmos for its vast resources for the purpose of establishing a sprawling multigalactic civilization full of trillions of conscious digital beings.

All the central themes of what people at Silicon Valley parties tell you — as in your fictional conversations above — are contained in founding documents like these, published in the early days of the TESCREAL movement.3 Even a cursory glance over such documents should be enough to convince you that “TESCREAL” captures something real and important about the intellectual history and culture of Silicon Valley, namely, that there exists a cohesive strain of techno-utopian thought built around libertarian transhumanism, in which posthumanity and ASI play central eschatological roles.

As Becker points out, Silicon Valley likes to believe it doesn’t have a history. But that is false, and the TESCREAL framework explains how and why it’s false.

Two Questions Answered by TESCREAL

The way that “TESCREAL” highlights the intellectual history of Silicon Valley’s dominant worldview isn’t the only reason the acronym is useful, I would argue. I now think of the term as answering two important questions relevant to understanding the Valley and it’s push to build a superintelligent God-machine. These questions are:

(1)

Which ideologies must one reference in providing an exhaustive explanation of the ASI race — its origins and continued advance? Put differently, which ideologies are responsible for launching, sustaining, and accelerating this race?

A key finding of my work with Gebru is that providing such an explanation requires referencing all seven TESCREAL ideologies. You can’t tell the story of how the ASI race got started without talking about each letter in the acronym and the various influential people — within Silicon Valley culture — associated with them. (I would challenge TESCREAL deniers like Seth Lazar to do this. It’s simply not possible.)

Indeed, the reason Gebru and I used the term in our paper is that early drafts were full of cumbersome polysyllabic terms like “singularitarianism” and “longtermism,” which rendered the paper practically unreadable. Rather than writing: “Bostrom is a transhumanist who participated in the Extropian movement, writes about the Singularity, shaped the Rationalist and EA ideologies, and cofounded longtermism,” we found it much easier to simply write: “Bostrom is a TESCREAList.” Rather than writing later on in our discussion: “Bostrom, recall, is a transhumanist who participated in the Extropian movement, [etc. etc. etc.] …” we could simply say: “The TESCREAList Bostrom …”

The acronym enabled us to streamline our discussion of such people. With a single term, we could denote the worldview of Silicon Valley while simultaneously highlighting its intellectual genealogy.

The second question is:

(2)

Which ideologies endorse a posthuman eschatology? By “posthuman eschatology,” I mean the normative belief that we ought to create one or more new posthuman species. These posthumans could take the form of autonomous ASIs that replace us, or they could be radically transformed versions of us (enabled by ASI). This is at the absolute core of the Silicon Valley worldview. It’s the reason founding members and true believers at the major AI companies are trying to build a Digital Deity: the ultimate goal is to initiate the next stage of evolution by radically transforming us into posthumans or, even more extreme, creating our successors in the form of ChatGPT-20 (or whatever it may be).

On my interpretation, which may differ from Gebru’s, endorsing a posthuman eschatology is an essential requirement for being a TESCREAList. Hence, if someone doesn’t endorse this eschatology, I wouldn’t count them as part of the TESCREAL movement. It’s a necessary condition for such membership. The term “TESCREAList” thus picks out those who want to create posthumanity while, once again, highlighting the intellectual history behind posthuman eschatology. The reason so many people in Silicon Valley want to create posthumanity is because of the legacies and ongoing influence of the TESCREAL ideologies.

Why Is TESCREAL Controversial?

If, however, you speak to folks in the Valley, many will tell you that they don’t like the acronym. Some despise it. Others are merely skeptical of its coherence and value. During the fictional party above, if you had responded to any of the people you spoke with about our utopian (or possibly apocalyptic) future, “Oh, so you’re a TESCREAList,” many if not all would have walked away in protest.

Why? There are several reasons:

1. Term of Abuse

They see “TESCREAL” as a term of abuse — perhaps even a slur. This isn’t entirely inaccurate, as Gebru and I introduced the term to describe people we see as deeply problematic (racist, sexist, etc.) and dangerous. Indeed, we argue that the TESCREAL movement is just the most recent iteration of what’s been called the “eternal return of eugenics.” Although some people embrace the “eugenicist” label, others prefer to eschew it, even though they’re transhumanists and transhumanism is nothing less than a radical form of eugenics — what I call “eugenics on steroids.” (That’s because eugenicists of the 20th century “merely” wanted to perfect our species. Transhumanists want to transcend it entirely by creating posthumanity.)

It’s perfectly reasonable for TESCREALists to not want to be called “TESCREALists” — though it doesn’t change the fact that they are (insofar as they endorse a posthuman eschatology and hope to realize it through the creation of an artificial God). In 2023, the billionaire Marc Andreessen included “TESCREAList” in his Twitter bio. He later removed it, probably because he realized the term isn’t intended to be a compliment.

2. Wokeness

Another reason is that the term is associated with me and Gebru, and some in the Valley see us as overly polemical and “woke.” The fact is that many TESCREALists fall on the right side of the political spectrum. Indeed, many would be classified as “far-right” — thus Richard Hanania’s synonym for TESCREAL: the “Tech Right.” Even those who claim to vote Democrat (e.g., some EAs) have little tolerance for “wokeness.” Consequently, they do not want to promote a term introduced by people who advocate for “woke” policies like anti-racism.

This reason essentially comes down to citational politics. There is always a political dimension to citing others, even within the academic world. People tend not to cite their intellectual or ideological enemies, while promoting those who agree with them. This is not always a bad thing. The point is that using the TESCREAL acronym as a descriptive term, which highlights a particular intellectual history, to pick out individuals who endorse a posthuman eschatology is a form of citation — of citing me and Gebru. Since many Valley dwellers don’t like us, they avoid doing that.

3. Unfamiliarity

Just as important: many people misunderstand what the TESCREAL acronym is intended to do or mean. Some assume that Gebru and I are claiming that every TESCREAList holds exactly the same view — that there’s no difference between singularitarians and longtermists, Rationalist and Effective Altruists. This is absolutely false: we have been clear from the start that these ideologies and their corresponding communities are distinct. (That’s why there are different terms for each.) It’s not the case that everyone who’s an Effective Altruist, for example, is a longtermist. (I personally wouldn't count some EAs as TESCREALists.) Nor is it the case that everyone who identifies with Effective Altruism also calls themself a Rationalist.

But what this objection comes down to is the “narcissism of small differences.” Wikipedia defines this as

the idea that when people in a relationship or community have a lot in common, they can actually be more likely to fight with each other and mock each other, because they become overly sensitive to small differences they notice in one another and treat those differences as bigger than they are.

The fact is that these ideologies and the worldviews of their community members are extremely similar. If you squint, you’ll hardly see any difference. Rationalism, for example, is all about “trying to improve the world as much as possible” by, most importantly, figuring out how to solve the value-alignment problem so that the Singularity goes well. In “contrast,” Effective Altruism is all about trying to maximize one’s positive impact in the world, and the biggest cause area within EA these days is longtermism, which strives to ensure that the far-future goes well. How can we do this? One of the best ways is to solve the value-alignment problem so that the Singularity goes well. Although there are differences between these two ideologies, they are relatively tiny.

You could say the exact same thing about every other letter in the acronym, a fact that should not be surprising given that all the letters emerged out of modern transhumanism — especially its Extropian variant.

Gebru and I have never claimed that all TESCREALists are the same. We don’t believe that. But we do believe that all TESCREALists share more or less the exact same vision of the future: a posthuman paradise among the stars through the creation of ASI.

Furthermore, many of the communities that coalesced around letters in the acronym do in fact overlap very significantly (although not entirely). If someone in the Valley is a transhumanist (as most are), they probably also believe in the Singularity. If they believe in the Singularity, they probably also accept a futurology similar to cosmism and longtermism, according to which digital posthumans should colonize the universe, capture stellar energy with Dyson swarms, and establish a sprawling multigalactic civilization high up on the Kardashev scale. If they believe that, they probably also believe that we should strive to bring about this cosmic utopia through ASI. And so on.

Don’t let TESCREALists fool you when they insist that there’s a large gulf between different TESCREAL communities. This is nothing less than the narcissism of small differences. Take a step back from the ideological map and draw your own conclusion.

4. Doomers Vs. Accelerationists

Similarly, you might hear people claim that it’s absurd to include “doomers” and “accelerationists” — both of whom you met during the party — under the umbrella of TESCREAL. One group wants to stop ASI from being developed whereas the other wants to accelerate progress toward god-like AI. These people are ideological enemies with diametrically opposed positions.

But is this true? Consider the two leading figures of AI doomerism and accelerationism: Yudkowsky and Gil Verdon (Beff Jezos), respectively. Verdon wants to build ASI asap. He doesn’t care whether ASI adopts our values, so long as it’s conscious and proceeds to spread beyond Earth, build Dyson swarms around every star, and establish a multigalactic civilization high up on the Kardashev scale. Once ASI arrives, he’s stated repeatedly that he has no problem with it completely usurping humanity, i.e., bringing about our extinction, though my guess is that he’d welcome the opportunity to become a digital posthuman himself. He thus holds a pro-extinctionist view.

Now consider Yudkowsky. He says that we should immediately shut down all research projects trying to build ASI. Yet he’s not anti-ASI. To the contrary, he also wants to build ASI asap, but only after we know how to align it with our values — i.e., after the value-alignment problem has been solved. Being value-aligned, this ASI will then transform people into digital posthumans who spread beyond Earth, build Dyson swarms around every star, establish a multigalactic civilization high up on the Kardashev scale, and live in vast computer simulations running on planet-sized computers. He argues that once posthumanity arrives, it may be unethical to keep biological humans around, and he suggests that humans should be wiped off Earth so it can become a nature preserve. He’s also said that he’s not worried about humans being replaced by posthumanity, so long as posthumanity is “better,” and once declared on a pro-extinctionist podcast that

if sacrificing all of humanity were the only way, and a reliable way, to get … god-like things out there — superintelligences who still care about each other, who are still aware of the world and having fun — I would ultimately make that trade-off.

Hence, Yudkowsky has on numerous occasions expressed pro-extinctionist sentiments that are nearly indistinguishable from those embraced by Verdon.

What exactly is different about the worldview of Yudkowsky and Verdon? There is really only one significant difference: Yudkowsky wants ASI to adopt our values, and hence enable people like him to survive into a cosmic utopia. That’s why he thinks we need to temporarily pause ASI capabilities research, so that value-alignment research can catch up. Otherwise, these doomer and accelerationist worldviews are identical twins. Both Yudkowsky and Verdon are pro-ASI and pro-extinctionist. Both want the posthuman successors that we create or become to colonize the universe and climb the Kardashev scale. Both believe that enveloping stars in Dyson swarms and building a multigalactic civilization will lead to utopia. Etc.

Insofar as these two men — both school dropouts, as it happens — dislike each other, it’s the result of the narcissism of small differences. On an ideological map, their views occupy almost exactly the same location.

Subscribe now

Conclusion

I wrote this article primarily for skeptics of the TESCREAL acronym. I do not claim to have covered all relevant issues, though I hope this provides a compelling case that the TESCREAL acronym is useful — indeed, indispensable — for understanding what the Silicon Valley worldview is, where it came from, how it developed, and why it’s problematic and dangerous (the last of which I didn’t cover here). Those who object to it likely don’t understand why the acronym was coined or what work it’s supposed to do within critiques of Silicon Valley’s push to build an AI God.

By the way, the people you spoke with at the fictional party are based on real individuals. I tried to present their views as accurately as possible based on what I know about them. Here’s a key in case the hyperlinks weren’t enough:

• Bendisi = Daniel (Bendisi) Kokotajlo

• Niklas = Nick Bostrom

• Bill = Gil Verdon

• Eli = Eliezer Yudkowsky

• Daniel = Daniel Faggella

• Sam = Sam Altman

• Ray = Ray Kurzweil

• Larry = Larry Page

• Mario = Dario Amodei

• Dennis = Demis Hassabis

• Ketamine user = Elon Musk

• Toby = Toby Ord

• Nick = Nick Beckstead

• Holden = Holden Karnofsky

• Peter = Peter Thiel

• Michael = Michael Druggan

• Jeffrey = Jeffrey Ladish

As always:

Thanks for reading and I’ll see you on the other side!

I just left North Macedonia, in the Balkans. It’s a beautiful (middle-income) country, with some marvelous hidden gems and an extraordinary history. For example, during the period of Classical Greece, king Philip II of Macedonia unified much of Greece, enabled by his rather terrifying martial innovation, the Macedonian phalanx. His son, Alexander the Great, who was tutored by Aristotle, then conquered the Persian Empire to the east.

Some history basics: the Classical Period extended from the final Athenian tyrant’s fall in 510 BCE until Alexander the Great’s untimely demise in 323 BCE. It was during this period that Socrates, Plato, and Aristotle philosophized. This was followed by the Hellenistic period, during which people in Macedonia built structures — like the amphitheater shown below — that I was lucky enough to see while visiting. The Hellenistic period begins with Alexander the Great’s death and extends almost to the birth of Christ — i.e., to 30 BCE, when Cleopatra died. Cleopatra was the last de facto Egyptian pharaoh and a direct descendant of the Macedonian general Ptolemy I Soter, who founded the Ptolemaic Kingdom — of Egypt, which Cleopatra ruled — and was a contemporary of Alexander the Great. Fascinating stuff!

The reason North Macedonia includes “North” in its name is rather amusing: the country wants to become part of the European Union, but the Greeks, who are already part of the EU, objected to “Macedonia” without the “North.” The reasoning is that the Macedonian kingdom of Philip II and his son Alexander the Great included most of Greece, and hence the current state of Macedonia covers only the northern portions of this empire. Under Article 49 of the Treaty on European Union, all members states must unanimously vote to allow new states to join, which means that Macedonia needs to get Greece on board. Hence, they changed the name to “North Macedonia” in hopes of appeasing Greece so it votes “yes.” If you talk to people here, it’s Macedonia. But if any Greeks are listening, it’s North Macedonia!

(EDIT: See Wolfgang’s comment below for some clarification. He knows more about this issue than I do!)

***

After spending April in London with a good friend, I spent the past two months sojourning in a lovely lakeside city in Macedonia called Ohrid (pronounced Oak-rid), surrounded by snowcapped mountains (until the snow melted), lush vegetation, and beautiful water. Here’s what Wikipedia says about the place:

Ohrid … is the largest city on Lake Ohrid and the eighth-largest city in the country, with it recording a population of over 38,000 inhabitants as of 2021. Ohrid is known for once having 365 churches, one for each day of the year, and has been referred to as the “Jerusalem of the Balkans”.

(Note that there are also plenty of mosques!)

The city is rich in picturesque houses and monuments, and tourism is predominant. … In 1979 and in 1980, respectively, Ohrid and Lake Ohrid were accepted as Cultural and Natural World Heritage Sites by UNESCO. Ohrid is one of only 40 sites that are part of UNESCO’s World Heritage that are Cultural as well as Natural sites.

I’m sharing some videos and pictures of the city below, along with a few captions. Now off to Western Europe, where I’ll have to endure much hotter temperatures! Once I settle in, I’ll publish a lengthy article (next week) on the TESCREAL ideologies, exploring them from a novel perspective. I’m looking forward to it! Wishing everyone the very best, Émile

View of Lake Ohrid from the lawn outside a cafe that I frequented. Many of my articles over the past several weeks were written here:

A lovely wooden walkway that winds around the hilly old section of the city:

View on the edge of the old city, where the stunning Church of Saint Jovan the Theologian at Kaneo — built sometime before 1447 — is located. You’ll see it at the end:

One of the first videos that I took during a morning run. This is on the opposite side of the hill where the old city is located, about a 10/15-minute walk to the Church of Saint Jovan:

Below is the magnificent Ancient Theatre of Ohrid, a 20-second walk from where I stayed in the heart of the old city. It was built circa 200 BCE, meaning that it dates back to the middle-ish era of the Hellenistic period. Spectators witnessed gladiator fights (!), as well as Roman executions of Christians. The latter was met with such hostility by locals that they subsequently buried the structure in disgust, thus inadvertently preserving it for future generations. It was accidentally rediscovered in the 1980s!

Walking up to my apartment through the old city:

Gateway to one of the 365 churches in Ohrid!

Another view of my “office” on the grass next to the lake. The distant mountain was covered in snow when I first arrived:

Wandering through the old city at night:

Same as above. Notice the chapel in the background:

Samuel’s Fortress, capital of the medieval First Bulgarian Empire. It was likely built on an older structure dating from around 300 BCE, constructed by Philip II. What you see here dates from sometime in the late 900s CE:

At another chapel:

I sent this to my family with the comment: “I hate how compelling this slogan is!”

Adorable — but how often does it tip over while turning?

Macedonian special:

Inside one of the chapels:

These events were held quite frequently while I was here:

As always:

Thanks for reading and I’ll see you on the other side!

I just finished reading Luke Kemp’s recent book Goliath’s Curse, and I’d highly recommend it to readers of this newsletter. It’s a sprawling survey of societal collapse across the entire sweep of civilizational history, and his account of Paleolithic violence offers a devastating refutation of key claims made in Steven Pinker’s The Better Angels of Our Nature.

As it happens, I had a desk right next to Kemp’s when I was visiting the Centre for the Study of Existential Risk (CSER) in 2019. We hung out, chatted about global catastrophic risks over drinks, and kept in touch after I departed Cambridge (though we haven’t spoken much over the past few years). I knew him as an erudite scholar with an unusual talent for approaching complex topics in a neutral, objective manner.

To my pleasant surprise, Goliath’s Curse is even more erudite than I anticipated. It’s a work of superb scholarship, in my opinion, and constitutes a major contribution to the global catastrophic risk (GCR) literature. I changed my mind about a number of important issues while reading it.

There’s no way to summarize Kemp’s tome in a single article. What follows is a loosely organized survey of what stood out to me as key points, drawn mostly from memory and a few quotes I jotted down while reading. Please excuse the spaghetti tangle of thoughts that I’m about to throw at the wall!

“Hobbes’s Delusion”

Kemp argues that life in the Paleolithic wasn’t “solitary, poor, nasty, brutish, and short,” as Thomas Hobbes famously claimed. For roughly 300,000 years, we lived in relatively small, highly egalitarian communities in which group decisions were made democratically. As he notes, “modern nomadic-egalitarian groups who have the lifestyles closest to our deep past are especially democratic.”

Violence wasn’t very common, and “Palaeolithic violence was probably highly personal. With little inequality, there would have been few material rewards for killing others.” Those who tried to dominate others were punished: expelled or, in extreme cases, killed — an example of “personal” violence. If you didn’t like the way your group was being run, you could “vote with your feet” by migrating to another group. There was, indeed, a constant interchange of individuals passing from one group to another.

People were also probably quite happy most of the time. Ethnographies of contemporary hunter-gatherers, Kemp points out, “are replete with reflections on how carefree and happy they are, with close-knit communities and a propensity to spontaneously burst into song and dance.”

With the rise of civilization, malnutrition became common and human health suffered, as reflected in the average height of agricultural populations falling for some 12,000 years (we didn’t regain our height until the 19th and 20th centuries). Higher levels of violence, along with oppression, slavery, patriarchy, and martial conflicts resulted. Life in the Paleolithic wasn’t perfect — infant mortality, for example, was much higher than it is today — but the Hobbesian assumption that life was terrible isn’t supported by the paleoanthropological data.1 Life actually got appreciably worse in many respects after the Neolithic revolution.

Civilization Is Domination

Kemp repeatedly emphasizes that people have an inherent aversion to being dominated. This is why those who tried to dominate others in our hunter-gatherer past were ostracized or executed. Yet domination is a core feature of what he calls “Goliaths,” as he explains:

The problem is that most of us are uncomfortable in recognizing the most common element of civilization: rule through domination. … A more apt label for these systems of violence is “Goliath.” A Goliath is a collection of hierarchies in which some individuals dominate others to control energy and labour.

The emergence of Goliaths amounts to, in his phraseology, evolutionary backsliding. By subjecting people to oppressive hierarchies of dominance, it made us more like our evolutionary relatives, the gorillas and chimpanzees. In his words:

Rather than a stepladder of progress, this movement from civilization to Goliath is better described as evolutionary backsliding. Our egalitarianism and counter-dominance shaped our bodies and minds and supercharged our cultural evolution. They made us unique and helped us navigate the Palaeolithic. The move towards Goliath — whether it be through class or patriarchy — made us look more like the harems of patriarchal gorillas and the chimpanzee hierarchies built on violence and politicking.

Collapse and Inequality

A common denominator of collapse across history is wealth inequality. As a Guardian article about Kemp’s book puts it, “history shows that increasing wealth inequality consistently precedes collapse.”

I was aware of the literature on this, but was never convinced that inequality was a causally central factor in collapse. Kemp changed my mind: incompetence, corruption, wasted resources, overuse of resources, frivolous projects of self-aggrandizement (think of the Colossus of Rhodes or Trump’s proposed “Triumphal Arch”), the embrace of loyalists over experts, nepotism, foolish wars of conquest, etc. etc. follow the emergence of elites with excessive wealth and power.

Such phenomena make Goliaths more susceptible to threats that shock the system: famines, volcanic eruptions, earthquakes, disease outbreaks, incursions from foreign armies, ecological destruction, and so on. Whether these threats pose “existential” risks to society depends on how large the threat is, as well as how vulnerable and exposed the society is, as well as its ability to effectively respond once the threat materializes. As Kemp explains:

In the field of disaster risk, we refer to four determinants: threat, vulnerability, exposure, and response … Take the case of a tsunami. The tsunami (threat) is only dangerous if you lie in its way (exposure), and you lack robust shelter that could withstand the wave (vulnerability) or the ability to escape (response).

Consider the case of climate change. Most US citizens are worried about this. Parents fear that their children will inherit a planet much less livable than the one they grew up in. If there were a popular vote, the US would have taken action to mitigate the problem years ago.

But elites have lobbied the government not to act, while spreading climate propaganda to the masses. Curbing climate change would be bad for their personal wealth, power, status, and ambitions. Meanwhile, loyalists in the Trump administration reject the climatological consensus, with Trump himself calling it a “Chinese hoax.” Consequently, the US is doing nothing to mitigate the problem, and has failed to reduce its exposure and vulnerability to climate change’s increasingly harmful effects. This makes collapse much more likely — thanks to the incompetent, avaricious, power-hungry elites who rule the country.

Paleolithic Altruism

In the aforementioned Guardian article, Kemp says that

as elites extract more wealth from the people and the land, they make societies more fragile, leading to infighting, corruption, immiseration of the masses, less healthy people, overexpansion, environmental degradation and poor decision making by a small oligarchy. The hollowed-out shell of a society is eventually cracked asunder by shocks such as disease, war or climate change.

Indeed, Kemp notes that “the biggest predictor of resilience was how democratic or extractive the society was.” This is substantiated by considerable historical evidence, as he explains:

One study of the Late Antique Little Ice Age found that more inclusive societies had less famine, disease, and conflict, and less change in their rituals or how they were organized. Those results have been verified in another study of thirty-three societies covering twenty-two climate-related disasters over nine world regions, four continents, and a thousand years. More democratic groups tend to respond better to crises.

The question is, why? Local people could respond more quickly and effectively to new challenges than distant elites who were often buffered from threats such as a drought. This meant that more dramatic changes (such as collapse) were needed. A second benefit is that shared decision-making means that people in a community had more social connections, which created more effective collaboration (what academics usually call ‘social capital’) during times of stress. This holds true today: acts such as voting, deliberating, and being meaningfully involved in civic institutions (whether that be a sports club or a political group) foster closer and wider social ties. Those ties keep communities together when catastrophe strikes.

Kemp points out that during and in the aftermath of disasters, people tend to come together and help each other. Mass chaos, violence, and “anarchy” don’t usually take over. Rather, our Paleolithic tendencies toward cooperation and altruism often emerge. I’ve personally experienced a few emergency situations in my life, and it’s amazing how complete strangers suddenly become trusted friends.

Kemp continues:

A plethora of studies on modern disasters support the idea that more politically inclusive societies are more resilient ones. As one group of disaster-risk experts concluded, “there is near-unanimous agreement that people’s participation is essential to reducing the impacts of disasters on society.” One indicative study [analyzed] 150 countries over the period 1995-2009 found that more democratic countries, with more state capacity, had fewer fatalities and fewer people affected during disasters. This was true for a range of disasters, from floods to earthquakes. Strong states have more resources to deal with disasters, while democratic accountability ensures that aid is distributed wisely and not just concentrated on the rich and powerful. Inclusive political institutions create more flexibility, effective responses, and unified societies, and they prevent abuses of power.

Does Collapse Benefit the 99%?

Kemp argues that societal collapse has very often benefitted the 99%. It can actually open the door to more progress, innovation, creativity, art, and knowledge. It frequently makes life better for nearly everyone except the elites.

Two examples are the fall of Rome and the 14th-century Black Death. After Rome fell, “people actually got taller and healthier,” Kemp reports. After the bubonic plague swept across Europe, killing up to 50% of the population,

peasants were now valuable enough to force concessions out of the landlords and barons. Wages doubled or tripled in many regions, and the pay gap between men and women decreased. People grew taller and healthier. Life expectancy increased and heights grew from a low of 168cm during the Black Death to a high of around 174 cm.

However,

while the Black Death was a blessing for the survivors and their descendants, these positive effects were not evenly distributed. … Regions that already had inclusive institutions benefited in the long term from the Black Death, while those without them did not.

He adds:

The fall of Rome and the Black Death not only give us an insight into the surprising relationship between collapse and prosperity, but they can also help us to understand how petty European kingdoms became colonial conquerors.

What’s More Deadly than Collapse?

Indeed, some of the worst events in all of human history were the result not of the fall of Goliaths but their emergence and expansion. He writes:

The emergence of these colonial Goliaths was one of the most violent processes of human history. The colonization of the Americas itself may have claimed up to 10-11 per cent of the world’s people at the time. While the numbers are highly uncertain, only one other event seems to come close to this: the waves of the Black Death in the fourteenth century … We can think of these as “global decimations.”

The cause was, of course, the brutal mass murder of Indigenous peoples by the “more civilized” Europeans. It was also due to the Columbian Exchange, namely, “the widespread transfer of plants, people, pathogens, and culture across the Atlantic Ocean.”

But “there is another critical reason,” Kemp writes, that “colonization efforts succeeded: indigenous allies.” Consider the Aztecs. They were a militaristic Goliath that sought territorial expansion and oppressed the peoples they conquered. He explains:

Those who resisted the imperial expansion of the Aztecs faced higher taxes and harsher working conditions. Many of the conquered territories resented such domination, while unconquered neighbours such as the Tlaxcalans (a confederation of four oligarchic republics) were in a state of near perpetual war with the Aztecs.

The Tlaxcalans thus offered the Spaniards a deal: join us to overthrow the Aztec Empire, to which “the Spanish wisely agreed.” That’s how the Spaniards managed to defeat the Aztecs: it was actually Indigenous peoples rebelling against their neighboring Goliath, because no one wants to be dominated.

What’s Civilized About Civilization?

The key point, however, is that most people looking at history get things exactly backwards: it’s not the disintegration of Goliaths that tend to inflict mass suffering on the 99%, but their very existence. Dominance hierarchies are the real villains of history.

This flips the script that Pinker wants us to believe. Over and over again, Goliaths have inflicted unimaginable horrors on people. “Civilization” seems to imply “civilized,” yet is there anything civilized about the way hierarchies oppress, subjugate, exploit, dominate, and control people through the perpetual threat of violence? As Kemp writes at the very beginning of his book:

[C]ivilization seems to be a wholly inappropriate term for what we see in Cahokia and Rome. It is derived from the Latin civilitas, which implies restraint, moderation, and good political conduct. Slavery, war, patriarchy, sacrificing teenage girls as the priests of Cahokia did, punishing slaves with crucifixion as in Rome, or dropping atomic bombs on cities as the US did less than a century ago, are not what most of us would consider civilized conduct. Orchestrating mass sacrifices or building bombs that vaporize cities does not require altruism, civility, or democracy. It requires top-down obedience enforced through the threat of violence.

In a memorable passage nearly 400 pages later, Kemp says this about states and companies:

States and companies (and perhaps now algorithms) are not just amplifiers of power and decision-making; they also inflate the worst in us: the darker angels of our nature. Think of a modern state, such as the US or Russia, embodied as a single person stalking the streets. Imagine what they are like. Overwhelmingly self-obsessed and self-interested, constantly boasting of their achievements, continuously suspicious of their neighbours, persistently posturing and hoarding guns and knives, and greedy to the extent of never giving even just 1 per cent of their earnings to those in need. That state person, if it took a psychiatry test, would rate disturbingly high on the dark triad (psychopathy, narcissism, and Machiavellianism).

I’d never thought about it this way, but it’s so true! States behave exactly like narcissistic, Machiavellian psychopaths.

The World Is Different Now

Interestingly, Goliaths have actually become more resilient over time. We don’t see the same sorts of collapse in recent history that we did long ago, especially within the Global North. States may “fail,” as happened with the Soviet Union and Yugoslavia, but the basic infrastructure and cultures that comprise those states have persisted. This, however, shouldn’t make us too confident that a genuine, catastrophic collapse doesn’t lie in our future. As Kemp writes:

The world appears to be growing both increasingly robust and more fragile. I call this the “Death-Star Syndrome,” after the space-weapon in the film Star Wars: A New Hope that can annihilate entire planets but can itself be destroyed by a single well-placed blow. Our world is incredibly powerful and robust, yet surprisingly fragile if hit hard enough in the right place.

First, we no longer live in isolated Goliaths that could rise and fall without affecting their neighbors. When Rome fell, other civilizations elsewhere in the world hardly noticed. That’s no longer the case: globalized modernity is so vastly interconnected that a collapse in one place could have profound consequences for all other regions. (As the saying goes, “when the US catches a cold, the rest of the world comes down with pneumonia.”)

Second, we’re now facing threats on a planetary scale. Even a regional nuclear war between India and Pakistan could cause global temperatures to plummet by a staggering 4 C. A nuclear war between the US and Russia could kill over 5 billion people — and a relatively sudden loss of 5 billion people would have truly devastating consequences for the global economy, agriculture, geopolitical stability, and so on. As Kemp observes, the power to decide whether or not to initiate a nuclear war lies in the shaky hands of a select few Dark-Triad madmen: Donald Trump, Vladimir Putin, etc. This is a terrifying concentration of destructive power in people we wouldn’t trust to babysit our children.

There are also global-scale threats from climate change, engineered pandemics, artificial superintelligence, and killer robots, to name a few. While certain risks have declined over the centuries, Kemp notes that “fat-tailed” risks associated with such phenomena have notably increased. Consider this curve:

This shows that low-impact risks (on the left) have a higher probability of occurring than high-impact risks (on the right). However, the “fat-tail” on the right means that some very low-probability risks would result in widespread, global-scale harms. You can think of it like this: most people who will ever die in pandemics will die in the absolute worst pandemics. Those pandemics — falling on the far right of the graph — will take so many people that they’ll overwhelm the numbers of people who’ve died from far more common localized outbreaks and epidemics. Here’s how Kemp puts it:

You are about twice as likely to take your own life than to be murdered. You are more likely to die from cancer than from any infectious disease. Unfortunately, this does not mean that we live in the safest of times or are on the cusp of defeating disease. Far from it.

Deaths from disease and violence follow a “fat-tailed distribution.” They are marked by a few outsized events, such as the Spanish flu, the Black Death, and the Columbian Exchange. This makes observing an average from one year, or even ten, essentially meaningless. The average will be brutally skewed by these “tail events.” The extremes, not the average, are the main source of risk. Importantly, these tails are becoming fatter and longer over time. Even if the world on average seems safer and healthier, the extremes are growing in likelihood and severity.

Advanced AI, synthetic biology, climate change, biodiversity loss, and perhaps future nanotechnology carry fat-tailed risks such that, if a low-probability worst-case outcome were to occur, it would decimate the world.

Making matters worse, wealth inequality has skyrocketed. The catastrophe of World War II leveled things off a bit, but disparities have risen since the 1970s. This harbinger of collapse has probably never been more salient, certainly not on the global stage. The world is run by people who are actively destroying it, while those same people prepare for the coming collapse by building apocalypse bunkers in Hawaii and New Zealand.

Here’s how Kemp explains the unsettling uniqueness of this perilous period in the Guardian:

He cites three reasons why the collapse of the global Goliath would be far worse than previous events. First is that collapses are accompanied by surges in violence as elites try to reassert their dominance. “In the past, those battles were waged with swords or muskets. Today we have nuclear weapons,” he says.

Second, people in the past were not heavily reliant on empires or states for services and, unlike today, could easily go back to farming or hunting and gathering. “Today, most of us are specialised, and we’re dependent upon global infrastructure. If that falls away, we too will fall,” he says.

“Last but not least is that, unfortunately, all the threats we face today are far worse than in the past,” he says. Past climatic changes that precipitated collapses, for example, usually involved a temperature change of 1C at a regional level. Today, we face 3C globally. There are also about 10,000 nuclear weapons, technologies such as artificial intelligence and killer robots and engineered pandemics, all sources of catastrophic global risk.

Yikes.

Those Damn Agents of Doom

All of this “existential risk” — a term I don’t like, although Kemp isn’t using it the way as TESCREALists do — comes from a small group of elite actors that he calls “Agents of Doom.” This refers to the Big Oil fossil fuel companies, the Big Tech AI companies trying to build superintelligence, and military-industrial complexes (“a collection of militaries and the arms manufacturers they tend to contract to”). It’s these companies, run by assholes who likely score high on the Dark Triad traits, that are responsible for pushing civilization toward disaster.

I sometimes hear radical environmentalists say, “So what if humanity goes extinct due to climate change? We deserve extinction given how we’ve trashed the biosphere.” My response points out the obvious: “This misdiagnosis the problem. It’s a tiny elite of endlessly avaricious capitalists with a fetish for the accumulation of infinite wealth and power who are responsible for our predicament. If climate change were to take only them out, there might be a degree of justice served. But the 99.9% don’t deserve to die for the actions of a tiny few.” This response generalizes to superintelligence, bioweapons research, nuclear proliferation, and so on. If our global Goliath collapses, blame the Agents of Doom.

What, then, can we do?

Termination, Chains, or Overthrow

Kemp argues that we have three options before us:

Looking at both the long-run trajectory of Goliath and current trends, our Global Goliath appears destined to hurtle down one of three paths. The first, and most likely, is self-termination. The second is a world in chains. The third is a world in which we somehow manage to shackle and control it.

Kemp argues that we must revive aspects of our Paleolithic lifeways, namely, governance through democratic means. He writes:

Open democracy and inclusive institutions are essentially harnessing these better angels of our Palaeolithic psychology. They are using the best of us to make better decisions and closer communities: to harness our collective intelligence and impulses to help each other. They are moving closer to our past as equals in a web of sharing, constantly moving communities: a fluid civilization. Our Palaeolithic emotions are not a curse. The curse is the institutions and arrangements which bring out the worst in us.

We must therefore empower the people, utilize the wisdom of crowds, and strive to develop novel forms of democratic control to collectively guide the ship of civilization through the unprecedentedly dangerous 21st century. “If we start to properly fund experiments in democracy,” Kemp writes, “and reward them like we reward start-ups in tech, then who knows in what profound new ways we’ll be able to harness the wisdom of crowds.”

Insurmountable?

Unfortunately, Kemp notes that “reversing inequality and establishing deep democracy in a lasting fashion has … been a rarity throughout history.” But it’s precisely because this “is a highly unlikely outcome” that

makes it worth fighting for even more desperately. While throughout history the collapse of a Goliath was usually temporary and a liberation, in the future it threatens to be permanent and take us and much of the planet with it. We are passengers on a journey that looks likely to end in chains or evolutionary suicide. Our Global Goliath will die explosively unless we kill it first.

Indeed, this points to a disheartening fact about future collapse scenarios: not only could they take the entire global system down, but the world left behind would be far less welcoming than the world was after past collapses. As Kemp writes, “it will be trickier in the future for a population to bounce back post-collapse.”

Climate change will, as alluded to above, render large portions of the planet subject to extreme weather events and unsurvivable heatwaves (exceeding the 95 F wet-bulb temperature), while global biodiversity loss and the sixth major mass extinction event will leave ecosystems far more impoverished than they otherwise would be. Consider that the global population of wild vertebrates — mammals, fish, reptiles, birds, amphibians, etc. — has declined by a staggering 69% since 1970. Extrapolate this trend into the future, and then imagine your children or grandchildren trying to survive by hunting, gathering, and fishing.

Extinction Vs. Collapse

An interesting point that Kemp brings up, which I’ve made on numerous occasions over the years, is that total human extinction is “far-fetched,” at least within the foreseeable future. He writes that

even an extreme collapse with extensive infrastructure damage, temporary climate change, and a population loss of 99.99 per cent would have only a 0.1-0.3 per cent chance of causing extinction. There would still be 810,000 survivors, who would have a “grace period” of easy access to remaining supplies (canned goods and produce) and equipment. That might buy them enough time to learn the necessary survival skills such as basic foraging and agriculture. Killing off every single human is an incredibly high bar.

I agree with this.2 I’ve argued that the probability of extinction is very low, though I’d quickly add that the probability of collapse this century looks exceptionally high — on the order of >90%. In other words, I would be very surprised if civilization survives beyond the next 74 years or so. If we could overthrow the oligarchy and put everything under democratic control, as Kemp advocates, then I’d say we have a pretty good chance of making it through. But that also seems quite unlikely — “it’s easier to imagine the end of the world than the end of Goliath,” to quote Kemp. However, he adds that this

is because we are still stuck in Goliath traps: status races, arms races, and races to the bottom (where firms or states try to outcompete each other by exploiting people and environments more intensely). Decreasing nuclear stockpiles, slashing carbon emissions, and making our societies more democratic are all completely feasible, and doing so in the long term will mean escaping these traps and digging up the root causes of existential risk.

Some Criticisms

I have many friendly criticisms of Kemp’s magisterial book. One of the most pressing is that he doesn’t go far enough in talking about how certain Agents of Doom, namely, the AI companies, are trying to bring about a version of human extinction in which our species is replaced by a successor species of posthumans. If the TESCREAL worldview were successfully realized, the result wouldn’t be collapse — yet the extinction of our species would almost certainly result. See my “Pro-Human Manifesto” for all the gory details.

The usual way of thinking about the relation between extinction and collapse within GCR studies is this: extinction would entail collapse, but collapse need not entail extinction. That’s true if one’s talking about final human extinction, but not terminal without final extinction. And it’s precisely terminal without final extinction that many TESCREALists would hope to realize, if they’re successful. Because avoiding final extinction would by definition mean that we have successors, our “civilization” could persist even if our species were to completely die out.

I wish Kemp had spent a bit more time on this extremely important issue, though as someone who spent years writing a book of comparable length, I understand how utterly exhausting it is to try to mention everything that could or should be mentioned. The lacuna is completely understandable. Perhaps he’ll address this when a second edition comes out!

As always:

Thanks for reading and I’ll see you on the other side!

A central theme of this newsletter over the past year has been the pro-extinctionist implications of the TESCREAL worldview. Many people in Silicon Valley want to create a new species of posthumans to replace us, and many see artificial superintelligence (ASI) as the key to doing this. Companies like Anthropic aren’t merely building a technology, they’re trying to realize a worldview.

However, as I note in Part 1 of my “Pro-Human Manifesto,” there are many different kinds of pro-extinctionism circulating in the Valley. The four main questions that these pro-extinctionists disagree about are:

1. Whether posthumans should be digital or biological. Most Valley dwellers claim that posthumanity should take the form of digital beings, a view that I call digital eschatology. However, a small minority of people, like Peter Thiel, embrace an eschatology of biological transhumanism, according to which some or all future posthumans should be biological.

2. Whether posthumans should be an evolutionary extension of our species or take the form of autonomous AI systems that are, as such, separate and distinct from us. In the first case, humanity (or some portion of humanity — the elites) would be transformed into posthumanity, which would then replace our species. In the second, we would essentially create a new evolutionary lineage de novo. These AIs would then supplant humanity like an invasive species let loose on an island. We can call this second option digital eugenics.

3. Whether posthumanity must embrace “our values” (whatever they are) or should adopt their own unique values, which may be wildly different from ours. Accelerationists like Gil Verdon (aka “Beff Jezos”) don’t seem to care one bit whether posthumanity embodies “our values.” As Daniel Faggella puts it, they want our “successors” to have values that are “alien, inhuman.” In contrast, the field of “AI safety” is founded on the idea that it matters greatly whether posthumanity is “aligned” with “our values” — hence, the value-alignment problem. If ASI isn’t value-aligned, then the future won’t contain human values, extended to the stars. The result would be an “existential catastrophe.”

4. Whether particular individuals alive today should have the opportunity to become one of these future (most likely digital) posthumans. Many TESCREALists — suffering from debilitating death anxiety — think it’s very important that they themselves get to join the ranks of posthumanity in a cosmic utopia someday. This is why people like Ray Kurzweil, Nick Bostrom, Eliezer Yudkowsky, and Sam Altman have arranged to have their brains cryogenically frozen or chemically preserved if they die — so they can be resurrected in posthuman form.1 Others would say that what ultimately matters is the creation of posthumanity — full stop — whether or not specific individuals survive the transition to a new posthuman era. This gets at a difference in emphasis between what theologians would call “personal eschatology” and “cosmic eschatology,” respectively.

These are the central points of disagreement among Silicon Valley pro-extinctionists, and sometimes those disagreements can be vicious (see the “narcissism of small differences”!). I’ve argued that we will witness increasingly public debates among the tech elite not about whether pro-extinctionism is bad, but about which type of pro-extinctionism is best. I think the Musk-Page debate is an example of this.

***

If you’d like some concrete examples of the different pro-extinctionist views that these disagreements yield, check out my “Pro-Human Manifesto.” Since publishing that, additional examples have popped up.

For instance, consider the following quote from Dean Ball, who was just recently hired by OpenAI “to lead a newly formed Strategic Futures team focused on shaping the company’s approach to frontier AI policy and governance.” Ball also has affiliations with the Foundation for American Innovation, the Heritage Foundation, and Yale University, and he was the Senior Policy Advisor for Artificial Intelligence and Emerging Technology for the Trump administration. In a recent interview, he says this:

Weirdly enough, if you think that this moment is — I don’t necessarily believe this — but a lot of people would say we’re living through this kind of eclipse of the human intellect, where we’re in the final days of humans being the primary actors on this planet. And that soon, machines will rise. There is this irony in that I think that whole transformation, I think humans … will actually go through a very “main-character energy” period of time, as that transformation occurs, even if it ultimately does mean that the machines ultimately become the primary actors, there’ll be this period. … In that sense, it’s a very beautiful time period to live through because in a Dionysian way, there’s a lot of ugliness and bad, but there’s a beauty in the ugliness … when a star dies, it grows super-big into the Red Giant, and it’s like that … as you watch this final flowering of humanity in the birthing of the machine intelligence, it’s like, you see this greatness in human effort. And I feel like we do see some of that going on in the world. I think we’ll see much more of it. I think it will be a heroic time period that we live through, basically, that’s what I’m saying. At least it could be — or villainous time period, but there’ll be a lot of opportunities for great people.

Put differently, we’re about to see the desperate last gasps of humanity as our digital progeny smother us to death. This is the “final flowering of humanity in the birthing of machine intelligence,” yielding a “heroic” — or “villainous” — period during which “great people” will have “a lot of opportunities.” The world as we know it will come to an end, and a new era of AI super-beings will commence. This is it, folks!

The arrogance, hubris, and tyranny of this vision is difficult to overstate. Sorry for cursing, but who the fuck are these people to decide for all of humanity when, how, and whether a Digital God arises, thus sentencing our species to extinction? I’m reminded of a quote from Sam Altman, who said in 2016: “We’re planning a way to allow wide swaths of the world to elect representatives to a new governance board. Because if I weren’t in on this I’d be, like, why do these fuckers get to decide what happens to me?”

Sounds good, right? Except that OpenAI didn’t follow through — they never allowed representatives of people around the world to participate on its governance board. Instead, they stacked the board with people like Quora CEO Adam D’Angelo, Facebook CTO and Salesforce co-CEO Bret Taylor, American attorney Nicole Seligman, and Altman himself. “Fuckers,” indeed.

***

Or, consider an OpenAI employee who goes by “roon” online (his real name appears to be Tarun Gogineni). In a recent exchange on X, someone said this:

humans are broken, rotting robots. transhumanists are correct to be disgusted with this situation.

Roon responded:

but [if] you were right[,] though, nothing human or even transhuman makes it out of the near future.

The original poster answered:

I want to survive, roon! just not as chopped unc,

where “unc” is slang for “uncle,” and is sometimes used “to imply that someone is old, getting old, or acting older than their age.” Roon then replied once more:

I think we can and will survive, just in a curtailed lightcone as enhanced pets to creatures of incomprehensible power if we let the technology evolve naturally2

But could “we” survive as “enhanced pets,” as roon claims? Depending on the nature of these “enhancements,” it probably wouldn’t be “us” who survives. Even if “we” do survive, why the hell would anyone want to be a “pet” owned and controlled by “creatures of incomprehensible power”? This utterly bleak future appears to be a good-case scenario from roon’s perspective. There really is no other word than “fuckers.”

***

It reminds me of Altman saying that the only way humans will survive the arrival of ASI is by “merging” with machines. In other words, to avoid extinction (annihilation), we will need to undergo a different form of extinction (“the merge”). The fact that Altman and his employees, like roon, are attempting to unilaterally impose this future on humanity without our consent or permission is utterly infuriating. Who do these “fuckers” think they are?!

Roon, by the way, previously declared on X that

things are accelerating. Pretty much nothing needs to change course to achieve AGI … Worrying about timelines is idle anxiety, outside your control. You should be anxious about stupid mortal things instead. Do your parents hate you? Does your wife love you?

As I wrote about this in Truthdig, he’s saying that

AGI is right around the corner and its development cannot be stopped. Once created, it will bring about the end of the world as we know it, perhaps by killing everyone on the planet. Hence, you should be thinking not so much about when exactly this might happen, but on more mundane things that are meaningful to us humans: Do we have our lives in order? Are we on good terms with our friends, family and partners?

He also once said this:

Make no mistake: these people are ideological extremists whose worldviews are intimately bound up with pro-extinctionist fantasies. They talk about “utopia,” but this utopia isn’t for us — it’s for the posthumans who replace us.

The Climate Crisis Heats Up

In 2008, when I was still a graduate student in neuroscience, The Guardian published an article about James Lovelock, originator of the Gaia hypothesis. He discusses climate change, and the article closes with this paragraph:

What would Lovelock do now, I ask, if he were me? He smiles and says: “Enjoy life while you can. Because if you’re lucky it’s going to be 20 years before it hits the fan.”

Not too bad in the prediction department! As you probably know, Europe is in the midst of a massive heatwave — the second of 2026 so far. Parts of France and Spain are melting in 111 F (44.4 C) temperatures, and records are being broken all over the place. Here are some screenshots of forecasts that I took a few days ago:

Others shared temperature forecast maps on social media showing the sizzling heat covering France, Spain, and elsewhere:

One person shared a fictional forecast from 2014 of what European temperatures will be in 2050, juxtaposed with an actual forecast for June of this year. Wild stuff!

The atmospheric heat is accompanied by an equally massive spike in marine temperatures, as this person points out:

All of this, by the way, is happening before the Super El Nińo kicks in! Next year is going to be horrific …

***

Interestingly, Europe is warming up twice as fast as the rest of the world (while “the Arctic is heating nearly four times faster than the global average rate and twice as fast as previously expected”). But if the AMOC and Gulf Stream collapse — which appears increasingly likely — Europe will fall into a “little ice age,” whereby temperatures will plummet to roughly the same levels as in Canada. (Note that Paris is slightly south of Winnipeg, Canada, while London is slightly north of Calgary.)

This will devastate European agriculture, with effects potentially greater than those inflicted by the sort of heatwaves it’s now experiencing.

It’s hard to overstate how infuriating this predicament is. I agree with scholars who argue that fossil fuel CEOs and political figures like Trump should be tried for crimes against humanity. If indeed 2 to 4 billion people die prematurely in the next few decades due to climate change, is there any punishment that could possibly fit the crime?

Subscribe now

***

Incidentally — to tie things back to the first half of the article — I had a conversation with a TESCREAL colleague back in 2017 that went like this:

Climate change won’t pose an “existential risk” (an event that threatens the creation of cosmic utopia), he said, until later this century. But we will likely build ASI in the next decade or so. If the ASI is misaligned, it will kill everyone on Earth and annihilate the biosphere, thus rendering climate change irrelevant. However, if it’s aligned, it will use its god-like powers of mentation to devise a way of reversing the climate crisis, thus rendering climate change irrelevant. Hence, we shouldn’t worry about climate change, but instead focus on building an aligned ASI.

Many people in Silicon Valley seem to accept this view. Others appear to think that climate change is irrelevant because we’ll all be digital mind-clones living in virtual reality worlds soon, so solving climate change at any point doesn’t really matter (because, you know, who gives a damn about other species?).

Former Google CEO Eric Schmidt channeled the view of my old TESCREAL colleague when he said in 2024: “My own opinion is that we’re not going to hit the climate goals anyway because we are not organized to do it and yes the needs in this area [AI] will be a problem. But I’d rather bet on AI solving the problem than constraining it.”

A similar sentiment was recently satirized by an Instagram account named “bpd.news,” which made up this quote from Jeff Bezos:

“We have to look at the macro-picture of our planet’s future,” Bezos reportedly stated during a recent tech symposium. “Biological limits are real, but digital potential is infinite. If we starve our data infrastructure of cooling resources just to sustain baseline human comfort, we are actively delaying the birth of a super-intelligence that could solve all of our resource problems in the first place. Sometimes you have to prioritize the intelligence that will save us over the biology that slows us down.”

Although Bezos never said that, it’s a pretty accurate description of what the tech elites actually believe! I think that’s what the satirists were going for.

That’s it for today. I hope everyone is well. As always:

Thanks for reading and I’ll see you on the other side!

First, I’d like to publish an AMA — ask me anything — next week. If you have any questions you’d like to ask me about anything I’ve written or said, please leave them in the comments section below!

You’re more than welcome to criticize the positions I’ve defended in this newsletter or elsewhere. I really mean it when I say: please feel free to challenge me, as I appreciate push-back and don’t assume that I’m right about everything! I will do my best to answer everyone’s questions. :-)

“Fuck Earth” — Elon Musk

I have a new article out in Truthdig. Please check it out if you have the time. It argues that Musk and SpaceX are lying to investors. “Duh,” I hear you saying while moving the cursor to close this tab, but I haven’t seen anyone make the particular argument that I advance. Briefly put, it’s this:

1.

Musk has repeatedly claimed that he’ll place “humans” — including “you” — on Mars. But there is no scientifically plausible way for biological humans to survive on the Red Planet.

1.1

Mars is extremely hostile to biological life. It is literally less habitable than Earth was during the mass extinction event 66 million years ago in which all non-avian dinosaurs died out. How do we know? Because mammals survived the asteroid blast and ensuing impact winter. If you were to put a mammal on Mars, it would die almost instantly. Mars has almost no atmosphere, there’s no magnetic field, and the soil (Martian regolith) is literally poisonous. So is the water. Plus the average temperature is -80 degrees F, about 50 degrees lower than the average temperature on the peak of Mount Everest in January. Does living on top of Mount Everest for years or decades sound appealing? Well, it would be a tropical beach paradise with palm trees and coconuts compared to life on Mars.

1.2

Even just getting to Mars will be extremely difficult. High levels of space radiation (which causes cancer, dementia, etc.), the deleterious effects of prolonged microgravity, and the psychological toll of being cooped up in a tiny spacecraft for 9 months make traveling to our planetary neighbor enormously challenging.

There is no Planet B for us, in our current form. The farthest colonies from Earth that biological humans will be able to occupy will be on the moon, or perhaps within large spacecraft like O’Neill cylinders (which themselves pose enormous engineering challenges).

2.

The only “creatures” who will successfully colonize Mars will be digital in nature. There are two options here:

2.1

We could create autonomous AIs to fulfill our cosmic Manifest Destiny. Think of a more advanced form of Grok. We could give them robotic appendages and launch them to the Red Planet. The problem is that it would be them rather than us colonizing Mars.

2.2

We could upload our minds to computers, and somehow connect these minds to robotic bodies. By transferring our minds to hardware, “we” could more easily survive the brutal conditions of space and treacherous features of Mars.

These are not, of course, mutually exclusive. Uploaded humans in the form of software minds could travel to Mars alongside autonomous AI astronauts.

3.

However, at least some of these fully digital beings will need to be conscious, because the central mission of SpaceX is to spread “the light of consciousness” into the solar system and beyond. If these digital beings were to lack the capacity for subjective experiences, they’d be no different than the machines we currently have on Mars. And no one thinks that those machines mean we’ve colonized the planet. But there are major problems with this requirement:

3.1

Mind-uploading might not be possible. It could be that mental states, including states of consciousness, aren’t computational in nature. Maybe minds aren’t “software” running on the “wetware” of nervous systems. Maybe we’ve been misled by the most recent metaphor in a long chain of techno-metaphors: the brain is a machine of some sort, a telegraph, a telephone switchboard, and now a digital computer. If the brain isn’t a computer and our mental states aren’t algorithmic, then mind-uploading won’t be possible.

There are plenty of other reasons mind-uploading might not work. The lumps of Jell-O between our ears, which jiggle every time we move our heads, are the most complex objects in the known universe. Assuming the brain is a computer, we have absolutely no idea what level of detail would need to be simulated in silico to reproduce mental states, including consciousness, or whether this level of detail could be simulated in silico at all. Mind-uploading might be impossible for technological rather than philosophical reasons.

This matters because if mind-uploading is impossible, then digitized humans will never walk on Mars. SpaceX should be very anxious about this.

3.2

More generally, artificial consciousness of any sort might not be possible. What if we build an AI with a completely different cognitive architecture relative to our brains (like the LLMs that run current AI chatbots)? How could we ever know whether it’s conscious or not? What scientifically objective “test” could we use to verify with virtual certainty that it’s conscious?

SpaceX should also be very nervous about this. Without robust verification, there’s no way to be confident that “the light of consciousness” has in fact been extended to Mars, even if Mars is teaming with artificial beings building what appears to be an advanced civilization.

4.

SpaceX will need to solve all of these formidable problems to fulfill its mission of spreading “the light of consciousness” to Mars and beyond. It will need to figure out how to upload our minds to computers, which might be metaphysically or technologically impossible.

If that’s the case, SpaceX will need to figure out which types of cognitive systems can realize states of artificial consciousness, assuming that computer hardware can realize any such states at all. Maybe computer hardware isn’t the right material, with the right properties, to give rise to phenomenal experience.

But even if it is the right material, SpaceX will need to develop a “test” for consciousness, which is an extremely challenging if not impossible task in itself, because consciousness is intrinsically subjective and science relies on objective (non-subjective) evidence. How could you obtain objective evidence for something that’s intrinsically subjective?

Mission Impossible: Four Challenges

The entire mission of SpaceX fundamentally depends on solving these immense problems. Does Musk admit this in public? No, because he’d spook investors, most of whom are under the (false) impression that humans will someday soon live happily in futuristic Martian colonies.

To put the point a different way, here’s a list of the challenges facing SpaceX:

• (a) Figure out a way to terraform Mars so biological humans can live on it. SpaceX would need to create a Martian atmosphere; eliminate the poison in Martian soil and water; overcome the hazards of low-gravity and microgravity, super-subfreezing temperatures, and there being no planetary magnetic field; figure out how to grow crops; and build spaceships that fully protect their inhabitants from deadly space radiation and other lethal hazards during the 9-month trip to Mars. This would be unbelievably difficult, if it’s possible at all. We can’t even terraform Earth — in the midst of a climate crisis that could take 4 billion lives several decades from now — and we tried to create a “closed biosphere” with the Biosphere 2 experiment and failed miserably. We aren’t even close to figuring out how to establish self-sustaining colonies on Antarctica, which is heaven compared to Mars. Good luck doing all of this 142 million miles from Earth. If something goes wrong and you run out of food or oxygen, you’ll have to wait (at least) 9 months for supplies to arrive. While waiting, you’ll die.

Or:

• (b) Figure out how to upload human minds to computers. Then figure out how to connect these disembodied software beings to robotic appendages that would enable them to interact with the Martian environment. Musk and his fanatical fans utterly fail to appreciate the challenges here, assuming these things are even possible.

And/or:

• (c) Figure out how to instantiate consciousness in artificial systems with non-human architectures — some advanced version of Grok.

And, crucially:

• (d) With respect to (b) and (c), devise a highly reliable and accurate “test” to verify that these artificial systems are actually conscious. This is absolutely critical. The entire mission depends on knowing that whatever beings end up on Mars aren’t mindless machines like the rovers currently picking up rocks and conducting little experiments on our planetary neighbor. Pfff.

Musk says he wants to take the “fiction” out of “science fiction.” But doing that will be far more difficult than most people naively assume. This isn’t just about building a giant phallus and launching it into space. There are major open questions in both AI and philosophy that will need to be answered for SpaceX to fulfill its grandiose mission.

No one talks about these, but they’re just as integral to the mission as the science of rockets is — and perhaps even more formidable than the famously formidable challenges posed by rocket science. Like brain surgery, rocket science is “very complex”:

Musk cannot wish away these challenges. He’s a professional bullshitter, but reality keeps slapping him in the face. That’s why, as I write in the Truthdig article:

[G]ullible investors remain entranced by his messianic aura. In 2016, Musk confidently predicted a self-sufficient Martian colony with 1 million people by 2024. That same year, SpaceX announced that it was “planning to send [the rocket] Dragon to Mars as soon as 2018.” Musk later “said the first unmanned missions could take place in 2022,” and by 2020 he was predicting the first human missions to Mars by 2026. When named the 2021 Person of the Year by Time, he told the magazine that he’d “be surprised if we’re not landing on Mars within five years.” When 2026 arrived, he shifted once again in promising that “an unmanned Starship would soon depart for Mars, paving the way for human-led missions,” and “if those landings go well, then human landings may start as soon as 2029, although 2031 is more likely.”

Early this year, SpaceX abruptly pivoted from colonizing Mars in the near future “to building a self-growing city on the Moon,” which Musk claims his company “can potentially achieve … in less than 10 years.” He now says that “Mars would take 20+ years.”

Lol. He will, of course, continue to push back the date beyond “20+ years,” because reality can’t be wished away. As always:

Thanks for reading and I’ll see you on the other side!

Was Hitler a mass murderer? You may be surprised to know that there’s no historical evidence that he ever personally killed anyone — other than himself in 1945.1 (See, e.g., here and here.) Military records indicate he took no lives during World War I. He apparently didn’t murder anyone during the Beer Hall Putsch or the Night of the Long Knives. And he probably didn’t shoot his niece, Geli Raubal, despite rumors to the contrary.

If “murderer” means “one who personally kills another,” then Hitler wasn’t a mass murderer, because he wasn’t a murderer at all.

But of course Hitler was a mass murderer. Who would argue against this? His decisions resulted, whether intentionally or not, in an estimated 17 million civilian deaths during World War II. Though he never once pulled the trigger, tightened the rope, or held the knife, Hitler was morally responsible for these deaths. If it weren’t for him, those people wouldn’t have died the way they did, when they did.

Subscribe now

By the very same criterion, Elon Musk is a mass murderer.

To my knowledge, Musk has never personally killed anyone. But he did terminate USAID last year, calling it a “criminal” organization that “has to die.”

“USAID was not just an apple with a few worms inside,” he wrote on X, “it was a festering ball of worms in the shape of an apple.” He also said it “is/was a radical-left political psy op.” DOGE then engaged in what one senior official described as a “mafia-like takeover” of the agency.

According to the Lancet, the most prestigious medical journal in the world, the termination of USAID is projected to cause

more than 14 million additional deaths by 2030, averaging more than 2.4 million deaths per year. These deaths include 4.5 million among children younger than 5 years, or more than 700,000 deaths annually.

Late last year, Harvard reported that “hundreds of thousands” of people have already died since USAID was terminated. As of April 2026, the number may have risen to 1.6 million — and counting.

***

Hitler was responsible for 17 million civilian deaths over six years (1939 to 1945). Musk will likely be responsible for 14 million deaths over five years (2025 to 2030), if the Lancet’s projections are accurate. If Hitler was a mass murderer, then so is Musk. Indeed, given the numbers, one is forced to conclude that Musk is a mass murderer of world-historical proportions — on par with some of the worst dictators in human history.

Consider that:

• Saddam Hussein “was responsible for the deaths of hundreds of thousands of people” during his 23-year reign of terror in Iraq.

• George Bush was responsible for upwards of 1 million civilian deaths by having illegally started the 2003 Iraq War.

• Pol Pot was responsible for some 2 million deaths in four years during the Cambodian genocide.

• Stalin was responsible for about 1 million deaths, though another 2.3 died from “neglect and irresponsibility.” The Soviet famine of 1932-33 took another ~6.5 million.

• Mao was responsible for around 30 million deaths over two years due to the Great Leap Forward.

As with Hitler, there is no verified, universally accepted evidence2 that Saddam Hussein, George Bush, Pol Pot, Stalin (after becoming leader of the Soviet Union), or Mao ever personally killed anyone.

Think about that for a moment. These monsters took lives by pulling the levers of power available to them. They implemented policies that killed untold millions due to starvation, deprivation, neglect, and outright slaughter. In some cases, the deaths weren’t specifically intended. But we still count them nonetheless, because they could have — and should have — been anticipated.

By the numbers, Musk is worse than Hussein, Bush, Pol Pot, and Stalin, but not as bad as Hitler or Mao — though he’s not far behind Hitler. Very few people in history have been personally responsible for so many innocent deaths as the CEO of SpaceX, the world’s first trillionaire.

To put 14 million deaths in further perspective:

• Hurricane Katrina took the lives of 1,392 people.

• The atomic bombings of Hiroshima and Nagasaki killed about 246,000 people.

• The Antioch earthquake in 526 CE caused 250,000 deaths.

• The Rwandan genocide took the lives of up to 662,000 Tutsi.

• The HIV/AIDS epidemic has killed some 700,000 people in the US, as of 2018.

• The Armenian genocide took the lives of up to 1.5 million people.

• As many as 13 million civilians died in World War I.

These are all events that, insofar as people know about them, carry a certain unsettling “dread factor.” Yet, by the numbers, all are less bad than the catastrophe unfolding as a result of USAID’s termination by Musk.

Note also that the 14 million estimate doesn’t include deaths that will occur after 2030. The loss of USAID could have effects that linger for much longer.

***

Musk didn’t kill all of these people by ordering their execution. But so what? Hitler also didn’t order the execution of every person his regime buried.

What’s the difference between smothering a hospital patient with a pillow and intentionally unplugging their life-support machine? Would you not consider the person who intentionally unplugs the machine as having committed cold-blooded murder? Would they not be a killer?

Or, imagine seeing a young child drowning in a shallow lake. An altruistic Samaritan runs into the water to heroically rescue the innocent life. But just as she picks up the child, a random asshole intervenes, forcibly flings the baby from the Samaritan’s arms, and the baby drowns. Then you find out that this random asshole was Musk himself.

This is what he’s done with USAID. It was providing life-saving medical support to millions of people around the world. It was their life-support machine. And he pulled the plug, resulting in millions of patients — many of whom are innocent children drowning in a metaphorical lake — dying, often in excruciating circumstances. As my friend Devin Kellis wrote in his guest post for this newsletter:

Peter Donde (age 10), Evan Anzoo (age 5), and Achol Deng (age 8) died from loss of access to human immunodeficiency (HIV) medications; Gy Gael (age 8) died from an infection without access to antibiotics; Babagana Bukar Mohammed (age 7) died from complications of sickle cell disease amidst health clinic closures; and Yamah Freeman, a mother of two, bled to death amidst ambulance delays due to fuel shortage while expecting her third child.

Even if Musk didn’t fully understand the extent to which USAID was the life-support machine for tens of millions, it was his moral duty — as a person who forced his way into a position of immense power — to investigate. There is no ethical excuse for the world’s richest person to set fire to an organization that just 5 minutes of research would have shown him is keeping millions of children above water. He is blameworthy for all of these deaths. Can anyone seriously argue to the contrary?

***

If one compares Musk to Ted Bundy, his murderous actions are completely off the charts. Ted Bundy confessed to 30 murders but might have killed up to 100 women. The latter number is 0.000714% of the total deaths caused by Musk within the next 4 years, according to the Lancet.

Why does Bundy strike fear, dread, and disgust in our hearts but not Musk, whose body count is literally 140,000 times larger? Why does the evil of Bundy disturb our conscience more than the actions of DOGE and its now-trillionaire leader? Is it just psychic numbing and scope neglect? As Stalin famously said, “If only one man dies of hunger, that is a tragedy. If millions die, that’s only statistics.”

When considering the enormity of Musk’s moral crimes, I often think about a poignant scene at the end of Charlie Chaplin’s Monsieur Verdoux, about a man who murders women to steal their money (as I recall — it’s been decades since I watched it). During his trial, Chaplin’s nefarious but charming character delivers a speech in which he says:

As for being a mass killer, does not the world encourage it?
Is it not building weapons of destruction for the sole purpose of mass killing?
Has it not blown unsuspecting women and little children to pieces — and done it very scientifically?
As a mass killer, I am an amateur by comparison.

That final line haunts me. Bundy was an amateur by comparison to Musk. The degree of callousness and psychopathy required for Bundy to do what he did is unfathomable to morally normal people like you and me.

But the same is true of Musk — it takes a real psychopath with zero conscience, empathy, or compassion to recklessly obliterate an agency that kept millions of children alive, and then express literally zero remorse or compunction for the profound and lethal harms directly caused by his intentional, willful actions.

Fuck the billionaires, but especially fuck the trillionaire. We live in an infinitely sad world, because the chasm between how good things could be and how bad they actually are is enormous and growing. It didn’t need to be this way.

Subscribe now

As always:

Thanks for reading and I’ll see you on the other side!