First, a Recap of the Past Week
Remember when the former OpenAI employee Joshua Achiam, who worked at the company for 9 years, wrote last September:
There is a fact about the future that I feel many people are not facing for reasons that are largely psychological: there are going to be rogue AIs that exist in the world, that will replicate in the wild, and that will attempt to acquire resources for themselves. There will be rogue AIs that try to get money and power. They’re going to be a facet of the information ecosystem going forward.
With respect to AIs replicating “in the wild,” recall that Anthropic warned just last month that “small AI models may be months away from being capable enough to self-replicate on commercial IoT [Internet of Things] devices like smart-ovens and off-brand Android TV boxes. A replicator like this would be challenging to shut down, since there’s no centralized point to shut off.” Pfff.
Well, perhaps we’re currently witnessing a preview of the future described by Achiam:
Last Thursday, Microsoft reported an outage with Outlook and Teams. People were unable to send emails, having to rely on text messages instead.
On Friday, the FAA grounded all flights in the United States, leaving some 1.5 million people stranded. This was the most widespread grounding of planes since September 11, 2001. Many (including me) suspect it might have resulted from a new AI system called “SMART,” fully implemented by the FAA around November 10, that aims to advise “air traffic controllers to assist in managing air traffic across the entire national airspace system.” Suspicious timing!
Then, on Monday, a widespread service outage hit major banks in the US, including JP Morgan Chase, Wells Fargo, and Bank of America, as well as the money transfer app Zelle. This comes after (unrelated) reports from late last month that “major U.S. banks are detecting unauthorized third-party AI agents operating inside customer accounts, with some agents attempting to take actions that appear to be independent of user instructions.” Hmm.
Also on Monday, United Airlines reported a “widespread system outage” due to “a technology issue.” NBC reports: “The union representing United pilots said in a statement that the outage had affected multiple pilot-facing applications, including crew communications and messaging systems, as well as the airline’s ability to provide final aircraft weight information for departing flights.” The company says that “there is no indication that this was a cybersecurity-related issue.”
Although it’s possible that none of these incidents resulted from a cyberattack or AI, one wonders if this is the sort of future we’re rushing into. AI will greatly enhance the ability of hackers to wreak havoc on our digital infrastructure, not to mention the potential damage that “swarms” of “rogue AI agents” of the sort that hacked into Hugging Face last July (thus committing a felony, for which no one has been held accountable) could inflict. As Axios recently reported, “OpenAI and other AI companies are privately gaming out scenarios for a public and political revolt after a catastrophic AI event,” which “many top AI researchers and executives believe … is inevitable” in the next 6 to 12 months.
Recall the massive cyberattack just last month against grocery stores across France, resulting in millions of dollars of wasted food. Then, a few weeks ago, there was a second wave of attacks targeting grocery stores in Belgium, Ireland, Estonia, Luxembourg, Finland, Sweden, and the Netherlands, once again resulting in millions of dollars of waste.
I fear that the coming year will witness more such attacks, and that the “top AI researchers and executives” are right that some of these incidents may be “catastrophic.”
Dario Amodei Is Misleading When He Promises that ASI Will Cure All Diseases
This raises the obvious question: Are these generative AI systems worth it? Are they a net benefit to society?
Dario Amodei, the TESCREAL CEO of Anthropic, proclaimed in a recent video:
You know, if today we look at things like cancer, Alzheimer’s, heart disease, mental illness, these will be relics of the past in, I think, less than a decade. … I think if we do this right, we can create an age of miracles where humanity is able to solve problems that have eluded us for millennia.
He reiterated this in another recent video:
We’re just at the beginning of what can be done in biology and medicine. … Every disease you can think of, heart disease, Alzheimer’s, cancer — we think of those the way we think of bubonic plague or smallpox today: that they’re relics of the past. I think we can have a much more beautiful world. I think we can have a world in which all of these problems are addressed.
It sounds starry-eyed. It sounds crazy. I really do believe it. I really do believe it can be done. I don’t know, five years, ten years, fifteen years, I don’t know exactly, but we’re going to try to do it as fast as we can.
This Isn’t the End Game
Here’s the problem: curing diseases isn’t the end game for TESCREALists like him. The goal isn’t to build an ASI that invents new therapeutic interventions to improve the human condition. That would be aiming for the moon rather than Alpha Centauri.
The real goal is to build an ASI that invents radical human enhancements to fundamentally alter the human condition, ultimately replacing it with a brand new, supposedly “superior” posthuman condition. Amodei’s focus on curing diseases in his public statements is part of a marketing strategy to trick people into thinking he’s on Team Human when, in fact, he’s a radical transhumanist on Team Posthuman.
Consider this passage from his essay “Machines of Loving Grace”:
I suspect AI-accelerated biology will greatly expand what is possible: weight, physical appearance, reproduction, and other biological processes will be fully under people’s control. We’ll refer to these under the heading of biological freedom: the idea that everyone should be empowered to choose what they want to become and live their lives in the way that most appeals to them.
“Biological freedom” is just Amodei’s less-weird-sounding term for what Extropian transhumanists called “morphological freedom”: essentially, the supposed “civil right” to be able to dramatically transform oneself into a posthuman however one chooses. His statements here may sound nice, at first glance, until one reflects on the broader societal consequences of “biological freedom” — indeed, on the implications of such “freedom” for the very existence of humanity.
After discussing the feasibility of “doubling the human lifespan” through AI-enabled breakthroughs and eugenic possibilities like “effective genetic prevention of mental illness,” he writes:
Taking this one step further, many people have experienced extraordinary moments of revelation, creative inspiration, compassion, fulfillment, transcendence, love, beauty, or meditative peace. … All of this suggests that the “space of what is possible to experience” is very broad and that a larger fraction of people’s lives could consist of these extraordinary moments. It is probably also possible to improve various cognitive functions across the board. This is perhaps the neuroscience version of “biological freedom” or “extended lifespans.”
That’s little more than a restatement of what transhumanists like Nick Bostrom and Toby Ord have said. In “Letter from Utopia,” “Transhumanist Values,” and “Why I Want to Be a Posthuman When I Grow Up” (lol), Bostrom argues that there is a potentially vast space of posthuman “modes of being” that could be far more “valuable” than our current human mode of being. The highest highs for humans today could be lower than the lowest lows for radically enhanced posthumans, he claims. That’s what Amodei is getting at when he talks about the “space of what is possible to experience.”1
Similarly, Ord declares in The Precipice that
peak experiences are not merely potential dwellings — they are also pointers to possible experiences and modes of thought beyond our present understanding. Consider, for example, how little we know of how ultraviolet light looks to a finch; of how echolocation feels to a bat, or a dolphin; of the way that a red fox, or a homing pigeon, experiences the Earth’s magnetic field. Such uncharted experiences exist in minds much less sophisticated than our own. What experiences, possibly of immense value, could be accessible, then, to minds much greater? Mice know very little of music, art or humor. Toward what experiences are we as mice? What beauties are we blind to?
Ord then draws the straightforward implication of this transhumanist-longtermist view: “Rising to our full potential for flourishing would likely involve us being transformed into something beyond the humanity of today. Remember that evolution has not stopped with humanity.” Elsewhere in the book, he says that “forever preserving humanity as it now is may also squander our legacy, relinquishing the greater part of our potential.”
In other words, our biological species must be radically transformed into a new species of posthumans, which is nothing less than a form of pro-extinctionism.
This is what Amodei is getting at, and given that he’s a diehard TESCREAList with deep roots in the longtermist tradition, there should be little doubt that he agrees with Ord’s “soft” pro-extinctionist conclusions. After all, from the radically eugenic, techno-utopian perspective of longtermism, why stop at “merely” perfecting humanity? Why not take that extra step and become an entirely new, supposedly superior posthuman creature?
In “The Adolescence of Technology,” Amodei argues that,
if we do get a century of medical progress in a few years, it is possible that we will greatly increase the human lifespan, and there is a chance we also gain radical capabilities like the ability to increase human intelligence or radically modify human biology. Those would be big changes in what is possible, happening very quickly. … There is also the issue of “uploads” or “whole brain emulation,” digital human minds instantiated in software, which might someday help humanity transcend its physical limitations … .
He adds that there are risks associated with mind-uploading, but of course such risks are just challenges to be overcome on the road to a posthuman utopia.
My point is that most of Amodei’s public-facing remarks focus on how ASI will improve the conditions of humanity. But the true goal is far, far more radical than this: the creation of posthumans through ASI. And what happens when “superior” posthumans rule the world? Our species, obviously, will be sidelined, marginalized, disempowered, and ultimately eliminated, as there are no good reasons for posthumanity to keep us around and multiple reasons to get rid of us. (And, indeed, various “ethical” reasons for euthanizing those who resist “enhancement,” as Eliezer Yudkowsky has argued.)
Hence, foregrounding the prospect of “curing all diseases” gives cover to the TESCREAL agenda that drives Amodei’s messianic pursuit of ASI. And this agenda, if successfully realized, would almost certainly entail the extinction of our species.
What About the Jobpocalypse?
Here’s a related issue that I haven’t yet discussed in this newsletter: how worries about the jobpocalypse, i.e., mass employment due to AI, completely miss the bigger picture — just as focusing on “curing all diseases” misses the bigger picture.
Amodei, along with other AI CEOs like Sam Altman and Elon Musk, have warned that advanced AI will soon take everyone’s job. This has triggered anxiety among the public about what will happen when humans become economically obsolete. It’s also why these CEOs have discussed “universal basic income” or, in Musk’s case, “universal high income.”
The problem is that widespread joblessness constitutes only a single tiny step in the larger process of replacing humanity itself with posthumans in the form of uploaded minds or autonomous ASIs. As Musk says, we are nothing more than the “biological bootloaders” for digital superintelligence.
Focusing on mass unemployment is like someone in the passenger seat of a fast moving car shouting: “Oh my god! Look up ahead! There are a bunch of potholes,” when 10 feet beyond those potholes is a massive cliff the car is about to careen over.
The potholes don’t matter, of course, because the car is seconds away from violently crashing into the valley below, killing everyone onboard in a spectacular explosion. Unemployment is a non-issue when one understands the eschatological journey these AI companies are taking us on.
Surgeon: “This Will Hurt, But It’s Necessary to Cut Your Life Short”
So, is generative AI worth it? A net benefit?
Thus far, the evidence clearly indicates “no.” GenAI poses increasingly catastrophic risks to our modern global civilization. What we’ve witnessed the past week may pale in comparison to what could happen, as an unambiguous and direct result of AI, in 2027. (Add to this the devastating effects of the Super El Niño. Next year will likely be horrific, and quite possibly a bloodbath.)
Here I want to emphasize a point I’ve made in previous articles: the TESCREAL extremists leading the AI companies will, when pushed, ultimately point to the promise of utopia as “justifying” all the harms they’re involuntarily inflicting upon society. Because their envisioned utopia is marked by literally “astronomical” amounts of “value,” it can “justify” virtually anything. When one zooms out and takes a cosmic perspective, the terrible shit that has already happened and will happen next year — including potential deaths caused by AI — will fade into nothingness. Compared to the vast quantities of “value” that ASI could produce if all “goes well,” such harms will constitute mere molecules in a single drop in the ocean.
And yet, in this utopia, our biological species will almost certainly go extinct. Think about that! Think about how the “justification” utilized by Amodei and other TESCREALists constitutes the worst possible insult to all the injuries they’re actively causing.
It’s like a surgeon telling their patient: “Look, I know that this procedure is going to hurt a lot. But it’s the only way to ensure that you die young.” That is what the TESCREALists at the AI companies are saying to us. Yes, there will be “inevitable” “catastrophic” AI incidents in the coming months, but these are worth it to create [checks notes] a paradise for posthumans.
In a previous article, I argued that if and when people around the world realize that those leading the AI parade are on Team Posthuman, it may very well spark a global apocalyptic conflict between Team Posthuman and Team Human. (Some would call it a “Butlerian jihad.”) I hope such a conflict is somehow avoided, but I fear this showdown may become increasingly unavoidable as AI wreaks havoc on civilization and the Overton window shifts toward TESCREAL (meaning that people wake up to the end game of these companies).
Conclusion
It’s worth savoring the moment, right now, because the past few months may be the least crazy period of our lives from here on out. Buckle up!
But what do you think I’m missing? What am I getting wrong? As always:
Thanks for reading and I’ll see you on the other side!
Here’s an excerpt of what Bostrom says in his tawdry “Letter from Utopia,” writing as if he’s a posthuman sending a missive backwards in time to current-day humans:
There is a beauty and joy here that you cannot fathom. It feels so good that if the sensation were translated into tears of gratitude, rivers would overflow.
I reach in vain for words to convey to you what it all amounts to… It’s like a rain of the most wonderful feeling, where every raindrop has its own unique and indescribable meaning – or rather a scent or essence that evokes a whole world… And each such evoked world is subtler, deeper, more palpable than the totality of the reality that you have encountered. One drop would justify and set right a human life, and the rain keeps raining, and there are floods and seas.


I wouldn’t say you’re getting anything wrong but, as to what you may be missing, I would say “ecology”. By this I mean that the whole radical transhumanist project relies on an astronomical amount of computing power, which relies on an astronomical amount of energy and resource extraction from a planet that’s already in ecological overshoot. So the “good” news is that Nature will eventually impose Her limitations upon these loonies and prevent them from completing their ultimate vision. But that’s cold comfort because the loonies will still cause (and already are causing) plenty of harm while they pursue their post-human goals. Your bullet points listing all the things that have gone wrong recently is a good example, with plenty more to come in 2027. Buckle up indeed.
“effective genetic prevention of mental illness,”
The irony is that the social and economical domain is actually where we could already be investing effort to prevent mental illness but we rarely do, while psychosis for instance is argueably the consequence of a neural creative network going haywire; by genetically addressing mental illness, they will be selecting away biologically healthy natural variation as soon as the dominant ideology culturally taxes it to dysfunction sometimes.