The writers of these AI scenarios should just drop the facade and embrace their actual passion: writing science fiction. Because that's what all of this is, basically. If presented as an epic speculative story of technological potential, this could be fascinating stuff. Presented as an actually possible projection of the future, it is pure fantasy and complete drivel, as you say.
After reading Yudkowsky's book on AGI I think its obvious these guys belong in the writing room of a shitty Hollywood movie. Very irritating these guys have any influence at all!
Robin Hanson also had an article on government against the prospect of space travel on grounds that it'd lead to new factions outside of their control which might turn against them.
You’re going a bit much on the italics in this one. And with the bold again. Less is more when it comes to formatting. 👍
And as I noted in my restacking to Notes, much of your critique of TESCREAL radicals is largely the same that we conservatives have made of leftist radicals for over a century.
Who are the leftists, anarchists, and Marxists to tell us how to live our lives? The same argument about why TESCREALists are wrong to try and force their ideology on you, apply to how it is wrong for you to force your ideology on us.
Interesting. I'd say that, speaking for myself, the essence of my far-left view is radical democracy. That's what it's all about. That's what Marx wanted -- in a world communist state, there are no people exploiting others, no classes, etc., and all decisions are made through a thoroughly democratic process. I want conservatives like yourself to have a say. That said, I'm highly critical of "utopian roadmaps" from any group, including leftists. This is one point of disagreement between myself and, say, Nathan Robinson (who says the concept of utopia is important and useful). Does that make sense?
Re: italics and bold, thanks again for your input. I've already altered my style (less bold and fewer italics) because of your feedback, and I may completely do away with such flourishes in future articles. I appreciate it (and if I could, I'd italicize and bold this last sentence!). :-)
“in a world communist state, there are no people exploiting others, no classes, etc., and all decisions are made through a thoroughly democratic process.” <— now who’s engaged in supernatural fantasy?
I agree with your piece by the way, but your politics are almost as baffling and fantastical as the TESCREALists (slight exaggeration there but ykwim)
I am 76 and in my 40’s i drifted into the “wise pessimism “ view of humanity. Thankfully I grew out of it and know that our collective resilient response to climate change and economic justice will be grounded locally but will use technology
I’m gonna have to give a hard disagree on the claim that Leftist politics are as “baffling and fantastical as the TESCREALists”.
As a Leftist myself, I understand the that some of my beliefs are incredibly difficult to bring to fruition, whether because of more powerful people with differing politics or the issue of human error. But I do believe that at the heart of Leftist politics there’s a notion of equality and trying to make the world a better place. I really don’t see that as fantastical, because if it were ideas like the Civil Rights Act would never have happened. People want and believe equality is a good thing, hence the Civil Rights Act happening.
But TESCREAList beliefs are inherently fantastical, because all their beliefs are based on hypotheticals and assumptions, and also ignore real-world problems that damage their hypotheticals (Like the end of Moore’s Law damaging their ideas of Exponential Growth in AI Intelligence).
Also about your top part of the comment quoting Émile and saying his views are “engaging in a supernatural fantasy”, why is this a problem exactly? Look maybe it is a fantasy…But is it necessarily a bad one? IMHO, it’s better than Capitalist fantasy where 3 whole companies control the market while continuing to say how “Free” the market is.
In the words of ParkerGetAJob: “How is it a supernatural fantasy? Argue against it instead of just calling it a fantasy.”
Also this comment isn’t attack on you or anything. I think we could hypothetically have a reasonable discussion about this.
The Civil Rights Act was an achievement of liberalism. The Black Panther Party was an achievement of leftism.
You’re welcome to call yourself whatever you’d like, my friend. But in our wonderful discussions I’ve come to see you very much as a principled liberal and progressive, rather than a leftist, socialist, or radical. These distinctions of ideologies on the Left are important. I’ve long seen them as akin to different denominations of Christianity.
Honestly in a lot of ways I do find myself agreeing with both regular Liberal Beliefs and Progressive Beliefs (Even more so it’s entirely possible my beliefs are more related to Liberal-Socialism).
However in this case, I was just using Leftist as a general statement in this discussion. Was that the wrong move? Possibly, but in the heat of the moment it seemed easier than listing my battleship coordinates of labels, haha.
I’m just wanting to make sure you understand that the Civil Rights Movement was a liberal achievement, not a leftist achievement. Liberal = based in John Locke. Leftist = based in Karl Marx. And progressive isn’t the same thing as leftist or radical either. Progressives like AOC and the squad work within the Democratic Party, electoral politics, and the constitutional systems.
Far-left radicals, Marxists, and anarchists don’t want to participate in changing or developing the existing system like progressives do. They want to destroy it and start from the beginning. They don’t want to amend the Constitution, they want to destroy it since they see it as an oppressive document meant to serve wealthy class interests, as Noam Chomsky and Howard Zinn taught.
Don't worry about me taking your comment as an attack! I don't!
I think your comment is basically identical to what a TESCREAList would say: "we are just trying to make the world a better place".
Also, you said "TESCREAList beliefs are inherently fantastical, because all their beliefs are based on hypotheticals and assumptions, and also ignore real-world problems that damage their hypotheticals"... I would argue leftist-communist beliefs suffer the exact same problems. For example, I notice you left out of your description of why leftist beliefs are hard to bring to fruition the notions of biology, human nature, ecological realities, etc. (you instead assume the only issues are powerful people, and human "error"). Therefore, you also leave out "real-world problems that damage [your] hypotheticals", and rely on unfounded assumptions about the malleability of human nature.
All of this you basically acknowledge by admitting that your beliefs basically are a fantasy, but a "good one" compared to some other fantasies. You ALMOST get to a substantive point with your mention of a market failure, but again, this provides no evidence or support for your views, which you admit are a kind of benevolent fantasy, and can easily be dispelled as a criticism of capitalism (especially if we are choosing the "best of a bad lot" when it comes to organising society, capitalism wins by far).
I would disagree that TESCREALists also want to “make the world a better place”, because they really don’t. Effective Altruists are a really great example of this: hypothetically speaking, being an EA sounds great on paper because it teaches the idea that if you are “effective” with your money, like donating to the right charity, you can maximize the amount of good you do in the world. But they really aren’t maximizing anything.
To quote Dom Helder Camara, “When I give food to the poor, they call me a saint. When I ask why the poor have no food, they call me a communist.” The reason I bring up the quote is because it relates to EAs. One of their favorite things to do is purchase mosquito nets for African nations to help prevent people being poisoned infected with malaria while they sleep. This is a great idea…in the extremely short term. Why not get at the source and just get ride of malaria? I know that maybe that can’t happen, but buying mosquito nets doesn’t fix the problem, it just puts a band-aid over it. EA has a LOT of funding invested in it. They have the money, and they choose not to do anything with it.
Also I disagree with your statement that I left out real-world problems that damage my hypotheticals. I mentioned “human error” (Which…throw me a bone here man that can be used interchangeably with human nature) and “powerful people” because they DO hurt the goals and beliefs of leftists. Additionally, how exactly do biology and ecological factors damage leftist beliefs at creating a better world? If anything, a leftist world allows ecological societies all over the world to thrive.
And one more thing, which is related to your statement that at the end that “when organizing society, capitalism wins by far”…wanna hear something crazy from a leftist? I actually think Capitalism, at the basic level, is a great idea!
A place where all men and woman are equal and have equal opportunity to enter a free market with products of their own creation and market it to local or the world market, and can decide how much to sell it for, and making good profits not just for themselves, but their workers and their communities?! Sigh me up!
But we don’t have that kind of capitalism. We have several major conglomerates that control the market which makes it very hard for people to break into said market. That’s honestly the exact opposite of capitalism, because what we have doesn’t promote a free market.
I’m so glad that you’ve found my encouragement on less formatting helpful! Here are some other writing tips that have long helped me, that came from an Oscar-nominated screenwriter who used to be my boss: overwriting is a sign that we lack confidence in what we are saying. When you write a 3,500-word article, how often do you really need all those words and how often could you have done it just as well in 1,500 words? If you establish the argument well in 1,500 words, do you really need 2,000 more words? Now sometimes you do and it’s totally fine. But more often than not, we do not. And we just go longer because we aren’t confident in our argument and feel like we need to do more. We’re not just trying to convince our reader then, we’re trying to convince ourselves too. And it shows for many readers.
And a piece in this length question also reflects on the degree you respect your reader. Another piece of advice that I got then: “Write as though your reader is smarter than you. Don’t assume you need to explain everything. They get it.” Just as less formatting is usually best, less words are usually better too. This is something that we in journalism and scriptwriting learn and develop which academics and philosophers almost never do. And now, of course, I will proceed to break this advice with a longer-than-necessary comment, but I gather you appreciate this greater depth and can recognize it as a sign of respect.
I understand your far-left view because I used to hold it 21 years ago. I was certainly a Chomsky fan during undergrad and still have all my old books of his with all the earnest little highlights and notes. Did you ever read Understanding Power? That was always my favorite of his and probably still. And I know the dissection of Chomsky's views very well because my professional mentors wrote and edited the book doing so. https://www.amazon.com/Anti-Chomsky-Reader-Peter-Collier/dp/189355497X/ Nowadays, I certainly appreciate David Graeber’s perspective a lot. I need to give The Dawn of Everything some real focused attention and need to pickup Bullshit Jobs. The radical anarchist critique and analysis are often thoughtful and useful, even if I take strong exception to the proposed solutions. Someone can be right in diagnosing a problem even if they are wrong in providing a solution.
The radical democracy that you and Chomsky advocate is very much a utopian roadmap and it's one of the reasons I moved on from it. It may feel right morally, but it’s just not practical and not in line with human nature. In America, our dominant ideology is not radical democracy, but constitutional democracy. We are a democratic republic. We are a representative democracy. Democracy is only one value in our nation and we balance it against others, particularly Rule of Law. And we conservatives have more than a say, we set the terms for the debate. We won this debate 250 years ago. The Constitution is what we established with that victory and it still endures today. It is the best political document ever written and the best form of government ever created. It defines American conservatism. Millions of Americans have risked their lives to defend it because they believe in it so much. Our fellow Americans who have joined our armed forces are willing to die to protect it. This is deeply serious stuff to most Americans. We were raised in the shadows of our fathers and grandfathers who went to war.
So how exactly are you and other Chomsky fans going to persuade the hundreds of millions of other Americans to abandon the belief in the inherent value of the Constitution and instead embrace "radical democracy?" How exactly are you going to get rid of the government based on the Constitution? Through writing Substack posts? Rhetorical questions, of course, but you are welcome to answer them if you wish.
In my view, your belief that you can achieve radical democracy is just as utopian as the TESCREALists thinking they can achieve AGI or the Singularity. Just as they are never going to achieve their utopia, you guys are not either.
But we conservatives have already achieved our utopia. We regard the Constitution and the government and country which has flowed from it, as the best that we’re going to get when it comes to utopia. The Constitution works. There isn’t a need to make fundamental changes to it. The debate between progressives and libertarians that has gone on for a century about how much welfare state to build on top of it will persist indefinitely and that’s fine.
Just as you see TESCREALists as engaged in a kind of secular religion - a critique which I share -- those of us on the Right see you the same way.
And I almonst never like to speak on behalf of the whole Right side of the spectrum as there is so much ideological diversity, but in this discussion I can do so on this point given that the far-left anarchists and communists being very wrong is something that even those of us on the center-right in #NeverTrump NeoCon Bulwark space can agree with the nuts on the far-right about. Heck, this is something that even gay libertarians and the Christian Nationalist bigots can agree on. And even on the other side of the spectrum, center-left liberals and many Democrat-partisan progressives will agree on it that radical democracy goes way too far.
This is a very longstanding ex-Marxist critique of the Left and one that I embraced 18 years ago. You’ve surely heard the expression “the god that failed?” Those who reject Biblical religion, always need something else to take its place. We have the God-shaped hole in our soul and if one does not fill it with an authentic spiritual tradition, then we will make the mistake of filling it with an inauthentic one.
I’m not trying to convert you to anything and you’re certainly welcome to continue holding your far-left views, but I hope that you’re finding one spiritual tradition or another or a healthy blend that can nourish you too, whether that’s East, West or whatever. The pursuit of radical democracy and left-wing politics will never be enough. Left or Right ideology is not the answer people are needing. Mystical practices which provide union with God are.
OK, I’ll stop overwriting now. You’re smart, you get it.
> We should have a say in how advanced AI is developed, and whether it’s even developed in the first place.
Given that both AI 2027 and AI 2040 warn about the risk of human extinction, it’s safe to say the authors are not necessarily in favor of AI being developed (even if AI 2040 also sees very good futures as an option).
AI 2040 also posits that the question of how to develop AI will be the main issue of the 2028 US presidential election. So it takes for granted that US voters, at least, will have a say.
The authors are not exactly writing from a position of political strength. Yes, they have some influence over AI companies, but not enough of it to get those companies to slow down. And when it comes to government they have almost no influence. It’s possible their influence might increase in the future if their factual predictions come true and they’re seen as prophetic. But for now they’re just taking a position.
> In their preferred future, Plan A, there is literally no consideration of climate change.
Plan A does mention that “The robot economy is shifting the bulk of activity to space, so that Earth’s environment and historic spaces will be protected.”
> Building a superintelligent machine isn’t going to somehow fix the problem before 2040.
No, but nothing is likely to fix the problem before 2040. Right now, most of the barriers to adopting clean energy are economic. An economic surplus would massively help. Thus, *if* superintelligent machines actually can be created and can have the absurd-seeming levels of economic benefit envisioned by AI 2040 (“even the poorest non-Americans get about $1M” inflation-adjusted per year), then the climate costs of operating them would likely be outweighed by the climate benefits on economic grounds alone. To whatever (less predictable) extent the machines accelerate development of technologies to fight climate change, that would also help.
If, on the other hand, AI doesn’t actually have such transformative potential, then yes, they represent significant climate cost for minimal climate benefit. But the authors are very much committing to a transformative scenario. If you don’t believe it then you don’t believe it, but there’s nothing really special about climate change here.
Quick response: I think the authors are very much pro-ASI, as long as the ASI is "value-aligned" (meaning "aligned with their TESCREAL worldview"). As Bostrom says, all paths to a utopian future go through ASI. My strong suspicion is that Kokotajlo endorses that. Also, it's worth disambiguating "human extinction" -- although Kokotajlo isn't very clear about the details of his preferred future in interviews, having known him in the past my guess is that he opposes final extinction (no posthumans) rather than terminal extinction (the disappearance of our species).
Re: climate change, despite the line you quote, AI 2040 doesn't take seriously the fact that by 2040 the world will be utterly torn apart by climate change. Hundreds of millions (maybe billions?) will have migrated or died (just think about the ramifications of that!). Entire ecosystems will have collapsed -- according to a Nature (I think it was -- or Science) paper from 2012 coauthored by some of the most renowned environmental scientists, there's a nontrivial chance of a sudden, catastrophic, irreversible collapse of the global ecosystem. This is just one of many possible tipping points that could radically alter, with devastating consequences, entire Earth systems. It will be a bloodbath, which is why I'm shocked the authors completely ignore it!!
Do you think the main barriers are economic, or political?
Thanks for these comments -- I really appreciate you sharing your thoughts and critiques! Thanks for reading. :-)
One has to imagine that some of these technologists have met the owners of their corporations? Surely they have bumped into the occasional VC in Palo Alto or San Francisco. Maybe when they came to their offices to pressurise management to make cuts to personnel or to outsource support to the Philippines. I certainly met a few when I worked in Silicon Valley; they forced me to outsource my whole team.
Most VC's are psychopaths they care only about power and money. For the VCs creative destruction is just destruction. Their private planes and their bunkers are ready, their pilots primed...
They intend to live a few months more like moles drinking champagne somewhere below New Zealand. Their libertarian personal freedoms finally guaranteed as they toast in crystal:
"I told you so!"
"Clink clink gentlemen."
"Think of all that value we have created."
"Shame about the prole population but there were too many of them anyway and they weren't so white, were they?."
Do the writers really believe that extra productivity from AI will be compatible with keeping millions of climate refugees safe, compatible with protecting the climate itself. How it this production going to be driven energetically? The mining, the fossil fuels, the waste, the water once used for irrigation now cooling data centres. What shall be the noxious side effects of this vast new production? Who will recycle the robots?
If the middle classes will be able to have exotic vacations, are we supposed to believe the planes that take them there be powered by green hydrogen? Get real! What about the wars that will break out everywhere when neighbouring populations squabble with machine guns over rivers in deserts ... Multiple rivers to multiple seas!
Get with the programme people. If we don't deal with climate chaos now there wont even be electricity in many cities by 2040.
The writers of these AI scenarios should just drop the facade and embrace their actual passion: writing science fiction. Because that's what all of this is, basically. If presented as an epic speculative story of technological potential, this could be fascinating stuff. Presented as an actually possible projection of the future, it is pure fantasy and complete drivel, as you say.
Totally agree. It will be fun to write articles in 2027 through 2032 about how wrong the authors were. :-)
I wonder if they credited Sid Meier? It seems they are riffing on Alpha Centauri and other games of his.
After reading Yudkowsky's book on AGI I think its obvious these guys belong in the writing room of a shitty Hollywood movie. Very irritating these guys have any influence at all!
Robin Hanson also had an article on government against the prospect of space travel on grounds that it'd lead to new factions outside of their control which might turn against them.
https://www.overcomingbias.com/p/most-likely-future-filter-world-govt
Interesting. Hadn't seen this! Thanks for sharing.
excellent
You’re going a bit much on the italics in this one. And with the bold again. Less is more when it comes to formatting. 👍
And as I noted in my restacking to Notes, much of your critique of TESCREAL radicals is largely the same that we conservatives have made of leftist radicals for over a century.
Who are the leftists, anarchists, and Marxists to tell us how to live our lives? The same argument about why TESCREALists are wrong to try and force their ideology on you, apply to how it is wrong for you to force your ideology on us.
Interesting. I'd say that, speaking for myself, the essence of my far-left view is radical democracy. That's what it's all about. That's what Marx wanted -- in a world communist state, there are no people exploiting others, no classes, etc., and all decisions are made through a thoroughly democratic process. I want conservatives like yourself to have a say. That said, I'm highly critical of "utopian roadmaps" from any group, including leftists. This is one point of disagreement between myself and, say, Nathan Robinson (who says the concept of utopia is important and useful). Does that make sense?
Re: italics and bold, thanks again for your input. I've already altered my style (less bold and fewer italics) because of your feedback, and I may completely do away with such flourishes in future articles. I appreciate it (and if I could, I'd italicize and bold this last sentence!). :-)
“in a world communist state, there are no people exploiting others, no classes, etc., and all decisions are made through a thoroughly democratic process.” <— now who’s engaged in supernatural fantasy?
I agree with your piece by the way, but your politics are almost as baffling and fantastical as the TESCREALists (slight exaggeration there but ykwim)
I am 76 and in my 40’s i drifted into the “wise pessimism “ view of humanity. Thankfully I grew out of it and know that our collective resilient response to climate change and economic justice will be grounded locally but will use technology
Embracing leftism isn’t the answer.
I’m gonna have to give a hard disagree on the claim that Leftist politics are as “baffling and fantastical as the TESCREALists”.
As a Leftist myself, I understand the that some of my beliefs are incredibly difficult to bring to fruition, whether because of more powerful people with differing politics or the issue of human error. But I do believe that at the heart of Leftist politics there’s a notion of equality and trying to make the world a better place. I really don’t see that as fantastical, because if it were ideas like the Civil Rights Act would never have happened. People want and believe equality is a good thing, hence the Civil Rights Act happening.
But TESCREAList beliefs are inherently fantastical, because all their beliefs are based on hypotheticals and assumptions, and also ignore real-world problems that damage their hypotheticals (Like the end of Moore’s Law damaging their ideas of Exponential Growth in AI Intelligence).
Also about your top part of the comment quoting Émile and saying his views are “engaging in a supernatural fantasy”, why is this a problem exactly? Look maybe it is a fantasy…But is it necessarily a bad one? IMHO, it’s better than Capitalist fantasy where 3 whole companies control the market while continuing to say how “Free” the market is.
In the words of ParkerGetAJob: “How is it a supernatural fantasy? Argue against it instead of just calling it a fantasy.”
Also this comment isn’t attack on you or anything. I think we could hypothetically have a reasonable discussion about this.
The Civil Rights Act was an achievement of liberalism. The Black Panther Party was an achievement of leftism.
You’re welcome to call yourself whatever you’d like, my friend. But in our wonderful discussions I’ve come to see you very much as a principled liberal and progressive, rather than a leftist, socialist, or radical. These distinctions of ideologies on the Left are important. I’ve long seen them as akin to different denominations of Christianity.
Honestly in a lot of ways I do find myself agreeing with both regular Liberal Beliefs and Progressive Beliefs (Even more so it’s entirely possible my beliefs are more related to Liberal-Socialism).
However in this case, I was just using Leftist as a general statement in this discussion. Was that the wrong move? Possibly, but in the heat of the moment it seemed easier than listing my battleship coordinates of labels, haha.
I’m just wanting to make sure you understand that the Civil Rights Movement was a liberal achievement, not a leftist achievement. Liberal = based in John Locke. Leftist = based in Karl Marx. And progressive isn’t the same thing as leftist or radical either. Progressives like AOC and the squad work within the Democratic Party, electoral politics, and the constitutional systems.
Far-left radicals, Marxists, and anarchists don’t want to participate in changing or developing the existing system like progressives do. They want to destroy it and start from the beginning. They don’t want to amend the Constitution, they want to destroy it since they see it as an oppressive document meant to serve wealthy class interests, as Noam Chomsky and Howard Zinn taught.
Don't worry about me taking your comment as an attack! I don't!
I think your comment is basically identical to what a TESCREAList would say: "we are just trying to make the world a better place".
Also, you said "TESCREAList beliefs are inherently fantastical, because all their beliefs are based on hypotheticals and assumptions, and also ignore real-world problems that damage their hypotheticals"... I would argue leftist-communist beliefs suffer the exact same problems. For example, I notice you left out of your description of why leftist beliefs are hard to bring to fruition the notions of biology, human nature, ecological realities, etc. (you instead assume the only issues are powerful people, and human "error"). Therefore, you also leave out "real-world problems that damage [your] hypotheticals", and rely on unfounded assumptions about the malleability of human nature.
All of this you basically acknowledge by admitting that your beliefs basically are a fantasy, but a "good one" compared to some other fantasies. You ALMOST get to a substantive point with your mention of a market failure, but again, this provides no evidence or support for your views, which you admit are a kind of benevolent fantasy, and can easily be dispelled as a criticism of capitalism (especially if we are choosing the "best of a bad lot" when it comes to organising society, capitalism wins by far).
I would disagree that TESCREALists also want to “make the world a better place”, because they really don’t. Effective Altruists are a really great example of this: hypothetically speaking, being an EA sounds great on paper because it teaches the idea that if you are “effective” with your money, like donating to the right charity, you can maximize the amount of good you do in the world. But they really aren’t maximizing anything.
To quote Dom Helder Camara, “When I give food to the poor, they call me a saint. When I ask why the poor have no food, they call me a communist.” The reason I bring up the quote is because it relates to EAs. One of their favorite things to do is purchase mosquito nets for African nations to help prevent people being poisoned infected with malaria while they sleep. This is a great idea…in the extremely short term. Why not get at the source and just get ride of malaria? I know that maybe that can’t happen, but buying mosquito nets doesn’t fix the problem, it just puts a band-aid over it. EA has a LOT of funding invested in it. They have the money, and they choose not to do anything with it.
Also I disagree with your statement that I left out real-world problems that damage my hypotheticals. I mentioned “human error” (Which…throw me a bone here man that can be used interchangeably with human nature) and “powerful people” because they DO hurt the goals and beliefs of leftists. Additionally, how exactly do biology and ecological factors damage leftist beliefs at creating a better world? If anything, a leftist world allows ecological societies all over the world to thrive.
And one more thing, which is related to your statement that at the end that “when organizing society, capitalism wins by far”…wanna hear something crazy from a leftist? I actually think Capitalism, at the basic level, is a great idea!
A place where all men and woman are equal and have equal opportunity to enter a free market with products of their own creation and market it to local or the world market, and can decide how much to sell it for, and making good profits not just for themselves, but their workers and their communities?! Sigh me up!
But we don’t have that kind of capitalism. We have several major conglomerates that control the market which makes it very hard for people to break into said market. That’s honestly the exact opposite of capitalism, because what we have doesn’t promote a free market.
I’m so glad that you’ve found my encouragement on less formatting helpful! Here are some other writing tips that have long helped me, that came from an Oscar-nominated screenwriter who used to be my boss: overwriting is a sign that we lack confidence in what we are saying. When you write a 3,500-word article, how often do you really need all those words and how often could you have done it just as well in 1,500 words? If you establish the argument well in 1,500 words, do you really need 2,000 more words? Now sometimes you do and it’s totally fine. But more often than not, we do not. And we just go longer because we aren’t confident in our argument and feel like we need to do more. We’re not just trying to convince our reader then, we’re trying to convince ourselves too. And it shows for many readers.
And a piece in this length question also reflects on the degree you respect your reader. Another piece of advice that I got then: “Write as though your reader is smarter than you. Don’t assume you need to explain everything. They get it.” Just as less formatting is usually best, less words are usually better too. This is something that we in journalism and scriptwriting learn and develop which academics and philosophers almost never do. And now, of course, I will proceed to break this advice with a longer-than-necessary comment, but I gather you appreciate this greater depth and can recognize it as a sign of respect.
I understand your far-left view because I used to hold it 21 years ago. I was certainly a Chomsky fan during undergrad and still have all my old books of his with all the earnest little highlights and notes. Did you ever read Understanding Power? That was always my favorite of his and probably still. And I know the dissection of Chomsky's views very well because my professional mentors wrote and edited the book doing so. https://www.amazon.com/Anti-Chomsky-Reader-Peter-Collier/dp/189355497X/ Nowadays, I certainly appreciate David Graeber’s perspective a lot. I need to give The Dawn of Everything some real focused attention and need to pickup Bullshit Jobs. The radical anarchist critique and analysis are often thoughtful and useful, even if I take strong exception to the proposed solutions. Someone can be right in diagnosing a problem even if they are wrong in providing a solution.
The radical democracy that you and Chomsky advocate is very much a utopian roadmap and it's one of the reasons I moved on from it. It may feel right morally, but it’s just not practical and not in line with human nature. In America, our dominant ideology is not radical democracy, but constitutional democracy. We are a democratic republic. We are a representative democracy. Democracy is only one value in our nation and we balance it against others, particularly Rule of Law. And we conservatives have more than a say, we set the terms for the debate. We won this debate 250 years ago. The Constitution is what we established with that victory and it still endures today. It is the best political document ever written and the best form of government ever created. It defines American conservatism. Millions of Americans have risked their lives to defend it because they believe in it so much. Our fellow Americans who have joined our armed forces are willing to die to protect it. This is deeply serious stuff to most Americans. We were raised in the shadows of our fathers and grandfathers who went to war.
So how exactly are you and other Chomsky fans going to persuade the hundreds of millions of other Americans to abandon the belief in the inherent value of the Constitution and instead embrace "radical democracy?" How exactly are you going to get rid of the government based on the Constitution? Through writing Substack posts? Rhetorical questions, of course, but you are welcome to answer them if you wish.
In my view, your belief that you can achieve radical democracy is just as utopian as the TESCREALists thinking they can achieve AGI or the Singularity. Just as they are never going to achieve their utopia, you guys are not either.
But we conservatives have already achieved our utopia. We regard the Constitution and the government and country which has flowed from it, as the best that we’re going to get when it comes to utopia. The Constitution works. There isn’t a need to make fundamental changes to it. The debate between progressives and libertarians that has gone on for a century about how much welfare state to build on top of it will persist indefinitely and that’s fine.
Just as you see TESCREALists as engaged in a kind of secular religion - a critique which I share -- those of us on the Right see you the same way.
And I almonst never like to speak on behalf of the whole Right side of the spectrum as there is so much ideological diversity, but in this discussion I can do so on this point given that the far-left anarchists and communists being very wrong is something that even those of us on the center-right in #NeverTrump NeoCon Bulwark space can agree with the nuts on the far-right about. Heck, this is something that even gay libertarians and the Christian Nationalist bigots can agree on. And even on the other side of the spectrum, center-left liberals and many Democrat-partisan progressives will agree on it that radical democracy goes way too far.
This is a very longstanding ex-Marxist critique of the Left and one that I embraced 18 years ago. You’ve surely heard the expression “the god that failed?” Those who reject Biblical religion, always need something else to take its place. We have the God-shaped hole in our soul and if one does not fill it with an authentic spiritual tradition, then we will make the mistake of filling it with an inauthentic one.
I’m not trying to convert you to anything and you’re certainly welcome to continue holding your far-left views, but I hope that you’re finding one spiritual tradition or another or a healthy blend that can nourish you too, whether that’s East, West or whatever. The pursuit of radical democracy and left-wing politics will never be enough. Left or Right ideology is not the answer people are needing. Mystical practices which provide union with God are.
OK, I’ll stop overwriting now. You’re smart, you get it.
> We should have a say in how advanced AI is developed, and whether it’s even developed in the first place.
Given that both AI 2027 and AI 2040 warn about the risk of human extinction, it’s safe to say the authors are not necessarily in favor of AI being developed (even if AI 2040 also sees very good futures as an option).
AI 2040 also posits that the question of how to develop AI will be the main issue of the 2028 US presidential election. So it takes for granted that US voters, at least, will have a say.
The authors are not exactly writing from a position of political strength. Yes, they have some influence over AI companies, but not enough of it to get those companies to slow down. And when it comes to government they have almost no influence. It’s possible their influence might increase in the future if their factual predictions come true and they’re seen as prophetic. But for now they’re just taking a position.
> In their preferred future, Plan A, there is literally no consideration of climate change.
Plan A does mention that “The robot economy is shifting the bulk of activity to space, so that Earth’s environment and historic spaces will be protected.”
> Building a superintelligent machine isn’t going to somehow fix the problem before 2040.
No, but nothing is likely to fix the problem before 2040. Right now, most of the barriers to adopting clean energy are economic. An economic surplus would massively help. Thus, *if* superintelligent machines actually can be created and can have the absurd-seeming levels of economic benefit envisioned by AI 2040 (“even the poorest non-Americans get about $1M” inflation-adjusted per year), then the climate costs of operating them would likely be outweighed by the climate benefits on economic grounds alone. To whatever (less predictable) extent the machines accelerate development of technologies to fight climate change, that would also help.
If, on the other hand, AI doesn’t actually have such transformative potential, then yes, they represent significant climate cost for minimal climate benefit. But the authors are very much committing to a transformative scenario. If you don’t believe it then you don’t believe it, but there’s nothing really special about climate change here.
Quick response: I think the authors are very much pro-ASI, as long as the ASI is "value-aligned" (meaning "aligned with their TESCREAL worldview"). As Bostrom says, all paths to a utopian future go through ASI. My strong suspicion is that Kokotajlo endorses that. Also, it's worth disambiguating "human extinction" -- although Kokotajlo isn't very clear about the details of his preferred future in interviews, having known him in the past my guess is that he opposes final extinction (no posthumans) rather than terminal extinction (the disappearance of our species).
Re: climate change, despite the line you quote, AI 2040 doesn't take seriously the fact that by 2040 the world will be utterly torn apart by climate change. Hundreds of millions (maybe billions?) will have migrated or died (just think about the ramifications of that!). Entire ecosystems will have collapsed -- according to a Nature (I think it was -- or Science) paper from 2012 coauthored by some of the most renowned environmental scientists, there's a nontrivial chance of a sudden, catastrophic, irreversible collapse of the global ecosystem. This is just one of many possible tipping points that could radically alter, with devastating consequences, entire Earth systems. It will be a bloodbath, which is why I'm shocked the authors completely ignore it!!
Do you think the main barriers are economic, or political?
Thanks for these comments -- I really appreciate you sharing your thoughts and critiques! Thanks for reading. :-)
hmmm... sounds like you're drinking the kool-aid ...
All these smug, smart geeks are out of their goddamned minds.
Imagine if they got their heads out of each other's asses and looked up.
One has to imagine that some of these technologists have met the owners of their corporations? Surely they have bumped into the occasional VC in Palo Alto or San Francisco. Maybe when they came to their offices to pressurise management to make cuts to personnel or to outsource support to the Philippines. I certainly met a few when I worked in Silicon Valley; they forced me to outsource my whole team.
Most VC's are psychopaths they care only about power and money. For the VCs creative destruction is just destruction. Their private planes and their bunkers are ready, their pilots primed...
They intend to live a few months more like moles drinking champagne somewhere below New Zealand. Their libertarian personal freedoms finally guaranteed as they toast in crystal:
"I told you so!"
"Clink clink gentlemen."
"Think of all that value we have created."
"Shame about the prole population but there were too many of them anyway and they weren't so white, were they?."
Do the writers really believe that extra productivity from AI will be compatible with keeping millions of climate refugees safe, compatible with protecting the climate itself. How it this production going to be driven energetically? The mining, the fossil fuels, the waste, the water once used for irrigation now cooling data centres. What shall be the noxious side effects of this vast new production? Who will recycle the robots?
If the middle classes will be able to have exotic vacations, are we supposed to believe the planes that take them there be powered by green hydrogen? Get real! What about the wars that will break out everywhere when neighbouring populations squabble with machine guns over rivers in deserts ... Multiple rivers to multiple seas!
Get with the programme people. If we don't deal with climate chaos now there wont even be electricity in many cities by 2040.