As you point out several times, Émile, their arguments are so often circular, and so obviously circular - transparent rationalisations of what they already want to do. It's childish, yet their power is very real.
Hi. I have a basic question re: post humans and Tessreal thinking. It is: whose “minds” will be uploaded and copied into the cloud and sent into the cosmos (and how will they be promulgated) in Tescreal writing? Will it be the messy result of all of us current humans (via romance and other forms of current reproduction. etc) or just three trillion copies of Dario Amodei, Sam Altman and Elon Musk? My hunch is the latter but wanted to check.
Quite frankly (as you know) I personally do not spend much time thinking about the dangers of ASI (as-a-real-entity) insofar I don't believe that we are anywhere NEAR such a capability and take any news of speeding up towards RSI with not only a grain of salt but rather a truckload thereof.
HOWEVER, if I was willing to grant that there was actually such a thing as a hyper-intelligent selfimproving system that rapidly tumbles towards a kind of omniscience (And I am not. For a number of reasons, only some of which have to do with the current techstack and some of which have to do with what I believe is Günther's adequate assessment of ontology in the age of cybernetics, but I digress)...
in any case, IF I were to believe that, then I would still not be particularly concerned. This has something to do with the fact that I not-so-secretly have the belief that knowledge is not as neutral as the instrumentalist view makes it seem. We are raised in a society that allows us (at least in principle) to see people that are chasing after absolutely self-defeating goals as smart or worse, business savvy. I think that says more about the absolute rottenness of our understanding of epistemology than about the discipline as such.
I think there is something to be said toward the old forms of wisdom and their necessary connection to knowledge... all of this just to say, I believe if you were really in a situation in which you would create such an apparatus, they would be far beyond interested in harming us and yet not so far beyond that they don't think about it.
This is neither a lovecraftian elder god nor a Terminator. This would be a machine that actually tries to work with the world. And I think the very structure of knowledge is biased towards making the world whole. "Totum relucet in omnibus", the Cusanus word, would need to apply for this machine after all anyhow: everything would need to be understood through everything, which would also include the different points of view, which would entail a sort of bias towards protection.
This is an almost theological position, sure.
But if we are going to go into the realm of wildspec you might as well go there.
Because the world is actually a secretly very friendly beautiful place with such nice surprises. Sometimes.
As you point out several times, Émile, their arguments are so often circular, and so obviously circular - transparent rationalisations of what they already want to do. It's childish, yet their power is very real.
Hi. I have a basic question re: post humans and Tessreal thinking. It is: whose “minds” will be uploaded and copied into the cloud and sent into the cosmos (and how will they be promulgated) in Tescreal writing? Will it be the messy result of all of us current humans (via romance and other forms of current reproduction. etc) or just three trillion copies of Dario Amodei, Sam Altman and Elon Musk? My hunch is the latter but wanted to check.
Quite frankly (as you know) I personally do not spend much time thinking about the dangers of ASI (as-a-real-entity) insofar I don't believe that we are anywhere NEAR such a capability and take any news of speeding up towards RSI with not only a grain of salt but rather a truckload thereof.
HOWEVER, if I was willing to grant that there was actually such a thing as a hyper-intelligent selfimproving system that rapidly tumbles towards a kind of omniscience (And I am not. For a number of reasons, only some of which have to do with the current techstack and some of which have to do with what I believe is Günther's adequate assessment of ontology in the age of cybernetics, but I digress)...
in any case, IF I were to believe that, then I would still not be particularly concerned. This has something to do with the fact that I not-so-secretly have the belief that knowledge is not as neutral as the instrumentalist view makes it seem. We are raised in a society that allows us (at least in principle) to see people that are chasing after absolutely self-defeating goals as smart or worse, business savvy. I think that says more about the absolute rottenness of our understanding of epistemology than about the discipline as such.
I think there is something to be said toward the old forms of wisdom and their necessary connection to knowledge... all of this just to say, I believe if you were really in a situation in which you would create such an apparatus, they would be far beyond interested in harming us and yet not so far beyond that they don't think about it.
This is neither a lovecraftian elder god nor a Terminator. This would be a machine that actually tries to work with the world. And I think the very structure of knowledge is biased towards making the world whole. "Totum relucet in omnibus", the Cusanus word, would need to apply for this machine after all anyhow: everything would need to be understood through everything, which would also include the different points of view, which would entail a sort of bias towards protection.
This is an almost theological position, sure.
But if we are going to go into the realm of wildspec you might as well go there.
Because the world is actually a secretly very friendly beautiful place with such nice surprises. Sometimes.
Emile, what do you think of Herbert Roitblat's recent post that RSI and by extension AGI are Silicon Valley's Skyhook and cannot be achieved?
His post is here: https://herbertroitblat859691.substack.com/p/recursive-self-improvement-the-ai