The shaky assumption that a mind or “self” can be “uploaded” to a computer is predicated on the even shakier presumption that matter gives rise to consciousness. There’s no real evidence for that, with even physicists like Max Planck famously asserting that consciousness is primary to matter (given the phenomenological and epistemological problems raised by quantum mechanics). I’m doubtful that the major current figures in the tech sphere have both the patience and wisdom to really understand this. “ The Blind Spot” by Frank, Gleiser, and Thompson is a great starting point for a quick primer on this millennium old problem.
I see I'm not the only one who got hung up on this specific part of your last article. :D You zero in on precisely the paragraph that made me emit some incoherent grumbles. Currently trying to rephrase those into human language. The long-and-short of it is that we're not going to get that test EVER for conceptual reasons. But I'll try to just finish writing that thing, that's hopefully more convincing then me just positing that.
Just got finished. There are a few loose threads, but I feel that they are sufficiently implied. A few times I get to refer to Günther's 3-value-logic but never mention it once, that feels at once disingenous and yet making it a lot easier to read. Let me know what you think! :)
I’ve never understood the hype surrounding mind uploading.
The followers of this transhumanist nonsense just don’t get the difference between copying and transferring.
Even if mind uploading works, even if the machine with the “mind” then has consciousness. Who benefits from it? Okay, the person who sold it. Certainly not the one who was uploaded—they’re just dead.
In the part on consciousness you write: "It’s rather shocking to me that TESCREALists don’t seem to have realized this. Almost no one in the movement has raised these issues"
Actually, it's not all that surprising to me ... I think many have realized it, but seeing they don't have an answer they don't raise the issue in the hope that no-one else does either :-D
On the debate video, and Yudkowsky's choice of costume: it looked to me like Yudkowsky didn't take the anonymous debator seriously for a few reasons, and he telegraphed that pretty loudly. As it happens, it looks like that stance coming in wasn't wrong: "47f" didn't seem to be there for the stated purpose of having a debate at all.
If you watched longer 16:51 and it improved, let me know. It didn't seem like a rewarding use of time to persist.
No, it doesn't get much better! There are a few interesting exchanges toward the end, but not worth watching the rest of it to get there, imo.
I agree about Yudkowsky not taking it seriously. I just don't understand why his _expression_ of not taking it seriously took the form of dressing up like a clown. Why on Earth did he think that was a good idea? (Hashtag: practical intelligence!)
To be certain that new digital beings are conscious as we understand and feel that notion it seems necessary to fully recreate our biological substrate in those digital beings, which would then render them useless for the heavy space travel the cosmic colonization effort will demand. A Catch-22 scenario, no?
To borrow some of the amateur Bayesianism that this people like to throw out: the scenario where inert rocks pretending to be conscious populate the universe is not only bad, it's much, much worse than the extinction of humanity itself (The doom in "p(doom)" these freaks are so scared of). Zombie colonization would monopolize all the possible resources in the universe, making it impossible for genuine consciousness to flourish.
Call it superdoom. Logically, we should be trying hard to reduce p(superdoom), and because p(superdoom) would be so much worse than p(doom), we should be trying to reduce it much harder than p(doom)! And yet, these tech companies headed by TESCREALISTs consistently attempt to improve A.I models, despite more evidence every day that it's extremely easy to fool average humans into thinking chatbots are conscious. They're cranking up p(superdoom) by the day!
I think this illogical is very similar a) if I die and upload my mind into a machine, I’ll live on, and b) as long as our digital, machine, or stone “successors” have consciousness, everything’s fine, and humanity will live on through them.
That’s completely crazy stuff and has similarities.
Yeah. To me, the idea that an unconscious machine could imitate me very well and lead on friends and loved ones for years after my death is much more existentially horrifying than the idea of dying, and that's a microcosm of the whole p zombie situation
I mean, even if we adopt a materialist and physicalist view of consciousness, a static “hardware” cannot support consciousness.
Our neurons, our cells, they die. They change. Some every minute.
If our cells are being compared to a computer hardware that does not change, then its structure and functions are different. Biologically very little is similar with that of a computer.
It's trivially obvious in a common-sensical way that cut/pasting your "mind" (so obviously an impossible feat, but nvm that) isn't the same as self-uploading, yet plenty of otherwise intelligent people struggle with this logical reasoning. It triggers my exasperation with humankind.
And no, the hard problem of consciousness has no solution - again, for logical reasons.
These TESCREALists always talk about "uploading their minds" to the great computer in the sky, but they never seem to explain how exactly that would happen, where this uploaded "you" would be, and a million other details that are conveniently ignored. It's a forever moving goalpost by design.
I mean, it'd be like… I guess, having a transporter accident and suddenly there being two copies of you, right? Both of you are conscious, both of you are alive, both of you have claim to be you… at least for a _really_ short time while both of you are still, fundamentally, the same.
But you'd diverge based on your experiences. One could have an illustrious career while another is marooned in a distant place, only to be rescued years later by the one that had an illustrious career. By that time the both of you would be different selves, different identities, and thus different people (guess the reference!)
Oh, and you were in the same office with Anders Sandberg? Heh. I'm surprised he never mentioned his participation in the giant TESCREAL shared universe thing he did in the early 2000s. His HPMOR moment, as it were, lmao. Well, at least his entries were short.
It's becoming clear that with all the brain and consciousness theories out there, the proof will be in the pudding. By this I mean, can any particular theory be used to create a human adult level conscious machine. My bet is on the late Gerald Edelman's Extended Theory of Neuronal Group Selection. The lead group in robotics based on this theory is the Neurorobotics Lab at UC at Irvine. Dr. Edelman distinguished between primary consciousness, which came first in evolution, and that humans share with other conscious animals, and higher order consciousness, which came to only humans with the acquisition of language. A machine with only primary consciousness will probably have to come first.
What I find special about the TNGS is the Darwin series of automata created at the Neurosciences Institute by Dr. Edelman and his colleagues in the 1990's and 2000's. These machines perform in the real world, not in a restricted simulated world, and display convincing physical behavior indicative of higher psychological functions necessary for consciousness, such as perceptual categorization, memory, and learning. They are based on realistic models of the parts of the biological brain that the theory claims subserve these functions. The extended TNGS allows for the emergence of consciousness based only on further evolutionary development of the brain areas responsible for these functions, in a parsimonious way. No other research I've encountered is anywhere near as convincing.
I post because on almost every video and article about the brain and consciousness that I encounter, the attitude seems to be that we still know next to nothing about how the brain and consciousness work; that there's lots of data but no unifying theory. I believe the extended TNGS is that theory. My motivation is to keep that theory in front of the public. And obviously, I consider it the route to a truly conscious machine, primary and higher-order.
My advice to people who want to create a conscious machine is to seriously ground themselves in the extended TNGS and the Darwin automata first, and proceed from there, by applying to Jeff Krichmar's lab at UC Irvine, possibly. Dr. Edelman's roadmap to a conscious machine is at https://arxiv.org/abs/2105.10461, and here is a video of Jeff Krichmar talking about some of the Darwin automata, https://www.youtube.com/watch?v=J7Uh9phc1Ow
TBF for the purposes of the TESCREALists the mind-self distinction is probably pretty academic, as long as the upload is conscious and fully functional (which are a whole different can of worms).
But it'd sure make for a LOT of practical legal problems to have both the original organic "template" and its digital copies running around simultaneously...
On the question of consciousness: I’ve been reading what seems like a metric fuck-tonne of material about consciousness lately, and I’m coming around to the position that we are all p-zombies relative to the thing people imagine the p-zombie is missing. The “consciousness” people are trying to protect may be a folk-metaphysical artifact.
What seems to be breaking people’s intuitions about uploads, downloads, multiple instantiations of a mind, and so on is the fact that memory, embodiment, and identity have formed something close to a circle for humans across our history. For ordinary human life in 2026, those three domains overlap so tightly that people casually treat them as one thing. The moment technology distorts that Venn diagram, things get weird fast.
I know from experience that getting into the weeds about how perception and memory actually work can destabilize people’s sense of self. But if we’re going to make claims about consciousness, and if we’re going to draw a line between embodied humans and anything else, then we have to accept that the reality of our situation differs sharply from the illusion of a coherent, singular Self that we carry around as our day-to-day operating model.
We are an electrified chunk of meat trapped in the utter dark and quiet of our skulls. We construct everything: the world, our place in it, ourselves, our histories, all of it. We update that construction using inputs from a passel of seriously sub-par peripheral senses. Worse, we update it less from direct inputs than from processed results of those inputs, after they have already been collated by neurological processes of which we are almost entirely unaware, absent drugs, injury, or unusual training.
Our perceptions are faulty. Whatever we perceive and retain in memory is faulty. The assemblage is faulty. Recollection is not the surfacing of a wholly intact sequence. It is the reconstitution of a faulty assemblage into something new in the moment, perceived inexpertly in that moment, then stashed away anew into its disparate parts.
If there is “a thing it is like” to be us, it is not the stable, cogent illusion we imagine ourselves passing forward moment to moment, like a baton in a well-synchronized relay race.
It is more like a madding crowd of savants caught in a wind tunnel and convinced they’re in a library.
The idea that the brain (the electrified chunk of meat) sits inside the dark skull and has to figure out what happens outside is dealt a serious blow by the fact that the brain itself is a cognitive construct. Mental processes give rise to various appearances, such as tables, people, … and brains. It’s well worth letting that really sink in. It helps foster an epistemic humility that does little to advance one’s career or status but is immeasurably useful for both awe and, ultimately, peace of mind.
The shaky assumption that a mind or “self” can be “uploaded” to a computer is predicated on the even shakier presumption that matter gives rise to consciousness. There’s no real evidence for that, with even physicists like Max Planck famously asserting that consciousness is primary to matter (given the phenomenological and epistemological problems raised by quantum mechanics). I’m doubtful that the major current figures in the tech sphere have both the patience and wisdom to really understand this. “ The Blind Spot” by Frank, Gleiser, and Thompson is a great starting point for a quick primer on this millennium old problem.
I see I'm not the only one who got hung up on this specific part of your last article. :D You zero in on precisely the paragraph that made me emit some incoherent grumbles. Currently trying to rephrase those into human language. The long-and-short of it is that we're not going to get that test EVER for conceptual reasons. But I'll try to just finish writing that thing, that's hopefully more convincing then me just positing that.
Very curious to know your thoughts on this!!
https://gedsperber.substack.com/p/what-it-is-like-to-be-an-llm?r=51xiwq&utm_campaign=post&utm_medium=web&showWelcomeOnShare=true
Just got finished. There are a few loose threads, but I feel that they are sufficiently implied. A few times I get to refer to Günther's 3-value-logic but never mention it once, that feels at once disingenous and yet making it a lot easier to read. Let me know what you think! :)
I’ve never understood the hype surrounding mind uploading.
The followers of this transhumanist nonsense just don’t get the difference between copying and transferring.
Even if mind uploading works, even if the machine with the “mind” then has consciousness. Who benefits from it? Okay, the person who sold it. Certainly not the one who was uploaded—they’re just dead.
Even my son as he was 12 years old got it.
In the part on consciousness you write: "It’s rather shocking to me that TESCREALists don’t seem to have realized this. Almost no one in the movement has raised these issues"
Actually, it's not all that surprising to me ... I think many have realized it, but seeing they don't have an answer they don't raise the issue in the hope that no-one else does either :-D
On the debate video, and Yudkowsky's choice of costume: it looked to me like Yudkowsky didn't take the anonymous debator seriously for a few reasons, and he telegraphed that pretty loudly. As it happens, it looks like that stance coming in wasn't wrong: "47f" didn't seem to be there for the stated purpose of having a debate at all.
If you watched longer 16:51 and it improved, let me know. It didn't seem like a rewarding use of time to persist.
No, it doesn't get much better! There are a few interesting exchanges toward the end, but not worth watching the rest of it to get there, imo.
I agree about Yudkowsky not taking it seriously. I just don't understand why his _expression_ of not taking it seriously took the form of dressing up like a clown. Why on Earth did he think that was a good idea? (Hashtag: practical intelligence!)
To be certain that new digital beings are conscious as we understand and feel that notion it seems necessary to fully recreate our biological substrate in those digital beings, which would then render them useless for the heavy space travel the cosmic colonization effort will demand. A Catch-22 scenario, no?
To borrow some of the amateur Bayesianism that this people like to throw out: the scenario where inert rocks pretending to be conscious populate the universe is not only bad, it's much, much worse than the extinction of humanity itself (The doom in "p(doom)" these freaks are so scared of). Zombie colonization would monopolize all the possible resources in the universe, making it impossible for genuine consciousness to flourish.
Call it superdoom. Logically, we should be trying hard to reduce p(superdoom), and because p(superdoom) would be so much worse than p(doom), we should be trying to reduce it much harder than p(doom)! And yet, these tech companies headed by TESCREALISTs consistently attempt to improve A.I models, despite more evidence every day that it's extremely easy to fool average humans into thinking chatbots are conscious. They're cranking up p(superdoom) by the day!
I think this illogical is very similar a) if I die and upload my mind into a machine, I’ll live on, and b) as long as our digital, machine, or stone “successors” have consciousness, everything’s fine, and humanity will live on through them.
That’s completely crazy stuff and has similarities.
Yeah. To me, the idea that an unconscious machine could imitate me very well and lead on friends and loved ones for years after my death is much more existentially horrifying than the idea of dying, and that's a microcosm of the whole p zombie situation
I mean, even if we adopt a materialist and physicalist view of consciousness, a static “hardware” cannot support consciousness.
Our neurons, our cells, they die. They change. Some every minute.
If our cells are being compared to a computer hardware that does not change, then its structure and functions are different. Biologically very little is similar with that of a computer.
It's trivially obvious in a common-sensical way that cut/pasting your "mind" (so obviously an impossible feat, but nvm that) isn't the same as self-uploading, yet plenty of otherwise intelligent people struggle with this logical reasoning. It triggers my exasperation with humankind.
And no, the hard problem of consciousness has no solution - again, for logical reasons.
And people would think *I'm* the crazy one for believing in God...
These TESCREALists always talk about "uploading their minds" to the great computer in the sky, but they never seem to explain how exactly that would happen, where this uploaded "you" would be, and a million other details that are conveniently ignored. It's a forever moving goalpost by design.
I mean, it'd be like… I guess, having a transporter accident and suddenly there being two copies of you, right? Both of you are conscious, both of you are alive, both of you have claim to be you… at least for a _really_ short time while both of you are still, fundamentally, the same.
But you'd diverge based on your experiences. One could have an illustrious career while another is marooned in a distant place, only to be rescued years later by the one that had an illustrious career. By that time the both of you would be different selves, different identities, and thus different people (guess the reference!)
Oh, and you were in the same office with Anders Sandberg? Heh. I'm surprised he never mentioned his participation in the giant TESCREAL shared universe thing he did in the early 2000s. His HPMOR moment, as it were, lmao. Well, at least his entries were short.
Well it’s got you talking about him, mission accomplished.
It's becoming clear that with all the brain and consciousness theories out there, the proof will be in the pudding. By this I mean, can any particular theory be used to create a human adult level conscious machine. My bet is on the late Gerald Edelman's Extended Theory of Neuronal Group Selection. The lead group in robotics based on this theory is the Neurorobotics Lab at UC at Irvine. Dr. Edelman distinguished between primary consciousness, which came first in evolution, and that humans share with other conscious animals, and higher order consciousness, which came to only humans with the acquisition of language. A machine with only primary consciousness will probably have to come first.
What I find special about the TNGS is the Darwin series of automata created at the Neurosciences Institute by Dr. Edelman and his colleagues in the 1990's and 2000's. These machines perform in the real world, not in a restricted simulated world, and display convincing physical behavior indicative of higher psychological functions necessary for consciousness, such as perceptual categorization, memory, and learning. They are based on realistic models of the parts of the biological brain that the theory claims subserve these functions. The extended TNGS allows for the emergence of consciousness based only on further evolutionary development of the brain areas responsible for these functions, in a parsimonious way. No other research I've encountered is anywhere near as convincing.
I post because on almost every video and article about the brain and consciousness that I encounter, the attitude seems to be that we still know next to nothing about how the brain and consciousness work; that there's lots of data but no unifying theory. I believe the extended TNGS is that theory. My motivation is to keep that theory in front of the public. And obviously, I consider it the route to a truly conscious machine, primary and higher-order.
My advice to people who want to create a conscious machine is to seriously ground themselves in the extended TNGS and the Darwin automata first, and proceed from there, by applying to Jeff Krichmar's lab at UC Irvine, possibly. Dr. Edelman's roadmap to a conscious machine is at https://arxiv.org/abs/2105.10461, and here is a video of Jeff Krichmar talking about some of the Darwin automata, https://www.youtube.com/watch?v=J7Uh9phc1Ow
TBF for the purposes of the TESCREALists the mind-self distinction is probably pretty academic, as long as the upload is conscious and fully functional (which are a whole different can of worms).
But it'd sure make for a LOT of practical legal problems to have both the original organic "template" and its digital copies running around simultaneously...
On the question of consciousness: I’ve been reading what seems like a metric fuck-tonne of material about consciousness lately, and I’m coming around to the position that we are all p-zombies relative to the thing people imagine the p-zombie is missing. The “consciousness” people are trying to protect may be a folk-metaphysical artifact.
What seems to be breaking people’s intuitions about uploads, downloads, multiple instantiations of a mind, and so on is the fact that memory, embodiment, and identity have formed something close to a circle for humans across our history. For ordinary human life in 2026, those three domains overlap so tightly that people casually treat them as one thing. The moment technology distorts that Venn diagram, things get weird fast.
I know from experience that getting into the weeds about how perception and memory actually work can destabilize people’s sense of self. But if we’re going to make claims about consciousness, and if we’re going to draw a line between embodied humans and anything else, then we have to accept that the reality of our situation differs sharply from the illusion of a coherent, singular Self that we carry around as our day-to-day operating model.
We are an electrified chunk of meat trapped in the utter dark and quiet of our skulls. We construct everything: the world, our place in it, ourselves, our histories, all of it. We update that construction using inputs from a passel of seriously sub-par peripheral senses. Worse, we update it less from direct inputs than from processed results of those inputs, after they have already been collated by neurological processes of which we are almost entirely unaware, absent drugs, injury, or unusual training.
Our perceptions are faulty. Whatever we perceive and retain in memory is faulty. The assemblage is faulty. Recollection is not the surfacing of a wholly intact sequence. It is the reconstitution of a faulty assemblage into something new in the moment, perceived inexpertly in that moment, then stashed away anew into its disparate parts.
If there is “a thing it is like” to be us, it is not the stable, cogent illusion we imagine ourselves passing forward moment to moment, like a baton in a well-synchronized relay race.
It is more like a madding crowd of savants caught in a wind tunnel and convinced they’re in a library.
The idea that the brain (the electrified chunk of meat) sits inside the dark skull and has to figure out what happens outside is dealt a serious blow by the fact that the brain itself is a cognitive construct. Mental processes give rise to various appearances, such as tables, people, … and brains. It’s well worth letting that really sink in. It helps foster an epistemic humility that does little to advance one’s career or status but is immeasurably useful for both awe and, ultimately, peace of mind.
If you mean to suggest that the sandcastle of the Self isn't on a beach, but on a cloud, I agree. That only makes my point rather stronger, I think.