A hallucinating abstraction machine

This is a post I started over a year ago but never got around to publishing. It began about six months after I finished reading (or I should say consuming) Hofstadter’s I Am a Strange Loop, once I had started to reflect on and digest its core ideas.

I first came across Hofstadter in David Deutsch’s The Beginning of Infinity, where, in one of the first chapters, Deutsch mentions Hofstadter’s dominoes, which I thought was just a great thought experiment. I did not even attempt to read Gödel, Escher, Bach (although I intend to in the near future), after I was told that Strange Loop was perhaps a better starting point to get into Hofstadter’s mind.

At the time, I had spent a lot of time thinking about consciousness and about what it means to be an “I”, especially in a world of AI. I wanted a viewpoint that didn’t treat humans as a special entity with “feelium”, that didn’t resort to humans having “that special something”, and that didn’t take the Chalmers path of a consciousness field (whatever that is). I didn’t want the dualistic argument, I was looking for a reductionist point of view: someone willing to acknowledge that we are just a clever pattern made from atoms, nothing less, nothing more.

The core idea of the book is that the mind is a pattern of information processing, so a detailed enough computer program could in principle be conscious. What matters is the pattern, not whether it runs on neurons or silicon.

Not only do I love his writing style, but the ideas were so fresh that it was really hard to put the book down. It is one of the only books that actually tries to address the “hard problem” of consciousness, and it does this by removing the notion of the bird in the birdcage, or the pilot in the cockpit. There is simply no bird, no pilot, no commander, or whatever you like to think of yourself as. It is just a hallucination manifesting as such.

You could argue that he has explained consciousness away, just as Dennett tries to, and that there really is no Cartesian Theatre. Many believe this is a cheat (it is a common criticism of Dennett’s book), but I find Hofstadter’s argument, through the idea of a strange loop, much more compelling and a very refreshing perspective. In fact, I find this viewpoint quite comforting.

Key takeaways

Before digging into the harder questions, here are the key points I learnt:

• Consciousness is a scalar quantity, not a binary one, and everyone has a different level of it. Maybe some of your colleagues have less of it than you think.
• The self is a hallucination. It emerges from a system trying to deal with complexity by forming abstractions, and one of those abstractions is the self.
• Self-reference is the root of it all. The sense of “I” arises from the brain’s ability to represent itself. Like Gödel’s incompleteness theorem and Escher’s Drawing Hands, thoughts can refer to thoughts, and symbols can point back to the system creating them.
• Consciousness is not special to humans. Because it is a pattern, it could in principle exist elsewhere, even in (very large) numbers. I believe Penrose and Hofstadter once discussed this over lunch, and Hofstadter conceded that this could very well be the case, much to Penrose’s surprise.

Another Great Demotion

If this is true, why do we resist it so much? Is it really that hard to let go of the bird in the birdcage? Are we (if I may use the term “we”, or “I” for that matter) going to be so arrogant as to jump to yet another centric view of things?

History gives a clue. For a very long time we thought we were the centre of the universe and that the Sun orbited our planet. How wrong was that? Yet we held on to that belief for quite a while, and it took a lot of convincing otherwise. Thankfully, we no longer sacrifice innocent people in the hope that the Sun will rise in the morning. We no longer believe the Earth is flat (I hope no flat-earthers read my blog). And, thanks to Darwin, we no longer believe that humans are the centre of life.

Carl Sagan summarised this well with the idea of the Great Demotions: the many scientific discoveries that widened the distance between humans and their special status as a “divine creation”.

Time and time again we have been mistaken, mostly because we put too much trust in our perceptions and our senses, which we know can easily be fooled. This is an argument from history, not proof. But I believe it is only a matter of time before the hallucination of the self joins that list as yet another Great Demotion.

The hard cases

The best chapters, in my opinion, come in the second part of the book, so it is worth reading to the end, although the domino computer is a nice warm-up act. The thought experiments he guides the reader through are great, although I am still not fully sure they resolve the important ones, in particular the teleporter and the copy of oneself.

Being someone else

I can relate to the idea of being someone else via VR goggles, with all my senses masked by another person’s. I can also relate to the idea of being a different person when I am in a different country, or even in a different room. But I still can’t fully comprehend how I feel like me. Hofstadter’s answer is that this feeling is the loop perceiving itself; there is nothing extra to explain. I am not yet sure that answer is enough.

Splitting in time

The teleporter question matters: it has “me” existing in two places at once (or in none?). But I think it is easier to think in terms of splitting in time, which is something we do all the time. Clearly I am not the same person I was ten years ago, nor the same person I was a week ago. But what about a day ago, a minute ago, a second ago? Here I fall back to Parfit, for whom “which one is me?” is a confused question: identity is not what matters, psychological continuity is. If we take the pattern (state vector of all atoms) to be what matters, then both copies have an equal claim to being you.

Swapping memories

Here is a thought experiment of my own. Imagine two people, Alice and Bob, who can exchange their memories perfectly. Every memory, every learnt habit and every association moves from one brain to the other. Then they both wake up.

The body that used to be Alice’s now remembers Bob’s childhood, Bob’s job and Bob’s name. It looks in the mirror and sees the wrong face. Each of them would insist that they had changed bodies overnight, and their friends would soon agree, because “Alice” now talks and acts like Bob. From the inside, it would feel exactly like being teleported into someone else’s body.

Yet nothing physical moved. The atoms stayed where they were, and so did the neurons. Only the pattern was rewritten. If the “I” seems to move anyway, then the “I” follows the pattern, not the hardware. This is the teleporter intuition without the copying: there is no second instance, so there is no “which one is me?” problem. (John Locke made a similar point centuries ago with his prince and cobbler, where memory decides who you are.)

Me as a state vector

Let’s entertain the idea that we are just a big (and I mean huge) vector of states, containing the state of every atom of a person, or maybe just the brain.

A problem appears straight away. We cannot know the exact position of an atom without its momentum becoming completely uncertain, and the same trade-off holds for energy and time. So how do we record every atomic state? And relative to what? Position and time are not absolute. But if the pattern is what matters, we may not need every atom. A copy at the level of neurons and synapses may be enough.

For the sake of the thought experiment, let’s say that we can define the configuration of a person’s consciousness at time $t$ as $|q\rangle$. (I borrow quantum notation here only as an analogy.) This state vector alone defines a person: a snapshot of the “person program” that could be instantiated again at a later time $t'$. Everyone has a different state vector, of course; otherwise we would all be the same.

The program then operates on this vector to produce a new one, giving a new state of consciousness:

$O\,|q\rangle = |q'\rangle$

The operator $O$ is the software of consciousness, moving one state to the next.

Now back to the teleporter. Do we have the same $|q\rangle$ in two places at once? Assume we have copied the whole state exactly, with no errors throughout the vector. We should then have two instances of the person. But immediately afterwards, different environments (one is now on Mars, the other still on the spaceship) give different inputs to each copy, and the consciousness forks into two. Which one is “I”?

There is a twist here. If $|q\rangle$ really were a quantum state, the no-cloning theorem says it cannot be copied exactly at all. But the pattern view suggests that a person is not a quantum state, so this limit may not apply.

Can I live in a computer?

If a person really is just a state $|q\rangle$ and an operator $O$, then a person is software. That raises the immortal question: can my mind be separated from my body and run on different hardware?

I am not convinced either way, and the idea both scares me and makes me optimistic that life “for me” will not end. It seems only a matter of time before we figure out what consciousness is, and when we do, we will be able to replicate it. The question is: will I be back again? Or will it be another instantiation of me, thinking it is me, but not the stream of consciousness I am experiencing now as I write these words?

There are two camps:

• Yes: David Deutsch. Because of computational universality, people are software running in meat bags. Once we understand the software, we can run it on any hardware. I am not sure whether Deutsch claims the same I is transferred, or a new instance that believes it is the person. But to an outside observer, there is no way to tell.
• No: Geoffrey Hinton. We are products of our own unique architecture. The knowledge is tied to the specific analogue hardware (the particular neurons and voltages inside each individual brain) and cannot be copied to a different design.

I side more with Deutsch, and find Hinton’s reasoning flawed. What matters is the digital internals: analogue computers struggle to scale because they cannot easily correct their errors, and error correction is much easier with discrete, digital states. Neurons fire in all-or-nothing spikes, which suggests that something close to digital does much of the real work, and that could be copied.

Questions that remain

The book left me with more questions than answers, which is probably a good sign.

• Where does a thought or a feeling live in the brain? If it can be located, then where is it? If not, why not? And can something that cannot be physically located even exist? If the mind is a pattern, perhaps a thought is no more “located” than a melody is located in one note.
• How many “I”s are there? If “I”-ness is all-or-nothing, the only sensible numbers seem to be 0, 1 or infinite. But if it is a scalar, there could also be partial “I”s. Hofstadter argues that we each carry low-resolution copies of the people we love. If it is an illusion, the answer is 0. Easy. If not, it is 1 or infinite. We believe it to be 1, but then we must ask “why this one?“. If a soul is real, infinite makes more sense than 1. This is the teleporter problem again, in another form.
• Are we in a simulation? To me this is a non-starter and an infinite regress, no different from the supernatural and believing in gods. It cannot be ruled out, but entertaining it doesn’t do much. It is interesting, though, that our physical world has certain roadblocks that could be interpreted as limits on a simulation to keep us locked in: the speed of light, or Heisenberg’s uncertainty principle.
• Is there a universal program for consciousness? This follows from Deutsch’s universality argument. Can this software be explained mathematically, or otherwise formalised, without building bigger and bigger neural nets and hoping things emerge from lots of data? If true, the implications are far-reaching. Did nature find this program to help humans survive, only for it to be obfuscated in the meat bags in our heads, waiting to be unlocked? Gödel’s incompleteness does not stop this. If anything, Hofstadter uses Gödel’s proof as the model for the strange loop itself.

And what about AI?

With all the recent hype around AI and the amazing power of the latest large language models (there’s no denying they are a very clever and useful tool), this nearly 20-year-old book is more relevant than ever.

If consciousness is a scalar and not a switch, then “is AI conscious?” is the wrong question. The better question is: how much “I” is there in an AI, and is there a loop that refers back to itself?

Hofstadter has since addressed this directly in “Is there an ‘I’ in AI?”. I fully recommend reading it.

Written by Thomas Stainer who likes to develop software for applications mainly in maths and physics, but also to solve everyday problems. Check out my GitHub page here.