What is Consciousness?

My curiosity about consciousness started in the late eighties when I asked myself if it was possible to make a conscious computer.

spacer-2px

federico-faggin

At that time I was working in the field of artificial neural networks at Synaptics, a company I co-founded in 1986, and the study of consciousness was not considered a proper subject of scientific inquiry, since scientists generally believed, and still do, that consciousness is an emergent property of the operation of the brain. Consciousness is largely considered to be a property arising exclusively from the brain that will eventually be completely explained by neuroscience as the knowledge about how the brain works increases. On the other hand, consciousness has been a serious subject of philosophical investigations since the beginning of recorded history, resulting in a large number of different speculations about its nature.

In the early sixties, artificial intelligence (AI) researchers, buoyed by some early successes in programming computers to play chess, and solve a few other problems that appeared to require much brain power, speculated that soon computers would surpass human intelligence. In those days intelligence and consciousness were considered near synonyms, and conscious computers were expected to naturally emerge with intelligent behavior. However, today’s computers are not one bit (pun intended) more conscious than the computers of the early sixties, despite being tens of billions of times more powerful than they were then.

pullquote-todayscomputers

Although the nature of intelligence is now far better understood and appreciated than in the early sixties, the nature of consciousness is still shrouded in mystery. I believe that the familiarity we have with consciousness tends to keep the mystery going because many capacities we take for granted, such as intuition or imagination for example, are underappreciated, and reveal their complexity, power, and mystery particularly when we try to endow our machines with our faculties.

divider-gold

Can a Machine Be Conscious?

Imagine a robot with a sophisticated vision and control system capable of driving a car better than most human beings; a system that by 2030 may be commonplace in our cars. Since we must be quite conscious when we drive a car, it is natural to think that the robot must be conscious as well. But is that true? Absolutely not! First of all, the robot has no visual sensations, no visual experience at all. Inside the robot it’s all “dark,” and if it communicates with neighboring cars to jointly negotiate priorities, for example, the robot is not aware of what it’s doing. It doesn’t understand at all.

emptyhead

Inside the robot’s mechanical brain, a number of algorithms compute the next action to take based on the automatic analysis of sensory and other information captured by its various input systems. Though the specific next action is consistent with the destination goal that has been instructed by the vehicle owner, inside the robot there are no sensations, and no feelings; no experience of color, shapes, textures, objects and relationships between objects; no understanding of what’s going on out there on the road. There is just a complex web of blind action-reaction patterns because there is no-self inside, there is no consciousness.

And yet such robot may well turn out to be better than most human beings at driving a car. Therefore it is natural to ask: “If an unconscious machine can do a better job than humans in a task that requires so much attention and consciousness from us, what is the leverage consciousness gives us?” The power has to be found in what distinguishes a machine from us: Our flexibility and freedom that allow us to handle unpredictable situations based on understanding of the situation, in contrast with the capacity of a machine to follow orders blindly and predictably.

As long as it is possible to arrange the external conditions in such a way that the problem to be solved becomes deterministic, a machine can do a better job than we can without needing to be conscious. However, failing this condition, consciousness is necessary to adequately solve those problems that are not deterministic, and where all the necessary data are not known.

In fact, a critical prerequisite to completely solve the problem of self-driving requires the cars to communicate with each other at high speed in order to negotiate difficult situations in accordance with strict protocols. Only in this manner will the ambiguities in the behavior of the neighboring cars be removed. Without it, it would be impossible for the control system to adequately perform in the difficult driving conditions occurring in the chaotic traffic in the center of big cities like New York or Rome. Therefore, to achieve the improvements in safety and circulation theoretically possible with automatic driving, all the cars must strictly follow the same rules.

It is human freedom that generates the unpredictability that leads to accidents or traffic jams, but that same freedom also leads to finding creative solutions to situations never before experienced.

spacer-2px

Machines are better than humans at doing repetitive work, as long as the environment can also be sufficiently controlled to be as deterministic as the program that directs them. But I am sure that even the best self-driving car will occasionally encounter situations where it must delegate to a human being the task of driving when the external conditions go outside the pre-programmed ones, or something breaks down. That’s when consciousness plays an indispensable role. And we should not forget that it takes many highly conscious programmers and engineers to create all the necessary programs and systems that can make the unconscious system work.

The value of consciousness to a human being is not just to drive a car but to learn to do literally thousands of different and difficult tasks; to correctly perceive the environment, to understand, to know, to be flexible, to be imaginative, to be creative, to dare, to love, to have humor, to have a conscience, to have a sense of self, to wonder, to feel connected with another being or with the universe; to be responsible, to understand the consequences of one’s actions, and ultimately to acquire judgment and wisdom.

Without consciousness we wouldn’t even know that we are alive, that we exist, and life would have no meaning and no purpose. Without consciousness we would simply be machines going through our paces; without feelings, without comprehension, with no sense of self, and no free will; only a long chain of action-reactions. In fact, we couldn’t even say a boring chain of action and reactions, because to feel boredom requires consciousness.

What would be the point of living our life then if we couldn’t feel joy and fulfillment, pain and sorrow, compassion, wonder and love? It is consciousness that allows us to feel. It is consciousness that gives us physical sensations and our emotional, mental, and spiritual dimensions. Without it, we might as well not exist!

pullquote-without

If there is one thing each of us is sure about is the fact of existing, which is the result of a direct inner perception. We know it because we feel it! Consciousness is the capacity to have an inner experience, and to direct one’s experience. A machine has no inner experience; and that is such a fundamental difference, so basic, so incredibly important, and so generally unrecognized, that we do not pay much attention to it, and tend to attribute consciousness to something that behaves like us without realizing that behavior and feelings are not necessarily one the consequence of the other.

divider-gold

Consciousness Defined

Since there isn’t an agreed upon definition of consciousness, I would like to convey my own preliminary portrayal at this stage of my understanding: I see consciousness as the interplay of five fundamental, irreducible, and interdependent aspects, like five facets of a whole. These basic aspects are: perception, comprehension, identity, free-will, and action.

diagram2

• Perception is the capacity to have a sensate or sentient experience, what philosophers call quale (the plural is qualia). Quale is what something feels like; for example, the smell of a rose or the taste of wine as specific sensations or feelings. We experience the world through qualia, but qualia are not patterns of bits in memory, far from it.

• Comprehension is the capacity to have an integrated understanding of our experience within the widest possible context. It is the capacity to capture the meaning of what is experienced.

• Identity is the fundamental property of being identified within itself as itself. It’s the ability to discriminate between self and non-self. Identity provides a unique point of view, a sense of agency, and the fundamental context within which perception, comprehension, action and free-will operate and have significance.

• Free will is the capacity to choose; to decide a specific course of action based on the available comprehension and consistent with the intention and goals of the identity or self. Free will is inextricably connected with the sense-of-self as an autonomous and independent agency.

• Action is the ability to affect the outer environment based on a free-will decision.

We live in a sea of consciousness and yet we are generally unaware of this most precious attribute because it is everywhere within ourselves. Like a deep-sea fish is probably unaware of the water that entirely fills its world, consciousness appears to us as the mental space that contains the objects of our awareness. If we take away all the objects, sensations, and feelings, nothing is left and therefore consciousness appears to be nothing. Yet consciousness is the very fabric that allows us to have experiences; it’s what gives substance to our experience. The very object we perceive out there, say, a tree, is actually shaped, textured, colored, placed in relationships with other objects and finally named in the mental space inside our head.

Out there, there is only a sea of electromagnetic energy: energy in the form of gazillions of zipping and vibrating atoms and molecules, gazillions of photons, and gazillions of other particles, like neutrinos for example, that we don’t even perceive. Our sensory system automatically filters out most of the data that are out there, filling in the missing data based on our previous experience, and foremost our expectations, to give us the impression of a stable and predictable world.

leaf

If we were a molecule of chlorophyll, say, inside one leaf in the tree in front of us, the world would appear very differently to us. If we imagine this molecule being also aware of all phenomena that are important to her own existence, we can imagine her elemental excitement at the capture of a photon — that thing that makes her vibrate a little faster — and then we can imagine her elemental desire to give its energy to a nearby molecule for which it has a natural affinity. We can also imagine that she is aware of the molecules around, because she receives little tugs as these molecules vibrate.

Maybe she is even able to recognize some of her neighboring molecules by the way their vibrations feel. She may also be aware of the “temperature” of her surrounding by sensing the overall vibrations of the neighbors. Little does she know, however, that she is inside a leaf and that her function is essential to the life of the leaf and to the life of the tree. Her instinctive “knowledge” is only about what she can directly sense. Therefore, she has no concepts of leaf, or tree, or forest, or planet earth.

At this point it comes natural to ask, then, if our condition may be similar to the one of the chlorophyll molecule, despite our far vaster awareness. After all, do we know the nature of the “leaf” of which we are a part? Do we know the tree to which our leaf belongs? Do we know the forest our tree belongs to? I would love to pursue this line of inquiry but it would take us far afield. Therefore I will return to the main questions in the hope of having stimulated the reader to think a bit more deeply about the nature of his or her own consciousness.

divider-gold

Perception

I would like now to consider the perceptual aspects of consciousness that provide our sensations and feelings, sometimes referred to as qualia, and I will use the word consciousness with this meaning in this section.

pullquote-electromagnetic

The most advanced hypothesis about perception affirms that the conscious perception of a complex object by a human is achieved when the various parts of the brain, each representing one particular aspects or quality of such object, reach phase coherence. Phase coherence means that all the neuronal spikes of the various brain subsystems involved are in phase with each other, thus creating the condition for binding all those aspects into a unitary representation of the object. This partially verified hypothesis is sensible because it explains how we may represent complex objects with the same hardware used to represent all other objects. Feature binding allows reutilizing the brain’s neural networks for any object by using phase-locking-in-time as the temporary specialization of the hardware to represent the particular object under consideration.

This explanation however says nothing about the sensation we have regarding the object. In fact, sensation appears unnecessary to the operational task of representation and recognition of objects. The basic question is then, “What are the physical principles that allow electromagnetic activity in the brain to be translated into feelings?” We know the physical principles that can potentially explain the complex electromagnetic activity of the brain that is associated or correlated with seeing the glass of wine, touching it, smelling it, and sensing the liquid in our mouth. But where do the “picture” of the glass, the “sense” of holding it, the “feeling” of liquid in our mouth, the “aroma,” and the “taste” of wine come from? Physics can explain how the machine encodes information to represent something, but not the feelings produced by such information. In fact, there is nothing in the laws of physics that can explain or predict feelings!

brain-swirls

If we examine the mental space that seems to be empty when we take away all the objects of perception, we begin to recognize that it is like a field of awareness, similar to a computer display, or the screen where a movie is projected, except it is invisible and multidimensional, because in that screen all types of patterns are projected, each type with its own characteristic family of feeling-tones — and not just visual information.

That inner field of awareness is illuminated by all types of sensations and feelings coming from translating the electromagnetic signals produced by (1) our sensory-brain system connected with the senses that face the outer world, and the proprioceptive sensory system that faces the inner world of our internal organs; and (2) the brain structures dedicated to representing our emotions, our thoughts, our imagination, our sense of self, and so on. And this perception captures the entire inner and outer state of a self in a unitary, integrated, holistic, and unique way.

We thus possess the ability to convert electromagnetic brain activity into a multidimensional display into which the inner and outer situations are mapped, forming a representation of the inner and outer aspects of the world that are relevant to us, using the language of feelings. We are aware of our perceptions only because the electrical activity of the brain is converted somehow into feelings. Without feelings we could still perceive and even act properly in simple situations, but we would not be aware, we would be zombies, exactly like our robots are.

I believe that such “display” is indispensable for the exercise of what we consider exclusively human characteristics such as thinking, reasoning, willing, imagining, figuring out, etc. All the machinery underneath the display, however complex and prodigious it may be, is irrelevant compared with the significance of conscious life because it is conscious life what gives meaning to our existence. But how is this putative translation from electrical activity into feelings taking place? How does consciousness work?

Science is completely at a loss to explain qualia within known physics, or computer science, or neuroscience; and this fact strongly suggests that we are missing something fundamental in our understanding of nature. If consciousness were just an emergent property of a complex information system, given the sophistication of our current information technology, we should already be able to create a robot with a primitive consciousness.

The fact that we do not even know how to begin to design a conscious robot, indicates to me that we are dealing with another order of reality, a reality way beyond mechanism.

Patterns in computer memory do not produce sensations; they can produce reasonable and appropriate automatic responses, even responses that humans may believe are similar to ours, yet robots do not have a field of awareness, they simply do what they are programmed to do, and it’s only our proclivity to project consciousness on anything that behaves like us that deceives us into believing they might be conscious. However, the robot’s life has no sensations, no feelings, and no meaning because meaning and consciousness are intimately intertwined; there isn’t one without the other.

It is our consciousness that produces feelings, and it is consciousness that gives us the strongest evidence that we are more than machines.

Sensations and feelings are a different category of phenomena than physical phenomena. Electrical currents and memory bits do not produce sensations in a computer, no matter how sophisticated the program is; nor do image sensors produce the sensation of light or color in a digital camera. When the sensor image data are properly processed and displayed in a screen that generates the light that enters our eyes, then sensations similar to those caused by the real world are produced in us. But again, this conversion is achieved by a human, not a machine.

If a computer, no matter how complex, doesn’t have any sensation, then it follows that the brain is not like a computer and furthermore, sensations and feelings require something in addition to the physical operation of the brain that is based solely on known physics.

To believe that the brain, as an isolated system, can cause a conscious experience is like believing that the picture on our TV originates from inside the TV. It is likely that the brain is instead more like a terminal than a computer, translating signals from the physical world into symbols that our individual consciousness perceives, comprehends and acts upon. Thus it is reasonable to assume that to explain consciousness we may need to discover some new physics. It may even be necessary to postulate that consciousness is a fundamental property of nature, present in everything that exists, and growing with the organizational complexity of matter, which would explain why it becomes apparent only in higher animals.

divider-gold

A Possible Explanation for Consciousness

We know from quantum physics (QP) that physical reality is an undivided wholeness, and that what appear to be separate parts is only a human construct, a fiction, an approximation. There are no disconnected parts because there are no real boundaries between parts and wholeness. An elementary particle in its unobserved and unconstrained state is everywhere at once. Only when it is observed it appears localized as a particle. Everything is connected with everything else. Therefore, the knowledge that can be gained by studying the individual parts in isolation will not be sufficient to understand the operation of the whole. Something vital will be inevitably left out. Therefore, the generally accepted idea that we can completely explain the operation of the whole by knowing the operation of its parts is fundamentally flawed.

pullquote-knowledge-whole

Despite the evidence provided by QP that reality is an undivided, nonlocal wholeness, we cling to the Newtonian view of reality provided by classical physics (CP) where separate parts are held to exist in isolation; a view that has proven to be false. Furthermore, non-locality is a fundamental property of quantum systems that defy CP because it instantaneously connects entangled particles, like two electrons that share a common property for example, as if space and time didn’t exist.

I believe we resist facing the fact that reality is much more complex than what our mathematical models say it is, in part because the complexity of our models is already daunting when we separate the parts, and if we had to also include the residual coupling between parts, then the problems we need to solve would become impossible to compute. Consciousness is a holistic property, a property of wholeness, and there is no shred of solid evidence that consciousness emerges from the operation of the brain and is the result of atoms banging against each other.

It is just assumed that consciousness must be produced by the brain, and therefore it must be an epiphenomenon of the brain since there is no other plausible way to explain how it might occur by the interaction of atoms and molecules. But this is circular reasoning, and I believe it is time to scientifically and seriously investigate the possibility that consciousness might have a different origin.

In the late nineties, I started considering the idea that consciousness may be a primary aspect of reality. In other words, consciousness may be an irreducible property, or aspect, of the primordial energy, or substance, of which everything is made. If true, this means that the energy of the Big Bang, the energy that created space, time and matter is aware energy; i.e. in addition to containing the seeds of space, time and matter this energy also contains the seed of consciousness, and therefore anything that exists must be inherently conscious. And the more I reflect about this conjecture, the more unification potential and explanatory power it seems to have.

From this hypothesis, it follows that the objective and the subjective worlds are then two faces of an indivisible wholeness right from the start. In other words, the nature of reality has inherently an inner and an outer aspect, and the two are co-emergent and co-evolving. The inner aspect is consciousness; the outer aspect is the physical universe of energy-matter and space-time. Therefore the physical evolution of the universe mirrors the evolution of consciousness, and vice versa. One supports the other, and thus the physical world represents the outer manifestation of universal consciousness, and consciousness connects everything from the inside.

The aware energy of the Big Bang is non-physical energy in the sense that it comes from “outside” the physical universe that it is forming, and the physical universe emerges within this “expanding” aware energy. Consciousness and the physical world are just two aspects of wholeness.

Like it or not, quantum physics and general relativity let the genie out of the bottle when they discovered the intimate connection between the observer and the observed.

The observer affects what is observed and there is no totally objective world out there that is independent from its observation. These new findings can no longer be wished away, and their implications about the nature of reality must be deeply explored, despite being difficult to accept. Since the nature of the observer is central to each of the two fundamental physical theories that have resisted unification for 90 years, I believe that physics cannot afford to ignore the study of consciousness because it might just be the missing link. By refusing to accept that consciousness may be a fundamental, or perhaps the fundamental property of nature, physics may not be able to unify its two most successful theories.

We have to take this conjecture seriously because there is much at stake. Consciousness is the big mystery; it’s the white elephant in the room; and the still unresolved interpretation of quantum mechanics — the so-called measurement problem — is another indication that the rabbit hole may be indeed very deep.

It is time to seriously and scientifically find out what’s going on, and I am glad that a few physicists are currently beginning to take notice of this fundamental and mysterious aspect of reality of which so little is known, and that has the potential to affect our physical theories in a major way.

divider-gold

Part 2: Life at the Cusp of Complexity