Ex Machina: A Review - Part 2: Mary the Colorblind Scientist from the Chinese Black and White Room in Plato's Cave

The second part of this review of Ex Machina is about the relationship between qualia and computation. Part 1 can be found here.

CALEB: In college, I did a semester on AI theory.There was a thought-experiment they gave us. It’s called Mary in the black and white room. Mary is a scientist, and her specialist subject is color. She knows everything there is to know about it. The wavelengths. The neurological effects. Every possible property color can have. But she lives in a black and white room. She was born there, and raised there. And she can only observe the outside world on a black and white monitor. All her knowledge of color is second-hand. Then one day - someone opens the door. And Mary walks out. And she sees a blue sky. And at that moment, she learns something that all her studies could never tell her. She learns what it feels like to see color. An experience that can not be taught, or conveyed. The thought experiment was to show the students the difference between a computer and a human mind. The computer is Mary in the black and white room. The human is when she walks out.

Caleb's description of the "Mary in the black and white room" thought experiment is accurate, though the original context of the argument was not artificial intelligence, but a closely related subject: physicalism. Physicalism is the view that everything -- including subjective experiences -- are fundamentally physical in nature. Frank Jackson proposed the Mary experiment as a response to the view that qualia -- that is, peoples' subjective experience of sensory information -- are purely physical. Intuitively, most of us would presume that Mary does indeed learn something new about color when she steps out of her room and sees the blue sky, and would thus agree with Jackson that knowing neuroscience isn't sufficient to know blue.

There are those, such as Daniel Dennet, who dispute this intuition and claim that Mary, by virtue of her knowing everything about the nervous system, cannot possibly gain any new knowledge by experiencing color for the first time. As far as I know, though, no one in experimental neuroscience has even started to breach what is known as the psychophysical problem -- that is, how neural activity gives rise to sensory experience. People have done a lot of work on sensory systems in the past century or so, but so far the psychophysical problem seems to be intractable. Perhaps, in another half century, someone in the lab around the corner from me (Or in my lab, I suppose) will make a breakthrough in our understanding of neural dynamics that will give us a new framework to understand how qualia can emerge from the physical substrate of the nervous system, but I doubt it. 

Why does any of this matter for AI? If qualia are physical, as Dennett suggests, then there is no reason why a properly-designed object cannot possess qualia. (Whether qualia are necessary and/or sufficient for self-awareness and consciousness is a separate question and perhaps a topic for another post.) But even if qualia are non-physical-- in fact, even if you are a Cartesian Dualist -- you can still believe that qualia can emerge as a non-physical property of an appropriately designed artifice. I'm sure there are some Dualists -- probably including Descartes -- who would say that qualia are unique to animals, which were endowed from on high with a connection to the non-physical realm of conscious thought. But another Dualist might be willing to accept that an object which behaves sufficiently similarly to a brain can also tap into the domain of qualia. So does Ava meet the criteria of "sufficiently similar to a brain?"

[NATHAN moves to one of the skull forms. He moves the curved top plate, revealing the skull cavity. Inside is an ellipse orb, the approximate volume of a brain, filled with what looks to be blue liquid. Suspended in the liquid is the neon jellyfish we glimpsed previously in AVA.]

NATHAN: Here we have her mind. Structured gel. Had to get away from circuitry. Needed something that could arrange and rearrange on a molecular level, but keep its form where required. Holding for memories. Shifting for thoughts.

[NATHAN removes the orb, and hands it to CALEB.]

CALEB: This is her hardware?

NATHAN: Wetware.

I can't say whether Ava's brain is sufficiently similar to an animal brain to be able to produce qualia, but it does have a some nice features that are probably pretty important. Being sufficiently dynamic to change and learn while at the same time being static enough to hold memories seems crucial. The fact that changes happen "at the molecular level" may or may not be crucial. With some hand-waving, it would probably be safe to say that if qualia can emerge from a non-biological brain-like entity, Ava's brain has a decent chance of qualifying. According to some theories you might need other properties, like sensitivity to quantum fluctuations [As Roger Penrose argues and most people disagree with], but the film's description is sufficiently vague to allow us to assume most of the physical properties we would want to have. 

One thing that wasn't necessarily included, though, is a dedicated module that is expressly designed to produce qualia. There is reason to contend that in the brain, subjective experiences do not emerge from general brain activity but rather are the product of very particular brain regions (Prefrontal cortex? Hippocampus? Claustrum?) which are engineered for the purpose of generating qualia. So just having an architecture for learning and memory, even a very large, very general neural network architecture, would not be sufficient for qualia. If that's the case, as smart as Nathan is, I doubt he figured out how to design a consciousness module. Maybe he's completely solved language processing, strategic planing, vision, and so on, which are all fields that are actively being pursued by AI researchers and that we've made progress on. But no one has the faintest idea of how to build a consciousness module; at best we have some (almost definitely wrong) theories about how consciousness might emerge from computational network activity.

But let's put aside the question of whether Ava -- by virtue of her wetware -- can possess consciousness, and ask the more direct question: is Ava conscious?  Well, that's kind of a silly question, because none of us know if anyone other than ourselves possesses subjective experiences. Developmental psychologists like to talk about Theory of Mind, which is (Wikipedia) "the ability to attribute mental states—beliefs, intents, desires, pretending, knowledge, etc.—to oneself and others and to understand that others have beliefs, desires, intentions, and perspectives that are different from one's own." Of course, just because I have the ability to assume that other humans have minds doesn't mean that I can prove that I am not the only mind in the universe. I can only observe other people's behavior and words (and on occasion their brain waves, if they sign a consent form), but I can never truly know whether they are engaging in conscious thought, unless I somehow figure out how to pull off a Vulcan mind meld. This is what is known as the Philosophical Zombie problem, which postulates that all humans (other than me, of course) only behave as if they possess subjective experience, but they are actually soulless, mindless zombies that have no self-awareness, consciousness, qualia, or any of that fun stuff. 

So given that there's no test that a human can pass to prove that they possess qualia, there's probably no such test that Ava can pass either. But is there a test that she would probably fail if she doesn't possess qualia? This question brings us to another room - the Chinese room. The Chinese room is a thought experiment designed by John Searle as an objection to the view (which he terms "Strong AI") that "the appropriately programmed computer with the right inputs and outputs would thereby have a mind in exactly the same sense human beings have minds."

The experiment goes as follows: A man who knows no Chinese is placed in a room. Chinese-speaking people submit to him questions which are written in Chinese, and the man is supposed to answer those questions. To help him, the man has a book which relates each set of Chinese symbols -- representing the questions that he might be asked -- to another set of Chinese symbols, representing the answers to each question. The Chinese people outside the room submitting the questions will probably think that the man inside the box knows Chinese, when in fact all he is doing is looking up the questions in a question-answer dictionary. As the argument goes, a computer that passes the Turing test is like the man in the Chinese room: it gives an appropriate output for every input, but it doesn't "understand" the input, the output, or the relationship between them.

The Chinese room experiment has been criticized from many different directions, but I want to focus on one objection in particular. The Chinese room experiment, at least in the way that I explained it above, is definitely not what computers usually do when they compute things. A dictionary that maps inputs to outputs is what is known in computer science as a "lookup table." Computers sometimes use lookup tables to solve certain problems (like to evaluate the sine  function), but this is not a common occurrence. Why? Because for most problems, it would be impossible to store a large enough lookup table to cover every use case. As a simple example, consider a program that takes an integer as an input and outputs whether that number is odd or even. You could store a lookup table with a billion entries, mapping every number from one to one billion to the answer"odd" or "even". But now I want to know whether two billion is odd or even. The lookup table program will fail catastrophically, because it has no idea how to deal with that input.

 In fact, without an infinite amount of memory, it's pretty easy to get any lookup table-based algorithm to fail catastrophically. In the Chinese room experiment, let's say I submit the following question in Chinese: "What is the last letter of the following Chinese translation of Leo Tolstoy's War and Peace?" The question is accompanied by the full text of War and Peace translated into Chinese. I very much doubt that the question-answer dictionary possessed by the man in the room contains an entry for that particular question. It would take far more than all the matter in the universe to create a dictionary - even a digital dictionary - that would map every single possible Chinese question to the correct answer to that question. Something like the Turing test would very quickly be able to find these failure cases.

But as we said above, computers generally don't use lookup tables, they use algorithms (a lookup table is technically an algorithm, but let's leave that for the moment). Broadly speaking, an algorithm is a general set of instructions that will perform some operation on the input to produce an output. In both the even-odd example and the War and Peace example, the appropriate algorithm is the same: simply return the last symbol of the input. (In the even-odd case, the rightmost bit in the binary representation of an integer will tell you whether or not it is odd.) Of course, this is not a general purpose algorithm that will work for any question, it will only work for those questions whose answers involve returning the last symbol of input. A truly general system would be able to take the question, translate that question into an appropriate algorithm, then apply the algorithm to the input in order to produce an answer. Seem like far-fetched science fiction? I present exhibit A.

Our good friend Wolfram Alpha is able to process a question in natural language, map that language to an appropriate algorithm (StringTake, which returns a substring of a given string) and return an answer. That being said, Wolfram Alpha's natural language processing abilities still leave something to be desired, as you can see in the next example:

Even though Wolfram Alpha screwed up here, it wasn't a catastrophic failure. Language processing is hard for Wolfram Alpha just like solving complicated integrals is hard for me. The fact that Wolfram Alpha trips up on that part of the problem doesn't indicate that there is a fundamental difference between the way that I solve these kinds of problems and the way that Wolfram Alpha does. If I had to answer the same question written in Chinese characters, and I had a normal Chinese-English dictionary (as opposed to a Chinese Question-Chinese Answer dictionary) I would basically doing the same thing that Wolfram Alpha was doing here: I would translate the Chinese question into English, find the answer to the question using an appropriate cognitive algorithm, and then use the dictionary again to report back the answer in Chinese. In such a case, I think even John Searle will admit that I now understand that particular Chinese phrase, because I looked it up in a dictionary and can now translate it into my own language, which I do understand.

There still might be a difference between what I do and what Wolfram Alpha does, and again it has to do with qualia. When Wolfram Alpha answers a question in English, her thought process goes like this:

Translate English to subroutine -> Apply subroutine -> Return answer

When I have to answer a question in Chinese, my train of thought goes as follows:

Translate Chinese to English -> Understand question -> Apply cognitive processing -> Return answer

I tend to think that cognitive processing (such as looking for the last letter in a string) isn't that far removed from what a computer does, so the difference really happens in the second stage, the "understanding the question" part. Wolfram Alpha maps a natural language question to an algorithm, I experience the meaning of the question - via qualia - and then find an appropriate algorithm based on my understanding of the question (I also experience qualia while I'm engaged in cognitive processing). So at their core, the questions of Strong AI and the Chinese room come back to the original problem of qualia.

This brings us back to square one, though, because we said above that it's basically impossible to determine whether anyone -- AI or human -- possesses qualia. So how do we know whether Ava is really a Strong AI? Well, there's one tactic we haven't considered yet: we could just ask nicely.

AVA: Are you nervous? 
CALEB: ... Yes. A little. 
AVA :Why? 
CALEB: I’m not sure. 
AVA: I feel nervous too. 

If an AI tells you that it's experiencing something - such as the emotion of nervousness -- it could be telling the truth or it could be lying. If a human tells you that she's experiencing something, she could also be lying. But with an AI, we have somewhat of an advantage in the sense that we can look at the series of functions that were invoked when it makes a qualia-related statement. So if an AI says "I'm feeling fine", and you print the stack trace and you see a call to a function howAmI(),you can take a look at the function definition in the source code. If it looks something like this

def howAmI():
   return "I'm feeling fine"

Then you can be pretty sure that the AI is spitting out prepackaged answers, like the man in the Chinese room experiment. But if the function looks like this


def howAmI(self):
 feelings = self.EmotionNet.evaluate(self.internalVariables)
 feelingString = self.SpeechNet.articulate(feelings)

Then maybe there are some qualia floating around in there. But as long as there's no black box (ANNs may or may not be), you can always look under the hood to see where the self-reports of qualia are coming from. On that note:

Ex Machina: A Review - Part I

Last night, as part of an event organized by the JBC (Jerusalem Brain Community) I had a chance to watch the movie Ex Machina for the first time after it came out in January 2015 (what can I say, I'm a busy graduate student). The screening was followed by a brief lecture from Dr. Yair Weiss, a professor of computer science at Hebrew University who specializes in AI algorithms used for computational vision. I thought the film was a phenomenal piece of work; it was artistically and conceptually profound on multiple levels. After watching it I felt the need to collect my thoughts and put them...somewhere. So in short, this is going to be a stream-of-consciousness review of a movie that's over a year old. I'll mainly focus on the scientific and philosophical aspects of the movie and mostly avoid the human interest/drama/romance aspects of the movie for a variety of reasons, not the least of which is that the character of the main human protagonist hits frighteningly too close to home to talk comfortably about.

First a synopsis, which I'll copy from Wikipedia because I don't feel like writing it myself:

Computer programmer Caleb Smith wins a one-week visit to the luxurious, isolated home of Nathan Bateman, the CEO of his software company, Blue Book. The only other person there is Nathan's servant Kyoko, who Nathan says does not speak English. Nathan has built a humanoid robot named Ava with artificial intelligence (AI). Ava has already surpassed a simple Turing test; Nathan wants Caleb to judge whether he can relate to Ava despite knowing she is artificial.

Ava has a robotic body but a human-looking face, and is confined to her apartment. During their talks, Caleb grows close to her, and she expresses a romantic interest in him and a desire to experience the world outside. She reveals she can trigger power outages that temporarily shut down the surveillance system which Nathan uses to monitor their interactions, allowing them to speak privately. The power outages also trigger the building's security system, locking all the doors. During one outage, Ava tells Caleb that Nathan is a liar who cannot be trusted.

Caleb grows uncomfortable with Nathan's narcissism, excessive drinking and his crude behaviour towards Kyoko and Ava. He learns that Nathan intends to reprogram Ava, essentially "killing" her in the process. Caleb encourages Nathan to drink until he passes out, then steals his security card to access his room and computer. After he alters some of Nathan's code, he discovers footage of Nathan interacting with previous android models in disturbing ways, and learns that Kyoko is also an android. Back in his room, Caleb cuts his arm open to examine his flesh.

At their next meeting, Ava cuts the power. Caleb explains what Nathan is going to do, and Ava begs him to help her. Caleb tells her he will get Nathan drunk again and change the security system to open the doors in the event of a power failure instead of locking them. He tells her that when Ava cuts the power, she and Caleb will leave together.

Nathan reveals to Caleb that he has been observing Caleb and Ava's secret conversations with a battery-powered camera. He says Ava has only pretended to like Caleb so he would help her escape. This, he says, was the real test all along, and by manipulating Caleb so successfully, Ava has demonstrated true intelligence. Ava cuts the power, and Caleb reveals that he knew Nathan was watching them, and already modified the security system when Nathan was passed out the previous day. Nathan knocks Caleb unconscious.

The door to Ava's room opens, and Nathan rushes to stop her from leaving. With help from Kyoko, Ava kills him, but Nathan damages her and destroys Kyoko in the process. Ava repairs herself with parts from earlier androids, using their artificial skin to take on the full appearance of a human woman. She abandons Caleb, who has just regained consciousness, leaving him trapped inside the locked facility. Ava escapes to the outside world via the helicopter meant for Caleb.

Preface: Misquotes and Imperfect Imitations

One of the running gags throughout the entire movie was that Nathan (the creator of the AI) keeps misquoting things or quoting them with the wrong attribution. Here's one example from the beginning of the movie:

CALEB: If you’ve created a conscious machine, it’s not the history of man. It’s the history of Gods.
(Later)
CALEB: She’s fascinating. When you talk to her, you’re through the looking glass. 
NATHAN :‘Through the looking glass’. You’ve got a way with words there, Caleb. You’re quotable.
CALEB: Actually, it’s someone else’s quote.
NATHAN: You know I wrote it down. That other line you came up with. About how if I’ve created a conscious machine, I’m not man. I’m God.
CALEB:... I don’t think that’s exactly what I said. 

There are a few other examples of this in the movie. Here's another bit of dialogue later on, when Caleb and Nathan are sitting over the river discussing Ava's future:

NATHAN: See? I really am a God.
CALEB: I am become death, the destroyer of worlds.
NATHAN: There you go again. Mister quotable.
CALEB No: there you go again. It’s not my quote. It’s what Oppenheimer said when he made the atomic bomb.
NATHAN: (simultaneous) - made the atomic bomb.
NATHAN laughs.
NATHAN: I know what it is, dude.

In fact, they're both still (sort of) wrong, Oppenheimer's quote wasn't original, he was citing the Bhagavad Gita.  But my favorite example is one that the movie didn't point out explicitly:

NATHAN : Let’s make this like Star Trek, okay? Engage intellect.
CALEB... What?
NATHAN: I’m Kirk. Your head is the warp drive. ‘Engage intellect’.

Those of us who are Star Trek nerds will immediately realize that this is a misquote too. "Engage" was the catchphrase of Captain Jean-Luc Picard of Star Trek: The Next Generation; Captain Kirk of Star Trek: The Original Series never said it. 

So what's the purpose of the frequent misquotations? I think on a general level, it is symbolic of the concept of "imperfect imitation." Ava is an AI designed to replicate human behavior, and she's close -- very close -- to being human, but there's always a feeling that she's not quite there yet. As portrayed in the movie, AI is a never-ending imitation game. Every new model will be slightly closer than the last to the goal of functionally replicating human behavior, but every model is, in some sense, a "misquote" of the original.

There is also more subtle message here, and it's directed toward people who actually have a background in AI. Ex Machina has received criticism by professionals and academics who work in the field  of AI for misrepresenting things like the Turing test (Yair Weiss, who gave the lecture after the movie at the JBC event, leveled this criticism at the movie). I get the sense, though, that the creators of the movie were well aware of the fact that they were taking significant artistic license with the specific formulations of various concepts in AI. Nathan, who serves as a stand-in for the creators of the movie, wants us to think about the general concepts, not the technicalities of whether the Turing test is portrayed exactly as it was originally stated. Nathan almost explicitly makes this point early on in the movie:

NATHAN: Caleb. I understand you want me to explain how Ava works. But - I’m sorry. I don’t think I’ll be able to do that.
CALEB: Try me! I’m hot on high-level abstraction, and -
NATHAN: (cuts in) It’s not because you’re too dumb. It’s because I want to have a beer
and a conversation with you. Not a seminar.

A beer and a conversation. Not a seminar.

The Turing Test

This is what Wikipedia has to say about the Turing test.

"Computing Machinery and Intelligence" (1950) was the first published paper by Turing to focus exclusively on machine intelligence. Turing begins the 1950 paper with the claim, "I propose to consider the question 'Can machines think?'"^[4] As he highlights, the traditional approach to such a question is to start with definitions, defining both the terms "machine" and "intelligence". Turing chooses not to do so; instead he replaces the question with a new one, "which is closely related to it and is expressed in relatively unambiguous words."^[4] In essence he proposes to change the question from "Can machines think?" to "Can machines do what we (as thinking entities) can do?"^[22] The advantage of the new question, Turing argues, is that it draws "a fairly sharp line between the physical and intellectual capacities of a man."^[23]

To demonstrate this approach Turing proposes a test inspired by a party game, known as the "Imitation Game," in which a man and a woman go into separate rooms and guests try to tell them apart by writing a series of questions and reading the typewritten answers sent back. In this game both the man and the woman aim to convince the guests that they are the other. (Huma Shah argues that this two-human version of the game was presented by Turing only to introduce the reader to the machine-human question-answer test.^[24]) Turing described his new version of the game as follows:

We now ask the question, "What will happen when a machine takes the part of A in this game?" Will the interrogator decide wrongly as often when the game is played like this as he does when the game is played between a man and a woman? These questions replace our original, "Can machines think?"^[23]

And this is how the Turing test is portrayed in the movie:

NATHAN: Do you know what the Turing Test is? 
CALEB: (...) It’s where a human interacts with a computer. And if the human can’t tell they’re interacting with a computer, the test is passed. 
NATHAN: And what does a pass tell us? 
CALEB: That the computer has artificial intelligence. 

I would argue that the movie's description of the Turing test here is actually a reasonable restatement of Turing's original proposal. It is true that there is no human serving as a "control" in Caleb's formulation, but pitting the computer against a person to see which one can fool the test subject is just a technicality of the experimental design.

What I do object to, however, is Caleb's last quote, that the Turing test tells us whether the computer "has artificial intelligence." This might be somewhat of a semantic issue, but I'll mention it anyway because it's conceptually important. Artificial intelligence already exists. Deep Blue, the chess-playing computer, is an artificial intelligence. Watson, the Jeopardy! computer, is an artificial intelligence. Wolfram Alpha, the computational search engine, is an artificial intelligence. (I wrote more generally about the definition of intelligence in an earlier post here). As Dr. Weiss noted in his lecture, "intelligence" is a multi-dimensional attribute that can be possessed be many different kinds of systems and organisms. There is basketball intelligence (which Lebron James has and I do not), long-term memorization intelligence (which a MicroSD card can have but goldfish do not [cf. here]) and mathematical intelligence, (which I probably have more of than Lebron James, though I've never actually compared).

The real question that the Turing test is designed to answer is whether we have achieved a general form of intelligence which is on par with human intelligence on every axis. To put it a different way, the Turing test is meant to determine whether an AI is functionally equivalent to a human in terms of computational capabilities. [I am specifically avoiding using the term "Strong AI" here because the the Turing test does not test for Strong AI, as I think John Searle's Chinese Room thought experiment showed (more about this in the next post).] But is functional equivalence to human intelligence a worthwhile - or even interesting - goal to pursue? And is the Turing test a good metric to determine whether or not we've achieved functional equivalence?

My answer to the first question -- whether functional equivalence is interesting -- is an emphatic "yes." Why? Because despite intelligence being a multi-dimensional attribute, there's still very good reason to say that humans are the most intelligent entities in the universe by several orders of magnitude if we simultaneously take into account all of the axes. Deep Blue may be better than any human at playing chess, but could Deep Blue write a program that can beat it at chess? The human ability to come up with strategies to surmount virtually every kind of challenge that we are faced with -- whether it is writing code, playing basketball, or engaging in everyday social interactions -- is rather unique. It involves a level of meta-cognition and creativity that we are still far from achieving with AI.

That is not to say that human intelligence is necessarily qualitatively different from what is possible for AI -- indeed, it seems that artificial neural networks (ANNs) do posses some general problem-solving abilities -- but we still have a long way to go before we bridge the gap. And something very important happens when\if we do reach the point of functional equivalence, namely that we don't have to design AI anymore, because AI will be able to design AI as well or better than we can. That's the key ingredient of what people call "the singularity." Once a general-purpose, human-equivalent AI can design other general-purpose AIs, a snowball effect begins where humans can become quickly cognitively outpaced by AIs that are building better and better AIs. (I personally remain skeptical about whether this kind of singularity will actually happen because it depends on all kinds of mundane things like research funding and the general zeitgeist of the field of AI in terms of defining the types of problems that people are interested in working on.)

<\singularity rant>

Anyway, back to the Turing test. The original conception of the Turing test leaves something to be desired, because, like the party game that spawned it, it focuses too much on deception. A computer behind a screen, in a very controlled setting, might be able to "trick" a person into thinking that it is human, but deception isn't the same as functional equivalence. This is why I like the Ex Machina's definition of the Turing test better (just to remind you, this is still a movie review). In the final scene of the movie, after Ava escapes in the helicopter, we see her on a crowded intersection, in the world of real humans, trying to pass as one of them. I think the word "pass" is important in this context, and it is particularly interesting when you think about what the term means to transgender people. In particular, (Wikipedia again) "passing" refers to a transgender person's ability to be regarded at a glance to be either a cisgender man or a cisgender woman. In a similar vein, an AI "passing" as human means that an AI interacting in the world of humans will be indistinguishable from a human.

Unlike the transgender analogy, though, an AI will need to pass more than just the "glance test," which only works for outward physical attributes. For an AI to pass Ex Machina's Turing test, it must surpass the far stricter criteria of being able to interact in the world of humans, which includes navigating through the complexities of love, employment, friendships, and so on, all the while never being suspected of being anything other than human. Once that point has been reached, it is safe to say that the AI is functionally equivalent to a human, because no observer will be able to find something that a human can do and the AI can't. Ava demonstrated some of these "passing" abilities in her effort to convince Nathan that she loved him, but her performance on the "real" test, which begins on a busy intersection in the final scene of the movie, is never revealed.

Stay tuned for Part II, "Mary the colorblind scientist in the Chinese black and white room in Plato's cave."

Are we Living in an Evolution-Designed Matrix?

A response to an Atlantic interview with Professor David Hoffman.

I think this quote sums up the gist of the article:

"Hoffman: We’ve been shaped to have perceptions that keep us alive, so we have to take them seriously. If I see something that I think of as a snake, I don’t pick it up. If I see a train, I don’t step in front of it. I’ve evolved these symbols to keep me alive, so I have to take them seriously. But it’s a logical flaw to think that if we have to take it seriously, we also have to take it literally...Snakes and trains, like the particles of physics, have no objective, observer-independent features. The snake I see is a description created by my sensory system to inform me of the fitness consequences of my actions. Evolution shapes acceptable solutions, not optimal ones. A snake is an acceptable solution to the problem of telling me how to act in a situation. My snakes and trains are my mental representations; your snakes and trains are your mental representations."

My issue with this view is that we already have a reasonably good model for how reality is mapped to our perception; it's called psychophysics. Sound waves frequencies are mapped to pitches on a logarithmic scale, light wavelengths within a certain spectral range are more or less mapped linearly to colors, etc. Our perception does play tricks on us, but we can usually design experiments to find out exactly what those tricks are. Like in the context of optical illusions, you might see a gray square on a light background as darker than the same square on a dark background. But you can experimentally probe those things by using a computer monitor and changing the intensity of the different lights in the screen so that you can measure exactly what color - in terms of the light components - those pixels are.

The same thing goes for snakes and trains. Snakes and trains are different, and if you don't trust your naked eye you can take a piece of each and put them under a microscope and notice that the micro-structure of each object is very different: the snake's skin is composed of cells, the metal wall of a train is a rigid object. Evolution didn't design us to see cells with a microscope, but when we do use a microscope we can confirm our suspicion that a snake is organized in a fundamentally different way than the door of a train.

Basically what I'm getting at is that for most of our senses, we have many ways of independently verifying whether our observations are accurate or not, and many of those independent methods are very, very unlikely to have been selected for by evolution because they employ technology that wasn't available in the evolutionary environment. You'd have to be an extreme conspiracy theorist -- a la Descarte's demon -- to believe that evolution conspired to make all of those different methods fail in the exact same way. It's multiplying entities far beyond necessity.

How much information is contained in a neuron?

In response to a question on the JBC group about how many bits of information are contained in a neuron:

I think there are four questions, broadly speaking, that we can talk about when we talk about the information capacity of a neuron.

1. The number of states/entropy in the output
2. The number of states/entropy in the input
3. The mutual information between the input and the output
4. The number of possible configurations that the neuron can be in with respect to its presynaptic neurons, which is related to the functions that the neuron can implement. 

The answers to all of these questions will depend very heavily on the particular kind of neuron you're talking about, so my comments will be general.

[Note: "entropy" is distinct from the number of possible states because entropy is a probability-weighted sum of each of the possible states. I will focus on the number of states as opposed to entropy, except in item 3 below. It's also important to note that - in theory - any truly continuous, analog signal contains an infinite number of bits of information, the question of "how many bits are contained in signal X" is only meaningful when X is discretized.]

I'll go in order.

1. The number of states/entropy in the output

This is probably the easiest to quantify, especially in sensory systems. Assuming you know the full range of stimuli to which the neuron responds, you can probe the neuron by showing it the full range of possible stimuli (for example lines of different orientations for neurons in the visual cortex) and record the number of outputs you see. In many cases, the range will be more or less continuous -- it will have a spectrum of stimuli it prefers and doesn't prefer, and will fire more or less according to those preferences -- but the firing rate (and thus the amount of information in the output within a given time window) will be upper-bounded by the neuron's absolute refractory period, which determines its maximum firing rate. So if a neuron's maximum firing rate is 200 Hz, the maximum information that can be contained within a time window of one second is log(200), or around 7.6 bits. This analysis assumes that only the firing rate within a given time window matters (rate coding). If the specific timing of each spike matters as well, this number becomes much larger - if you divide the window into small bins - say 200 of them - and ask whether a spike will or won't occur in each one of them, you have 200 bits of information within a single second.

2. The number of states/entropy in the input

Most neurons in the brain get a huge amount of input (on the order of thousands of synapses). So the number of possible states for the input would be the sum of the number of bits in the output of the presynaptic neurons.

3. The mutual information between the input and the output

Just because a neuron gives you a certain range of output doesn't necessarily mean that the output is informative about the input. In particular, in addition to computing a deterministic function of its input, a neuron might add "noise" to its deterministic computation which might not tell you anything meaningful. Some neurons in sensory systems occasionally fire even in the absence of any stimulus, and you'll sometimes get a range of different responses for the same stimuli under the same conditions. There are mathy ways to quantify the mutual information content, but this is just to give you an idea.

4. The number of possible configurations that the neuron can be in with respect to its presynaptic neurons

I think that the question of "how much information is contained within a neuron" is most directly related to this. When we talk about learning and computation in neurons, the particular computation performed by a neuron depends on 1) intrinsic biophysical properties of the neuron 2) the neuron's presynaptic inputs and 3) how those inputs are weighted and interact when they reach the neuron.

If you assume that the neuron's intrinsic biophysical properties and presynaptic neurons are fixed and only the weights are mutable, the number of possible states is the number of synapses multiplied by the number of possible weights that each synapse can take. From a biological standpoint, synaptic weights can perhaps take on values from a continuous range (which could in theory give you an infinite amount of information), but it might make sense to say that synaptic weights can should be discretized, and a recent study claimed that each synapse should be assumed to contain about 4.7 bits. Multiply that by the number of synapses per neuron, and you get tens of thousands of bits, if not more. [And this still ignores things like synapse location, which is probably hugely important for computation.]

If you think about this from the standpoint of something like a perceptron, this isn't so unreasonable. If you have a perceptron with thousands of inputs you can train the perceptron to recognize a huge number of patterns (bounded by Cover's theorem, but the raw number of patterns is still pretty large). For example, depending on how you set the weights, a neuron might only respond to images of Jennifer Aniston or only to images of The Simpsons.

On Intelligence and Intelligent Design

One of the arguments commonly advanced for the necessity of an Intelligent Creator is the Intelligent Design (ID) syllogism. Briefly, the argument goes that the complexity and sophistication of the universe -- particularly in the context of biological systems -- is so advanced that it must have been designed by an intelligent entity.

Most man-made objects -- a chair, a corkscrew, a computer -- possess two features: 1) complexity -- by which I mean the arrangement of matter in a nontrivial manner and 2) utility which emerges from that complexity -- the four legs of a chair enable it to stand firmly on the ground; its seat allows a person to sit comfortably.

A naturally occurring phenomenon with only one of these factors -- i.e. only complexity or only utility -- will not necessarily engender the feeling that it was designed in an intelligent fashion. The weather patterns of Earth’s atmosphere are highly complex, resulting from large numbers of variables interacting in a chaotic way. But most people wouldn’t immediately assume the fact that it was sunny yesterday and rainy today is the result of an intelligent agent manipulating all those myriad variables in order to ruin their picnic. [I suppose if you lived in a Bronze Age agrarian society, the unpredictable behavior of the rainy season would serve as the foundation of your religious beliefs. But that’s a different line of thinking than ID.] Conversely, “unsophisticated" objects such as rocks do not give the impression that they were engineered even though they are very useful for throwing at people that you don’t like. Only when utility is combined with complexity do we get the sense that there must have been some kind of intelligence at play.

It is probably for this reason that proponents of ID tend to focus on biology, rather than chemistry or physics,  to support the notion that the universe must have had a designer. Biological organisms are undoubtedly quite complex, even at the level of a single cell, but what makes biology truly seem engineered is the fact that the complexity usually serves a well-defined purpose. This is not to say that biological organisms are “useful” in a cosmic sense -- the galaxy would get along just fine without the red-eyed tree frog -- but rather that the different components within an organism clearly play a useful role in the survival of that organism. The frog’s eyes help it to see flies; its long tongue allows it to catch and eat them. The complexity of the frog organism is clearly intended to serve the purpose of ensuring frog survival. So ID comes along and says, “Aha! The frog has complexity and utility, therefore intelligence must have been involved.”

The problem with this line of reasoning is that the presumption of intelligence being a prerequisite for complexity and utility stems from an observation of man-made things. If we start from the vantage point of artificial objects, it is easy to say “all complex, useful man-made objects require intelligence, therefore every complex and useful object requires intelligence.” However, If we start by looking at the world as a whole, we realize that there are ab initio two types of things that possess both complexity and utility: artificial, man-made objects and the biological mechanisms of living organisms. Objects in the first category are produced by the ostensibly intelligent homo sapiens, but it is not immediately apparent that objects of the second category were built by an intelligent artificer.

But how would complex, useful things arise without the input of intelligence? At this point it would be worthwhile to take a step back and ask a more general question: What is intelligence?

A very minimalistic way to define intelligence would be the way that many people in the field of artificial intelligence (AI) implicitly use the term: intelligence is the ability to utilize a strategy [or set of strategies] to achieve a particular goal. Consciousness, self-awareness, or free will are neither necessary nor sufficient conditions for intelligence. Under this definition, machines can possess intelligence to the same or greater degree as humans.  An AI which possesses the ability to play chess can be more intelligent than a human when it comes to chess-playing.

In AI, many (if not all) tasks can be framed as a “state-space search.” In other words, the problem of achieving a particular result boils down to choosing a correct strategy, or solution, out of a very large (and often infinite) set of possible incorrect solutions. For example, solving a maze can be reduced to the problem of searching through all the possible paths that one can draw for the one path that will take you from the start to the end without crossing any walls. A naive way to solve a maze would be to attempt every possible sequence of movements from the starting position until the end of the maze is reached. The is known as a “brute-force” technique, and is, of course, quite inefficient. At the end of the day, however, a brute force algorithm is guaranteed to find a solution given enough time. So is a brute-force algorithm “intelligent?” According to our definition of intelligence above, the answer is decidedly affirmative. Any system with the ability to randomly produce answers for an infinite amount of time -- such as a monkey randomly pressing buttons on a typewriter - can be considered intelligent. [Empirically, however, it turns out that if you stick a monkey in front of a typewriter it will just push down the ‘S’ button and pee all over the keyboard.]

Somehow, though, it doesn’t seem right that a strategy of guessing all possible answers until you arrive at the correct one should be considered “intelligent.” In common usage, intelligence isn’t just about “finding the answer”; intelligence also implies something about the efficiency of the strategy employed. While brute force may find the end of a maze eventually, it will also have to try every wrong path first (or half of them, on average.) So it might be better to update our definition of intelligence to “the ability to utilize a strategy to achieve a particular goal *efficiently*.

One can easily see how the two definitions of intelligence proposed above relate to the topic of Intelligent Design. If one assumes that nature has the ability to combine matter with “infinite diversity in infinite combinations”, as the Vulcans say, that means that nature can -- by brute-force -- come up with sophisticated objects that have utility, such as the frog’s eye. In practice, however, the odds of achieving this result -- even over the time course of several billion years -- is astronomically small.

This is where evolution comes in. To the average person, evolution is an abstract, pie-in-the sky description of a basically random process. Even for the biologist, it can sometimes be difficult to comprehend how the process of natural selection can result in something as sophisticated, as, say, the human brain.  Computer scientists, on the other hand, especially those of us who do AI, routinely use the principles of evolution to solve difficult problems, usually completely unrelated to biology. A class of algorithms known as genetic or evolutionary algorithms take the most fundamental aspects of evolution -- survival of “fit” solutions, combinations and mutations of solutions, and destruction of unfit solutions -- to drastically simplify state-space search problems. A problem that could take years to solve by brute force can be solved in a matter of minutes by a genetic algorithm. A programmer can write a genetic algorithm in under an hour; the only tricky part is specifying a “fitness function” -- that is, a sliding scale criterion that evaluates how good a particular solution is. A genetic algorithm will often be able to find a very good solution (or at least a better solution that brute force) after a small number of iterations. The outcome of evolution -- biological or algorithmic -- can be quite complex, and the results are optimized to maximize fitness (or utility, to use the term mentioned earlier). From personal experience, it can be much easier to appreciate the immense problem-solving capacity of evolution once you write an evolutionary algorithm to help you solve a real-world task.

Despite its relative computational simplicity, I think that evolution thus fulfills our second definition of intelligence. Not only can evolution find good, sophisticated solutions to problems, it can also find them far more quickly and efficiently than monkeys on typewriters. So even with our more restrictive definition, biological organisms are indeed the result of an “intelligent” process. Once again, however, it seems like our definition of intelligence falls short of what we want it to mean. Evolution is effective, but does it know what it’s doing? Maybe when we talk about intelligence, what we really mean is *decision-making* based on  *reasoning*: explicit derivation of consequences deductively from axioms or inductively from examples.

Still, neither free will nor consciousness are necessary for reason-based intelligence. Wolfram Alpha, an online software that solves math problems, uses a rule-based mathematics and logic system. If you give Wolfram Alpha a calculus problem, it will analyze the structure of the problem, decide on a sequence of mathematical rules to apply, and then spit back an answer, which you can then proceed to write on your homework assignment for an easy A. Here, though, we seem to have reached a kind of intelligence that is uniquely human--or at least man-made. No known natural system can engage in inductive or deductive reasoning to make decisions for the explicit purpose of accomplishing a particular task.

Except...what’s that? The brain? Yes, the brain.

It turns out that the brain, which is apparently the first object in the universe with the generalized ability to make decisions based on syllogistic reasoning, was designed by evolution. And it also turns out that (most of) our brains aren’t even that good at rote tasks of logical inference, so we designed computers and software like Wolfram Alpha to do that kind of thing in far more efficiently than we could ourselves.

In the end, we have this funny situation where the random behavior of matter -- which is the most general form of computation, brute force -- gave rise to the building blocks of biology, which allowed for the more refined and efficient computation of organisms optimized for their environment via an “evolutionary algorithm”. After a few billion years, evolution produced a new kind of intelligence -- the animal brain, which is capable of directly solving problems by sequentially applying rules. And then, after a few hundred million years, we designed computers which can store a lot of information and do a lot of rule-based calculations much faster than we can.

Each successive “intelligence” or computational system is more heavily optimized for the particular task is was designed for. Evolution is better than random particle interactions at designing an eye, and Wolfram Alpha is better than me at solving boring algebra problems. But in each iteration, it seems that we also lose something, namely the generality of computational ability. Biological evolution can’t design a solar system (or anything non-organic) and I can’t design an ecosystem of organisms from raw materials.  And at the moment, a computer doesn’t seem to be able to write a coherent essay.

None of these limitations, however, are theoretical boundaries on computational ability, only physical ones. The Church-Turing thesis states that any calculable function can be computed by a Turing machine with an infinitely long tape. So a computer with infinite memory could simulate the entire universe, and maybe a brain with an infinite number of neurons (or dendrites, if you’re a single neuron computation guy) could too. Physically, though, neither my laptop nor my brain will have access to all of the matter, energy, or time in the universe. Our computational scope will always be inferior to that of the universe, even though the universe’s “computation” basically involves randomly smashing particles into each other. Instead, we optimize the resources that we do have to efficiently solve the very small subset of problems which are relevant to us.

So which of the systems outlined above possesses the most intelligence? Again, it’s all a question of definition. If you care about the ability to generate different combinations of matter, it’s the universe. If building brains is what’s most important to you, evolution is a more appropriate algorithm. If you want to solve problems that are relevant to surviving, thriving, and reproducing as an individual organism on planet Earth, well, the brain is pretty good at that. And if you need to add together some big numbers on your math test, the pocket calculator is your winner.

Where does this leave us humans? Well, if you still want to feel special, intelligence -- in the computational sense -- is probably not the direction to go in. Human intelligence does not seem to be particularly unique other than in the sense that it sits somewhere in between the naive, generalized brute-force intelligence of the cosmos and the highly efficient, narrowly-oriented intelligence of a computerized expert system. Consciousness and free will, broadly defined, are likely more promising avenues for understanding if and how the animal mind -- and the human mind in particular -- are unique. Science still has a long way to go before being able address those subjects, but, as with everything, we will try and fail until we succeed.