TITLE: StrangeLoop 4: Computing Like The Brain AUTHOR: Eugene Wallingford DATE: September 27, 2012 5:33 PM DESC: ----- BODY: Tuesday morning kicked off with a keynote address by Jeff Hawkins entitled "Computing Like The Brain". Hawkins is currently with Numenta, a company he co-founded in 2005, after having founding the Redwood Neuroscience Institute and two companies most technophiles will recognize: Palm and Handspring. Hawkins said he has devoted his professional life to understanding machine science. He recalls reading an article by Francis Crick in Scientific American as a youth and being inspired to study neuroscience. It was a data-rich, theory-poor discipline, one crying out for abstractions to unify our understanding of how the brain works from the mass of data we were collecting. He says he dedicated life then to discovering principles of how the brain works, especially the neocortex, and to build computer systems that implement these principles. The talk began with a primer on the neocortex, which can be thought of as a predictive modeling system to controls human intelligence. If we take into account all the components of what we think of as our five senses, the brain has millions of sensors that constantly stream data to the neocortex. Its job is to build an on-line model from this streaming data. It constantly predicts what he expects to receive next, detects anomalies, updates itself, and produces actions. When the neocortex updates, we learn. On this view, the brain doesn't "compute". It is a memory system. (I immediately thought of Roger Schank, his views on AI, and case-based reasoning...) The brain is really one memory algorithm operating over all of our sensory inputs. The key elements of this memory system are: Hawkins spoke briefly about hierarchy and sequence memory, but he quickly moved into the idea of sparse distributed representation (SDR). This can be contrasted to the dense, localized memory of traditional computer systems. For example, ASCII code consists of seven bits, all combinations of which we use to represent a single character. Capital 'A' is 65, or 1000001; the digit '5' is 55, or 0110111. The coincidence of '5' and 55 notwithstanding, the individual bits of an ASCII code don't mean anything. Change one bit, and you get a different character, sometimes a very different one. An SDR uses a large number of bits, with only a few set to 1. Hawkins said that typically only ~ 2% of the bits are "on". Each bit in an SDR has specific meaning, one that has been learned through memory updating, not assigned. He then demonstrated several properties of an SDR, such as how it can be used to detect similarities, how it can do "store-and-compare" using only indices, and how it can perform remarkably well using on a sampling of the indices. Associative look-up in the brain's SDR produces surprisingly few errors, and those tend to be related to the probe, corresponding to similar situations encountered previously. The first takeaway point of the talk was this: Intelligent systems of the future will be built using sparse distributed representation. At this point, my note-taking slowed. I am not a biologist, so most of what Hawkins was describing lies far outside my area of expertise. So I made a master note -- gotta get this guy's book! -- and settled into more focused listening. (It turns out that a former student recommended Hawkins's book, On Intelligence, to me a year or two ago. I should have listened to Allyn then and read it!) One phrase that made me smile later in the talk was the semantic meaning of the wrongness. Knowing why something is wrong, or how, is a huge step up on "just" being wrong. Hawkins referred to this in particular as part of the subtlety of making predictions. To close, Hawkins offered some conjectures. He thinks that the future of machine intelligence will depend on us developing more and better theory to explain how the brain works, especially in the areas of hierarchy and attention. The most compelling implementation will be an embodied intelligence, with embedded agents distributed across billions of sensors. We need better hardware in order to create faster systems. recall that the brain is more a memory systems than a computation device, so better memory is as or more important than better processors. Finally, we need to find a way to increase the level connectivity among components. Neurons have tens or hundreds of connections to other neurons, and these can be grown or strengthened dynamically. Currently, our computer chips are not good at this. Where will breakthrough applications come from? He's not sure. In the past, breakthrough applications of technologies have not always been where we expected them. I gotta read more. As a student of AI, I was never been all that interested in neurobiology or even its implications for my discipline. The cognitive level has always excited me more. But Hawkins makes an interesting case that the underlying technologies we need to reach the cognitive level will look more like our brains than today's computers. -----