Mirror Neurons and HCI

The November 2006 Scientific Ameracn has a Neuroscience article entitled "Mirror in the Mind". To quote:

Actions performed by one person can activate motor pathways in another's brain responsible for performing the same action. The second understand viscerally what the first is doing because this mirror mechanism lets her experience it in own her mind.

In other words, we understand another person through a virtual simulation of that person. This simulation enables us to understand intentions; if we assume the wrong intention, then the simulation "will not stick", exposing our error: Playing the action, starting with the assumed intention, would lead to an action that does not match what we see or hear. A companion paper associates autism with a malfunctioning of the mirror neuron system, resulting in an inability to understand other people's intentions and, hence, an inability to communicate.

Should be adopt the concept of a mirror neuron system to human-computer interaction? Current user interfaces are low level; attempts to provide a higher-level interface (the famous Microsoft paper clip) have been a total failure. The "paper-clip" helper of Microsoft was annoying because it did not have an adequate model of the human user: it often gave advise that was totally irrelevant, and it could repeat the same advise ad nauseam: the help agent did not understand the user.

How does one understand the user? The common approach would be to build a formal model, e.g., a finite automaton model, with state transitions triggered by computer events -- and possibly train this model to respond well to actual users. The model is specific to the restricted domain and set of restricted responses that the user interface provides: the paper clip agent does not really need to know what we think, it needs to know what is the appropriate feedback it should provide in any given situation. Such model does not attempt to model the user, but just to model the suitable interactions with the user.

The mirror neuron model suggests that to understand a human being, you need to be a human being. In other words, it is possible that computers will be able to have a truly intelligent dialog with human beings only when they will be able to simulate the cognitive processes of a human being. Otherwise, machines will continue to be "autistic" repeatedly exposing their inability to understand us. The paper clip agent will decide which feedback to provide to a user by "putting itself in the place of the user" and checking whether a particular piece of advise would have been useful to itself (or himself?).

Rather than building ad-hoc user models for each interface and each application, this would suggest building a general purpose user model that is then specialized to each interaction. True, general AI has been a failure. But as people such as Hans Moravec have pointed out, general AI has been tried at a time where computers had a compute power and memory size that was ridiculously small as compared to the human brain. Beside, building a human model is not "general AI", it is only "human AI".

Has the time come for a major new effort in this area?

Green Technology or White Technology?

In a recent thought provoking recent article, "our Biotech Future", Freeman Dyson discusses the future of biotechnology. I strongly recommend reading the article, that appeared in September in the New York Review of Books, for its wonderful insights and powerful vision. I also strongly believe that the article is spectacularly wrong in several of its fundamental assertions.

Dyson argues that, for millennia, we lived in an era of green technology, namely agriculture, that has its foundation in biology. We then shifted, with the industrial revolution, to gray technology, that has its foundation in physics and chemistry, and moved to towns. Dyson predicts that we are entering the century of biology and, with genetic engineering, shifting back to green technology.

In fact, it is quite obvious that we are shifting very fast into white technology, a technology based on information. Most of the value produced by our economy is not in physical products, be they green or gray; it is in information. More than 70% of our GDP is in the service sector. Most of the value in the service sector is not in hamburger flipping or hair cutting, but in information: To quote "In soft-sector employment, people use time to deploy knowledge assets, collaboration assets, and process-engagement to create productivity (effectiveness), performance improvement potential and sustainability". Information drives insurance, banking, retail, education, and so on. We are all somewhat mystified by the huge profits enjoyed by Wall Street outfits that seem to produce nothing concrete. Indeed, they do not produce anything material -- they produce information. To a person with a materialistic mindset, this large information creation activity seems parasitic on true wealth creation that resides in material goods; some centuries ago, many saw manufacturing as parasitic on true wealth that resides in land and agriculture. Food production by green technology is still essential to our existence, but it employs little more than 1% of our population. Widget production by gray technology will continue to be essential to our well being, and may shift to green technology; but it will employ a decreasing fraction of the population as more people shift to the creation, communication and transformation of information.

Dyson lays great hopes in genetic engineering that will put an end to the "Darwinian interlude" -- this period where genes where carefully kept within species, so that a successful gene could only slowly propagate. Instead, we are going back to a period of "horizontal gene transfers", where genes are freely shared across all organisms; a period that, according to Dyson, existed before tight genetic regulation appeared and species differentiated -- a period that saw fast evolution.

In fact, as Dyson remarks, horizontal gene transfer in the norm in cultural evolution: ideas are not kept secret and carefully transfered from parent to child, but are broadly circulated. People and cultures learn from each other, and freely exchange ideas (or steal ideas from each other); cultures that encourage the free exchange of knowledge evolve faster and are more successful.

The white technology is, at its most fundamental, a technology that accelerates the horizontal transfer of "information genes", and thus accelerates cultural evolution. Therefore, an increasingly important research topic in computing and information is "social informatics" or "community informatics" -- the study of the socio-technological infrastructures that facilitate collaborations and sharing of ideas. White technology threatens states and organizations that want to control the flow of ideas and encourages new organization forms, where the free and efficient sharing of "information genes" is the fundamental mode of operation. Pace Dyson, we are not entering an era of "open source biology", but an era of open source information, including biological information.

Dyson hopes that a shift back to green technology will help reducing rural poverty. In fact, white technology is likely to help most. White technology reduces distances, as face to face interaction is increasingly replaced by digital interaction that is largely distance insensitive. White technology reduces transportation costs -- it costs next to nothing to transport information, so that information creation jobs can be easily moved from urban areas to rural areas, or from developed countries to developing countries. The change will be progressive: the human race has perfected face-to-face communication over millions of years, and it will take time for electronic communication to catch up; but the change is already occurring; it is transforming our daily life, changing urban patterns and shifting jobs across boundaries.

The information revolution has only started; its profound transformation impact on our culture and our economy will take decades -- perhaps centuries -- to be fully felt.