seminar on eye cry.......

Will Ever Any Computer Cry Digital Tears?...

Abstract

For decades, researchers have tried to understand how emotions are generated, expressed, and recognized. A variety of theories have emerged. At one extreme is the idea that emotions are the experience of physiological changes such as the increased heart rate that accompanies anger. Researchers at the other extreme see emotions as purely cognitive, merely another form of thought. Psychologists leaning toward the first concept tend to look for universal physiological changes that correspond to emotions (such as raised eyebrows when a person is surprised). There is presently no widely accepted, comprehensive theory of emotions, although much has been learned. HAL is the computer who is able to understand and also react on Human Emotions.

Unlike our 1990’s computers, HAL speak; HAL understand what you say; HAL think; HAL see; HAL lip read; HAL show emotion; HAL disobey; and HAL respond with violence when threatened. HAL’s artificial intelligence stirs the imagination of everyone who is familiar with him HAL handily beat Frank Poole at chess. When IBM’s “Deep Blue” computer won the first game of its 1996 match with chess master Garry Kasparov you may have thought that computers had achieved a high level of intelligence. Don’t be fooled! Deep Blue uses “brute-force” processing power to search out 100-million positions per second! It will beat most humans. But it does not think. It does not reason. It lacks common sense. Deep Blue cannot play chess and pilot a spacecraft at the same time. Even with its “attention” totally focused on chess, it lost to Kasparov. Computer scientists, once very optimistic about creating human-like intelligence have a long, long way to go. 

How might HAL, or tomorrow's affective computer, develop a better relationship with you? For starters, it might be endowed with basic perceptual abilities, such as vision or hearing and speech understanding. To date, however, the emphasis in these research areas has been predominantly on tasks such as recognizing who you are and what you are saying Recognizing who is speaking and what is being said is important, but at times these observations are not as important as the expression on the speaker's face and how she or he said it.

This Paper tries to focus on the Features Humans want to program on Computers so that they will be able to Love, Hate, fear and feel & express all such emotions that are expressed and felt by Humans. Will it be really possible? If yes, then what will be the Advantages? Will any drawbacks affect these innovative Ideas? Will ever any Computer cry Digital Tears? .........

Contents

1)      Introduction

2)   Emotion Sensing & Recognition

3)   Responding To Emotions

4)   Teaching Computers To Recognize & Express Emotion

5)   Towards Trully Personal Computers

6)   Emotions With Reasons

7)   Computers That will have Emotions

8)   Where Do We Go From Here?

9)   Conclusion

10)          Blibography

Introduction

HAL in 2001 was affective: he had specific abilities relating to, arising from, and deliberately influencing people’s emotions such as: he could sense and recognize human emotion, respond rationally to it, express emotion, and even give the appearance of “having” emotion. In fact, HAL was the most emotional character in the film 2001. As the millennium dawns, we find that most computers today do not have affective abilities, but there is active research beginning to succeed in giving them a subset of such abilities.

Is it beneficial to make computers affective? Alternatively, is this just a theatrical gimmick, something that makes film characters entertaining but we wouldn’t really want in real life? The latest neuroscience  evidence supports the former: emotions are essential not only to dealing effectively with social-emotional interactions, but also they perform important regulatory and helpful biasing functions within the body and brain, even aiding in rational decision making (Damasio, 1994) and perception (LeDoux, 1996). These and a variety of other important roles of emotion, together with findings from neuroscience, cognitive science, and social-psychological sciences, have been argued to be important reasons for giving machines emotional abilities if they are to be intelligent (Picard, 1997).

What is the state of the art regarding giving machines emotional abilities? Can machines really “have” emotion? HAL’s emotional state is associated with detrimental consequences for human life; thus, wouldn’t giving machines emotion-like mechanisms be potentially dangerous? These are just a few of the questions that arise regarding affective computing: computing that relates to, arises from, or deliberately influences emotion. Affective computing also involves giving machines skills of emotional intelligence: the ability to recognize and respond intelligently to emotion, the ability to appropriately express (or not express) emotion, and the ability to manage emotions. The latter ability involves handling both the emotions of others and the emotions within one self. Because it is a large discussion whether computers can “have” emotions (and even a “self” to experience emotion) and the topic is addressed in a recent book chapter (Picard, 2001), this topic will not be addressed here. A discussion of the ethical issues related to HAL and an in-depth treatment of the state of the art as of 1997 regarding affective computing and HAL’s abilities appear as chapters in the book HAL’s Legacy (Dennett, 1997; Picard, 1997). 

Today, more than ever, the role of computers in interacting with people is of importance. Most computer users are not engineers and do not have the time or desire to learn and stay up to date on special skills for making use of a computer’s assistance. One can argue: if the role of technology is to serve people, then why must the non-technical user expend so much time and effort getting technology to do its job? In 2001, HAL’s emotional abilities were intended to help address the problem of interacting with HAL. When the BBC reporter asks about HAL’s emotional abilities, crewman Dave Bowman responds,

“Well, he [HAL] acts like he has genuine emotions. Of course he’s programmed that way to make it easier for us to talk with him….”

The emphasis when the film was released in 1968 was that HAL’s emotional abilities would make things easier for people, leading naturally to a smoother interaction. Today, the tendency for people to interact socially with machines, even when the machine has no visible life-like face or audible voice, has been demonstrated via dozens of experiments (see, e.g., Reeves and Nass, 1996). The role of emotional skills is an essential part of social intelligence (Gardner, 1983) and the ability to perform a certain set of emotional skills has been argued to comprise a form of intelligence (Salovey and Mayer, 1990). An argument has even been made that such socalled emotional intelligence is more important for success in life than are the traditional mathematical and verbal capabilities that IQ tests attempt to measure (Goleman, 1995). A subset of these emotional skills: the ability to sense, recognize, and respond to human emotion, form the focus of the rest of this presentation: where does technology stand today with respect to giving machines these abilities?

Emotion Sensing & Recognition

Emotions in people consist of a constellation of regulatory and biasing mechanisms, operating throughout the body and brain, modulating just about everything a person does. Emotion can affect the way you walk, talk, type, gesture, compose a sentence, or otherwise communicate. Thus, to infer a person’s emotion, there are multiple signals you can sense and try to associate with an underlying affective state. Depending on which sensors are available (auditory, visual, textual, physiological, biochemical, etc.) one can look for different patterns of emotion’s influence. The most active areas for machine motion recognition have been in automating facial expression recognition, vocal inflection recognition, and reasoning about emotion given text input about goals and actions.

A fair bit of progress in this area since my 1997 treatment of the topic in Affective Computing. The research focus originally was on detecting six “basic” facial expressions (anger, sadness, happiness, disgust, surprise, fear) from still images, and then from video, with best results ranging for the latter from around 80-98% accuracy when the data was pre-segmented into one of the six categories and the lighting and position of the face were carefully controlled. More recent work (Bartlett et al, 1999; Cohn et al, 1999, and Donato et al, 1999) have focused less on a half dozen basic categories and more on  recognizing dozens of facial actions, specific muscle movements that combine to form a much larger vocabulary of expressions. Some of these methods are now performing comparably to human ability to recognize facial actions (Donato et al, 1999). Problems remain, however, in tracking faces that move in front of the camera, handling changes in lighting, and recognizing facial expressions while a person is speaking. In short, current technology is still far behind what people can recognize from one another’s faces.

Most pattern recognition researchers are familiar with variety of tools for representation of patterns –including discrete categories, fuzzy or probabilistic categories, and dimensioned spaces, to name a few that are particularly relevant to emotion representations. Emotion theorists do not agree upon a definition of emotion, but most of them fall into one of two camps in how they describe emotion either as basic discrete categories, e.g., fear, sadness, joy, etc., or as locations within a dimensioned space, the two foremost dimensions of which are usually termed “arousal” and “valence.” The arousal dimension tends to refer to the overall excitement or activation of the emotion, while the valence dimension tends to refer to how pleasing (positive) or displeasing (negative) the emotion is. Peter Lang and his colleagues, for example, have measured how people respond to hundreds of images in terms of the dimensions of arousal, valence, and dominance, recording physiological patterns that exhibit significant differences especially with respect to the arousal and valence dimensions (Lang, 1995).

A given emotion can of course be represented in multiple ways. For example, anger can be represented as a discrete category, defined by some collection of attributes, such as by facial actions that typify its expression, or by some bodily parameters that lie within a negative valence, high arousal portion of a dimensioned space. In general, facial expressions are good at communicating valence (positive, negative) while vocal inflection (especially pitch and loudness) is good at communicating arousal. Combinations of facial and vocal analysis tend to strengthen the inference of the underlying emotion. We know that HAL could detect emotions such as displeasure or distress from his lines such as,

“Look, Dave, I can see you’re really

upset about this. I honestly think you

ought to sit down calmly, take a stress

pill, and think things over.”

From the Clarke and Kubrick novel (written after the 1965 screenplay), we learn that HAL detected stress by listening to voice patterns. Vocal analysis of emotion continues to be an area of active inquiry, although progress since my 1997 treatment of this topic in Affective Computing has not been dramatic. Machine and human recognition of affect in speech still remains around the same as I described then, typically well below 100% recognition accuracy, usually hitting around 60-75% when given data from one of six to eight categories. Polzin’s thesis is perhaps the most recent work trying to recognize several categories of affect (Polzin, 2000) with our work providing one of the more recent efforts on stress recognition (Fernandez and Picard, 2000). In our work we built models of driver’s speech under mild to moderate stress conditions, comparing methods such as factorial hidden Markov models (HMM’s), hidden Markov decision trees, auto-regressive HMM’s, a mixture of HMM’s, Support Vector Machines, and a neural network. The mixture of HMM’s gave the best performance to date, although the results are very person dependent and the best results are still well below 100%. (See the references in these works for pointers to many more recent articles addressing vocal affect and stress recognition.)

Presumably, HAL could reason about emotion – knowing, for example, from inference about human value for the lives of ones colleagues, that Dave should be upset and stressed about the death of his crewmates. Although such reasoning does not always imply how somebody actually does feel, when combined with observations of behavior, such as Dave’s seriousness and increasing tension, HAL’s inference of Dave’s state should be strengthened. Machine reasoning about affect is one of the areas of artificial intelligence that has been explored the longest, and my review in Affective Computing of this area is still fairly up to date. In my opinion, the real breakthroughs that are needed to improve emotion recognition are not so much in reasoning about emotion, but are in perceiving accurate information with which to reason, and in detecting the affective tone of the context. The latter relates more to problems in common sense learning (and generalization of what one has learned) than to reasoning per se. Thus, context sensing, perception of what the situation is, perception of the emotional nuances present in a situation, and perception of how the people are responding are critical inputs to combine with an affective reasoning system. 

Although HAL apparently observed people through visual and vocal cues, most of today’s computers still rely upon physical keyboard/mouse input, where sensors might attend to not only what is typed or clicked, but how it is typed or clicked (speed, pressure, and other skin-surface cues.) Recent efforts toward building wearable computers also open up a lot of new affect sensing possibilities – especially through skin-surface sensors that detect muscle tension, skin conductivity, heart activity, temperature, and respiration. Progress in these areas includes new sensors such as IBM’s “emotion mouse” and a variety of tangible and wearable interfaces designed and built by the Affective Computing Group at MIT. Using pattern recognition of physiology, we have achieved recognition rates of 81% accuracy for a set of eight emotions in a person-dependent forced-choice pattern recognition scenario and rates of up to 96% accuracy in assessing level of stress in a subject-independent study of twelve Boston drivers.

Responding to Emotions

Computers are in their infancy with respect to recognizing emotion; however, suppose that, like HAL, they could recognize some of our expressions of emotion; how then should they respond? Although one might argue that the answer to this is more of a social science or psychology issue than an engineering one, it is still an important question for us to consider with respect to thinking through the potential capabilities of affective systems and how they might be constructed and used, for better or worse. In 2001, we saw that HAL’s response to recognizing stress was to suggest that the stressful person (Dave) sit down calmly and take a stress pill. How would you feel if your computer, after being the source of your irritation, told you to sit down and take a stress pill? Chances are there would be a wide range of reactions, some of which might include increasingly negative feelings.

One of the advantages of a system that can recognize affective expressions, especially those of pleasure or displeasure, is that it can try out different responses on a user, to see which are most pleasing. Indeed, a core property of most learning systems is the ability to sense positive or negative feedback – affective feedback – and incorporate this into the learning routine. Most dogs are better than computers when it comes to sensing this feedback.

Here is a scenario of how a computer tutor of the future might use recognition of affect to, perhaps, help you learn to play the piano. As you show interest in the topic and make rapid progress, it might provide optional interesting side avenues to explore. If you become distressed, perhaps because you are being pushed too far too fast, it might slow down and give encouraging suggestions, or revisit fundamentals. It might have the dual goal of maximizing learning and bringing pleasure but not pursue the latter goal 100 percent of the time, as some distress appears to be necessary for learning to occur. To be successful, the tutor would need to at least recognize and express affect. Ideally, it would also combine emotional intelligence (such as how and when to use empathy and how to adjust its teaching based on the student's affect) with other forms of knowledge - such as the subject matter and the best way to teach it.

The basic idea is that a system reflect that it has somehow understood the user’s emotion, even in a limited way – much like a dog might put its ears back and tails down if it sees its master is upset. Such a display of apparent empathy, even by a dog, can have a powerful impact toward alleviating the strong negative feeling of a person, in this case the dog’s master. The ability of a computer to not only detect its user’s emotion, but to influence it by choosing a careful response, is an important one, which raises many ethical and social issues. We address many of these in a forthcoming article (Picard and Klein, 2001), but it is important to raise this issue  here as well, so that designers of these systems can be aware of at least one potentially powerful way such technology may be used.

Above, I mentioned success we have had in detecting stress in drivers. The automobile environment is another place where the response of the system needs careful consideration. For example, if the system threatened the user’s privacy by reporting driving behavior to the insurance company, this would not be acceptable to most users. However, if the system processed data in real time, saved no identifying or potentially incriminating  information, and used the affective cues only to determine its own behavior – like routing an incoming cell phone call to voicemail during stressful attention-demanding driving situations, or adjusting its presentation of neighborhood or navigation information – then it might be considered of benefit to the consumer’s safety and peace of mind. Such considerations influence how we design recognition algorithms – for example, aiming for real time analysis with minimal storage of state information.

Respect for the user’s privacy and sense of control has influenced many of our design decisions. For example, we have built a pair of glasses that senses changes in the brow muscles, to detect furrowing of the brow. This wearable sensor, while initially seeming more awkward, can also be perceived as less intrusive than video. A camera pointed at one’s face is nice in that you don’t have to do anything but be visible; however, it may also record and extract information that you may not want to share – such as who you are and what you look like. In contrast, a small wearable sensor that just registers muscle tension presents the computer only with the furrowing information, while having the advantage that the user is in complete control of whether or not the sensor is allowed to operate. (It is easy to remove the glasses, or to detach the sensor from them without awareness of such to the system.) Items that are worn, that exist in the user’s personal space, tend to give a greater sense of empowerment to the user. In contrast, when the sensing is “in the walls” like HAL’s red eyeball, then the user may have information sensed without their awareness, as when HAL read the lips of Dave and Frank, a capability they did not know that he had.

Teaching Computers to Recognize and Express Emotion

Research in computer recognition and expression of emotion is in its infancy. Two of the current research efforts at the MIT Media Lab focus on recognition of facial expression and voice affect synthesis. These are not, of course, the only ways to recognize affective states; posture and physiological signs like gestures and increased breathing rate, for example, also provide valuable cues.  Computers, like people, can use cognitive reasoning -- a form of common sense -- to understand a person's goals and predict his or her affective state when they are disrupted. For example, HAL may predict that because "I killed Dave's colleagues and won't let him back on the ship, Dave will be upset." If prediction and observations agree, the computer is likely to strengthen its belief in that line of reasoning. If they disagree, it will see it as an interesting (perhaps even puzzling) event: "Most people would be enraged by all this, but Dave doesn't look very upset. He is great at concealing his emotions. Or maybe he knows something important I don't know?" 

One way to recognize an expression is to record facial movements during a short video sequence, digitize the sequence, then apply the tools of pattern recognition. Recognition from a moving sequence is generally more accurate than recognition from a still image. If, for example, a person's "neutral" expression is a pout, only deviations from the pout (captured by video as movement) will be significant for recognizing affect.  Using this method requires a video camera, a digitizer, and a computer running video-analysis and pattern-recognition algorithms. Pattern recognition can utilize a variety of techniques -- such as analyzing individual muscle actuation parameters or (more coarsely) characterizing an overall facial-movement pattern. In a test involving eight people, recognition rates were as high as 98 percent for four emotions. Studies are underway to determine how the recognition rate changes when there are more experimental subjects. As yet, this method of recognition doesn't work in real time; it takes a few seconds to recognize each expression. However, advances in hardware and pattern recognition should make recognition essentially instantaneous in the near future -- at least for familiar expressions. 

Although facial features are one of the most visible signs of underlying emotional states, they are also easy to control in order to hide emotion. Having a good "poker-face" that reveals none of your emotions is valuable, not only for playing cards, but also in the cutthroat worlds of business and law. The social-display rules of emotion specific to our culture are impressed upon us all as we grow up. I have seen a student who was undergoing great personal pain resist crying, while his eyes twitched unnaturally to hold back his tears. He was taught at an early age never to show emotion in public. Nonetheless, the healthy human body seems unable to suppress emotion entirely. He might not cry, but his eyes may twitch. She might not sound nervous, but she may, literally, have cold feet.  Emotional expression is not, clearly, limited to facial movement. Vocal intonation is the other most common way to communicate strong feelings. Several features of speech are modulated by emotion; we divide them into such categories as voice quality (e.g., breathy or resonant, depending on individual vocal tract and breathing), timing of utterance (e.g., speaking faster for fear, slower for disgust), and utterance pitch contour (e.g., showing greater frequency range and abruptness for anger, a smaller range and downward contours for sadness), As these features vary, the emotional expression of the voice changes. The research problem of precisely how to vary these features to synthesize realistic intonation so far remains unanswered. 

The inverse problem - intonation analysis, or recognizing how something is said, is also quite difficult. Research to date has limited the speaker to a small number of sentences, and the results are still closely dependent on the particular words spoken. A method of precisely separating what is said from how it is said has not yet been developed. You will note that no one method - whether recognition of facial expression or of voice intonation - is likely to produce reliable recognition of emotion. In this sense, affect recognition is similar to other recognition problems like speech recognition and lipreading. It is probable that a personalized combination taking into account both perceptual cues (say from vision and audition) and cognitive cues (such as HAL's reasoning about how Dave would respond) is most likely to succeed. These cues will undoubtedly work best when considered in context: is it a poker game, where bluffing is the norm, or a marriage proposal, where sincerity is expected?

Toward Truly Personal Computers

Does HAL have affect-recognition abilities beyond facial expression, vocal intonation, and common-sense reasoning about some typical emotion-inducing scenarios? We don't, of course, really know. Although Dave Bowman carefully controls his facial expression in the scene where HAL won't let him back on board, his anger may have been betrayed by some other body response -- perhaps an increase in body temperature or breathing rate. Sensors that can detect these two forms of physiological expression, among others, currently exist. Affective computers in the future may have other perceptual sensors that are not limited to human senses. For instance, a humidity detector might reveal that someone is anxious, even before she or he breaks out in a full sweat. 

Consider, for example, the fact that people who use computers touch the machine a lot. Whether through a mouse, keys, joy stick, or touch screen, many people have more physical contact with computers than they do with other people. Moreover, you can now wear computers -- in your shoes, shirt pocket, or belt, for example. Wearable computers, especially when they become as common as underwear, will have unusual opportunities to get to know you in a variety of situations. They could have access to your muscular tension, heart rate, temperature, and so on. Instead of being restricted to perceiving only your visible and vocal forms of affect expression, they could get to know you intimately - or as well as you will permit them to. At this point, they will also, like underwear, probably cease to be shared and will become truly personal computers. 

Suppose you have too much stress in your life and your doctor suggests that you learn to relax more. Your wearable affective computer could help you learn what events cause you stress and figure out ways to reduce it. While you are engrossed in playing with the kids, your affective wearable might whisper in your ear, "see how relaxed you are now." A little feedback device you could turn on or off might not only help reduce stress-related disorders, it might also assist in gathering important medical research data or helping patients in recovery. The key to the wearable computer is its constant presence; it is not limited to gathering data in the lab or doctor's office, but can get to know your range of responses during the daily routine. 

Affective information could also be communicated in unconventional ways. Imagine that your wearable computer could detect the lilt in your walk as you leave the office and broadcast it to your spouse -- encoded, of course (lest a salesperson learn of your happiness and take this auspicious opportunity to telephone you). The result would be a sort of "mood ring" that alerts you to your spouse's affective state -- one that is more accurate than the dime-store temperature sensors once advertised on late-night television.  Applications of affective recognition could extend to entertainment as well; for example, interactive games might detect your level of fear and give bonus points for courage. When we measured the responses taken of a student playing the computer game DOOM in our lab, we expected the electromyogram of jaw clenching to peak during high-action events -- such as when a new deadly enemy starts an attack. However, the biggest peak -- and it was significantly higher than the others recorded -- occurred when the student had trouble configuring the software!

What if software companies could obtain similar affective information about people interacting with their products? Unlike questionnaires, an affect-sensing computer could identify the parts of the software that provoke the greatest annoyances and those that produce the greatest pleasure. Not only would the timing of affective responses be easier to relate to specific causes, but they would tend to capture product qualities that are hard to put into words. All makers of environments -- architects, automobile manufacturers, software designers, decorators, hotel managers -- benefit from learning how people feel when they are in their spaces. 

Computers coupled with suitable sensors and pattern-recognition algorithms should soon be able to recognize the basic affective states of a willing individual in a typical context. The emphasis on willing participant here is important. Measurements of affective states obtained in an underhanded manner are not likely to be accurate. People who want to deceive such systems will probably succeed. One fellow, for example, managed to fool a polygraph by putting a thumbtack in his shoe under his big toe; he stepped on it every time he was questioned in a particular way. Affective information will be most accurate, and useful, when it is willingly communicated, presumably for the mutual benefit of everyone involved.

Emotions with Reason

We've seen that HAL possesses abilities for expressing and recognizing emotion and noted some of the ways we are giving today's computers these abilities. But what about creating computers that actually have emotion? What could that possibly mean? This question is partially one of philosophy and goes beyond the scope of this chapter. But it also relates to the structure of the human brain and touches on a paradox about the role of emotions and reason.

Perhaps the simplest description of the human brain is Paul MacLean's triune brain,which distinguishes three regions: the neocortex, the limbic system, and the reptilian brain (see figure 13.5). Although it is greatly oversimplified, this description has influenced how people think about brain functions. For example, many have assumed that the physically highest level of the brain, the neocortex, dominates the other, lower levels. However, this assumption is contradicted by evidence that the physically lower limbic system can effectively hijack the brain; that is, emotions can overtake so-called higher mental functions when they need to. The limbic system - the primary seat of emotion, attention, and memory - contains such structures as the hypothalamus, hippocampus, and amygdala. It helps determine valence (e.g., whether you feel positive or negative toward something) and salience (e.g., what gets your attention); it also contributes to human flexibility, unpredictability, and creative behavior. It has vast interconnections with the neocortex, so that brain functions are not either purely limbic or purely cortical but a mixture of both.

We have all, of course, seen emotions overwhelm reason (at least in others), which is one reason why the word emotional has negative connotations. (For example, people who panic out of fear may cause more harm to themselves than if they had "kept a cool head" and made rational decisions.) Nonetheless, it is clearly beneficial for our survival that fear can hijack our brain and cause us to jump out of the way of a rapidly approaching object before we can consciously perceive and analyze that a bus is about to hit us.

These kinds of emotions, which seem to be hard-wired or innate, are sometimes called primary emotions. They include responses such as the fear example above and involuntary reactions to surprise. Other emotions, the secondary emotions, appear to develop as we mature. They connect cognitive events with lower-level physiological responses and occur as a result of joint neocortical and limbic activity. Such emotions play an especially important role in decision making, even in decision making that appears to be purely rational.

Findings on the importance of emotions for rational decision making seem paradoxical. They are based on a remarkable story told by A.R. Damasio about the patient "Elliot." Elliot, and patients like him, have a particular kind of brain damage that affects a circuit between the prefrontal cortex and the amygdala, a communication channel between the neocortex and limbic system that appears to be essential for secondary emotions. At first glance, Elliot appears to be like Star Trek's Spock - emotionally unexpressive, unusually rational. One might think that Elliot would therefore be superb at making rational decisions. However, unlike the fictitious half-human Spock, Elliot's lack of emotions severely impairs his decision-making ability and causes tragedies in his business and personal life.

Although Elliot's IQ and cognitive abilities are all normal or above average, when confronted with a simple decision, such as when to schedule an appointment, he disappears into an endless rational search of "well, this time might be good" or "maybe I will have to be on that side of town so this time would be better," and on and on. Although a certain amount of indecisiveness is normal, Elliot apparently doesn't experience the usual feelings of embarrassment when someone stares at him for taking so long to make up his mind. Nor is the indecision accompanied by the healthy limbic responses that normally associate positive or negative feelings with certain decisions, responses that help us limit a search by nudging us away from possibilities with bad associations. Instead, Elliot tends to search an astronomical space of rational possibilities and seems unable to learn the links between dangerous choices and bad feelings; so he repeatedly makes bad decisions. Elliot's lack of emotions severely handicaps his ability to function rationally and intelligently.

In other words, not only does too much emotion wreak havoc on reasoning, but also, paradoxically, too little emotion wreaks havoc on reasoning. Apparently, a balance is needed: not too much emotion, not too little emotion. Computers, except for HAL, do not have enough emotion. Artificial intelligence systems to date are not unlike Elliot: they have above-average knowledge (usually consisting of a huge set of rules) of some area of expertise, but are disastrous at making decisions. They are too rational; they cannot associate judgments of value and salience with their decisions. Little has been done to imitate these judgments, which are essentially products of the limbic system, in computers.

Computers That Will Have Emotions

So far, our discussion has focused on computers that can recognize, express, and predict emotions. These abilities alone could create the impression that a computer has emotions even when it really doesn't have them. But what does it mean for a computer to actually have emotions? Consider the following exchange about the Discovery mission, in which the BBC reporter asks Dave about HAL.

Reporter: One gets the sense that he is capable of emotional responses. When I asked him about his abilities I sensed a sort of pride...

Bowman: Well, he acts like he has genuine emotions. Of course he's programmed that way to make it easier for us to talk with him. But whether or not he has real feelings is something I do not think anyone can truly answer.

Bowman's answer parries a difficult question that is more in the domain of philosophy than in that of science: Can computers have emotions? The answer, of course, depends on the definition of emotions, which theorists still argue about; so at present there is no good answer. This question parallels the question "Can computers have consciousness?, where consciousness is also difficult to define. In the novel, Clarke endows HAL with self-consciousness, a necessary prerequisite for certain kinds of emotions, such as shame or guilt (see chapter 16).

Let's consider two scenarios in which a computer might be seen as having emotions. In the first, the emphasis will be on primary emotions (the more innate, hard-wired kind). In the second, the emphasis will be on secondary emotions, which typically involve cognitive evaluation.

·         Scenario 1.

A robot used to explore a new planet is given some basic emotions in order to improve its chances of survival. In its usual, nonemotional state, it peruses the planet, gathering data, analyzing it, and communicating results back to earth. At one point, however, the robot senses that it has been physically damaged and changes to a new internal state, perhaps named "fear." In this state it behaves differently, quickly reallocating its resources to drive its perceptual sensors (e.g., its "eyes" might open wider) and provide extra power to its motor system to let it move rapidly away from the source of danger. However, as long as the robot remains in a state of fear, it has insufficient resources to perform its data analysis (like human beings who can't concentrate on a task when they are in danger). The robot's communication priorities, ceasing to be scientific, put out a call for help. This so-called fear state lasts until the threat passes, then decays gradually over time, returning the robot to a state of no emotion in which it resumes its scientific goals.

·         Scenario 2.

A computer is learning to be a smart personal assistant, to aid you in scheduling meetings and retrieving important information. It has two ways of getting feedback. In the first, you give it feedback directly by selecting preferences (essentially programming it). Alternatively, it watches how you respond to its assistance and programs itself. It enters a state called "feel good" when (1) you feel good or express pleasure at its performance, and (2) when you succeed at a task more efficiently and accurately than usual. It might also have a corresponding "feel bad" state for the reverse situation, as well as a neutral "no emotion" state, a "feeling curious" state, and an "I'm puzzled" state. When the system has been in its feel-good state for several days, it becomes more curious trying out new ways to help you and taking more risks. When it lingers in a feel-bad state, it allocates more resources to trying to understand your wishes. When you make a complicated set of demands, it weighs the feel-good and feel-bad associations and tries to choose an action that satisfies goals (1) and (2). Unlike a fixed computer program, it doesn't expect you to behave consistently nor require precise rules telling it how you want it to behave. It copes with your human fickleness by aiming for a dynamic balance, recognizing that you will often not show pleasure when it performs well and will sometimes complain or show approval inconsistently. At such times, depending on how calm or agitated you are (measured from your norm), it either asks for clarification or makes a note to come back later and try to understand the situation - perhaps when you are not so agitated. It's use of emotions helps it make flexible, creative, and intelligent decisions.

In both scenarios, the computer's emotions are labels for states that may not exactly match the analogous human feelings, but that initiate behavior we would expect someone in that state to display.  In both cases, giving the computer emotions serves some ostensibly greater human good, such as survival - save humans the cost of building and dispatching another robot - or performance - save humans time, money, and frustration. In neither case are emotions provided to dignify the machine by creating it in the image of a human being. Doing the latter would raise issues of computer slavery and computer rights that are many decades down the road! In any case, discussing them would take us far from the aims of this chapter.

Where Do We Go from Here?

“Sometimes the truth of a thing is not so much in the think of it, but in the feel of it. “

-- Stanley Kubrick, 2001: Filming the Future

HAL's emotions are no longer surprising, given what we know now about the important role emotions play in rational and creative decision making, in natural friendly communication, and even in art appreciation. No longer should we think of emotion as a luxury added to HAL's character just for emotional appeal. Instead, we can see HAL as the prototype of a truly affective computer -- one whose abilities to recognize and express emotions are essential for communicating as well as for user-friendly responses. The ability to experience emotions, or at least states that seem to parallel human emotional states, appears to be critical to flexible and intelligent computer decision making. There is, however, danger in all of this, a danger that machines will have emotions, but not sufficient intelligence to use them properly. Nonetheless, the problem of HAL's life-threatening behavior in 2001 is probably not as imminent as our need for emotional intelligence in machines.

Do we, then, want to build an intelligent, friendly, flexible machine like HAL? Yes. Are emotions necessary to such a machine? Apparently, yes. In fact, lack of emotions may be a key reason why artificial intelligence has failed at this task to date. But there is another question -- and I don't know the answer: are people ready for affective computers?

Conclusion

This presentation has highlighted some of the affective abilities that the HAL 9000 computer had, with emphasis on the sensing and recognition of emotion, and on responses to human emotion. HAL also had many other affective abilities, such as the ability to express emotion (through the emotive human voice of actor Douglas Rain), an ability that speech synthesizers still cannot emulate. Additionally, HAL acted as if he “had” emotions, especially fear and paranoia, as expressed not only through his behavior but also his famous words to Dave, “I’m afraid, Dave, …, I’m afraid, …” as he was being disconnected.

Although work in affective computing includes all of these aspects of emotion, our work at the MIT Media Lab has focused on giving machines a subset of affective abilities especially related to improving interaction with people, improving the machine’s skills related to socialemotional intelligence. With this focus, we have tried to steer away from some of the hard AI problems of“understanding and experiencing” emotion, reposing the issues as problems in sensing, signals analysis, and pattern recognition. However, this is only part of the frontier where work needs to be done – researchers are needed from many areas, including engineering, social sciences, psychology, and cognitive science, to work in collaboration to build systems that are truly (following the words of Bowman) easier for us to interact with. In particular, careful regard must be made in the design of these systems so that they do not further irritate, annoy, or bring about unwanted stress to their users; suchconsequences would be antithetical to the goals of  affective computing, which involve honoring the emotions of people above any such abilities that might  be given to machines.

BIBLIOGRAPHY

1. W. Ark, D. C. Dryer, and D. J. Lu (1999), “The

Emotion Mouse,” Proc. of HCI International ’99,

Munich Germany, August 1999.

2. M. Bartlett, J. C. Hager, P. Ekman, and T. J.

Sejnowski, ‘‘Measuring facial expressions by

computer image analysis,’’ Psychophysiology, vol.

36, pp. 253--263, 1999.

3. J. F. Cohn, A. J. Zlochower, J. Lien, and T. Kanade,

``Automated face analysis by feature point tracking

has high concurrent validity with manual FACS

coding,’’ Psychophysiology, vol. 36, pp. 35-43, 1999.

4. A. R. Damasio, Descartes’ Error: Emotion, Reason,

and the Human Brain. Gosset/Putnam Press, New

York, NY 1994.

5. D. Dennett, “Did HAL commit murder? ” In D. G.

Stork, editor, HAL’s Legacy: 2001’s Computer as

Dream and Reality, The MIT Press, Cambridge, MA

1997.