An Autonomous Agent with an Evolved, Affect-based Decision System

Paul G. Joseph1,Haim Levkowitz1

Department of Computer Science, University of Massachusetts, Lowell, MA, USA

Abstract - We investigate the modeling of affect with the goal

to create an agent that behaves "emotionally" closer to humans.
Our previous work showed evidence that modeling the fundamental emotions of the mammalian brain offered the basis of a
flexible decision system for an autonomous agent. Earlier work
hard-coded the parameters used to model affect; now, we let the
parameters evolve instead. Agents learn dynamically from interactions with their environment and use this learning in a
speedy, general-purpose decision system. The agent's interactions with its environment and consequent impact on its wellbeing promise an affect-based approach to semantics. We subject an agent to a Turing-like test for affect in which it has to
pass a hostile predator to reach its goal. We compare our approach with others; the strict basis of our approach is mainstream neuroscience. Agents independently assign affect valuations that provide a basis for semantic understandings.

standing of how the mammalian brain handles affect and makes

affect-based decisions, thus advancing research in cognitive
and neuroscience. In addition, modeling emotions in a single
agent and in groups of agents will be useful to researchers in
other fields including ethology and social sciences..
Previous work [18, 19, 20] identified the fundamental emotions as understood by the neuroscientist Dr. Jaak Panksepp and
created a decision system based on these fundamental emotions. The approach showed promisecreating a fast decision
system that seemed to be flexible and useful to an autonomous,
intelligent agent. However, the parameters of this system were
hard coded with no real justification for their values other
than that trial and error showed that they worked. In this paper,
we allow the autonomous agent to evolve its parameters based
on its interaction with its environment. We then study the resulting system with a Turing-like test for affect.

Keywords: autonomous agent, affect, decision system, mammalian brain, semantics

2. Review of previous work

1. Introduction
This paper describes continued work in a new kind of decision
systemone based on emotions or affect and whose implementation is directly modeled on and constrained by those systems in the mammalian brain that are presently thought to handle affect. The underlying premise is that humans (and all
mammals) emotionally evaluate actions in terms of how these
will influence their well-being and that consequently, being
able to reproduce these affect-based evaluations in autonomous
agents will provide for an additional method of making and/or
supporting decisions. We intentionally constrain ourselves to
the underlying empirical evidence from mainstream neuroscience and adopt the philosophical stance that when presented
with two approaches, we limit ourselves to that which has a
sounder basis in the underlying biological empirical evidence.
The approach offers the promise of advancing understanding
and capability in three areas: autonomous decision-making,
semantics, and affect modeling. Using a search heuristic in
which the external environment modifies internal affect with
the aim of maximizing internal positive affect provides the basis of an autonomous decision engine, which, because of its explicitly non-cognitive bias, is fast enough for use in real-time
applications. The affect generated in the agent by interacting
with objects in its environment in terms of the impact of these
on the agents well-being, promises an affect-based approach to
semantics. The modeling of affect will also advance under-

Previous work [18, 19, 20], proposed that a major part of decision-making involves emotion-based evaluations of objects
and ideas. We proposed an architecture that follows closely
findings in neuroscience. Our goal is to articulate and validate
the concept that non-conscious software agents can develop a
sense of value of the objects with which they interact. This
evaluation is in terms of how these objects influence the agents
well-being, i.e., its ability to function effectively. A key measure of well-being is the agents affective state.
Such an approach promises the ability to create systems
that can interact with real-world objects with some understanding of what these objects mean that is over and above analytical
understandings typical of standard computing. The promise is
that we can then use this extended understanding to demonstrate behavior and adaptability typically seen in humans and
animals. These behaviors include rapid decision capabilities
with autonomously driven goal setting. Such an affect-based
approach also offers promise in providing a new approach to
semantics and other problems, including, eventually, even the
ability to make analogies and be creative [19].
The basis of our earlier work [18] was the neuroscience
model of affect by Dr. Jaak Panksepp. Panksepp [30] postulates the existence of seven core (fundamental) emotional systems that map directly to seven physical neuronal systems that
he had identified in the mammalian brain which generate
well-organized emotion sequences that can be evoked by localized electrical stimulation of the brain. He called these systems the SEEKING, RAGE, FEAR, PANIC, LUST, CARE,
and PLAY systems, with the latter three qualified as serving
socioemotional needs. Panksepp distinguishes his emotions
from basic drives such as for example hunger. Per Pank-

sepp, basic drives form the physical/bodily substrate that forms

the basis of emotions.
Following from the theory of core/fundamental emotions,
any emotion is some combination of the core or fundamental
emotions. For example, the fear felt by a teenager on his first
date would be viewed as some combination of (perhaps)
PANIC, CARE, PLAY, SEEK, and LUST; resentment or indignation as combinations of RAGE, CARE, SEEK. The
SEEK response in particular is associated with a large release
of dopamine and a sense of well-being and is the key driver of
emotions like joy, satisfaction, contentment, enjoyment, etc., variations of the theme of gratification of the
SEEKING system and the attainment and benefit of the goal
sought.
Initial work implemented the Panksepp model in the PacMan game [11] with Pac-Man coded using Python and with a
heuristic to make it behave as a utility reflex agent [37] that
maximizes its overall wellbeing. The heuristic was a simple
dynamical systems model that used the Panksepp variables,
with Pac-Man conveniently modeling prey and the Ghost, a
predator. The results were promising and justified further research.
We then did a detailed review of other approaches to emotions [2, 3, 15, 23] and meta-analyses on affect processing in the
mammalian brain [21, 24, 35]. Greene [15] describes his theory
that the human brain has two modes of responding to situations.
One, a set of efficient automatic responses driven by emotions,
the other a manual mode used in response to non-standard situations that involves significant cognitive evaluation. LeDoux
[23] posits that the former typically operates in timeframes of
about 10 ms, while the latter in timeframes closer to 500 ms.
Consequently, we use the 10 ms number as a heuristic: our explicitly non-cognitive model should takes no more than about
10 ms to compute a decision on an ordinary single CPU laptop.
Eventually, the brain evaluates decisions from this layer in conjunction with those from an independent analytical layer in the
ventromedial-prefrontal-cortex through a process not fully understood [15].
In our review of the mainstream neuroscience literature on
emotions, we found no support for appraisal theories of emotions. The basis of cognitive appraisal theory is the JamesLange theory of emotions. It holds that we extract emotions
primarily from our cognitive appraisals of events, and that emotions follow from the actions and feelings that result from these
appraisals. Mainstream neuroscience no longer considers the
James-Lange theory and cognitive appraisal theory to be valid
[21, 35]. Todays mainstream neuroscience models are premised on the understanding that the most important contributions
to affect come from the autonomic nervous system and that affect is largely facilitated by the limbic system [21, 24, 35].
Panksepps model is explicitly non-cognitive.
Based on our review and meta-analyses we decided to focus
initially on that subset of Panksepps emotions that appear to
have general acceptance in mainstream neuroscience. Specifically, these are SEEKING, FEAR, and RAGE. We did not implement PANIC as a fundamental emotion as it appears that the
meta-analyses view it more as based on FEAR, rather than an
emotion in its own right. We also did not implement any of

Panksepps more sophisticated, special purpose, socioemotional systems, i.e., PLAY, LUST, and CARE.
Every module of the brain is made up of neurons whose activations are known to usually activate exponentially (see for example Armony et al. 1997, Greenfield, S. 2000). This motivated us to use exponential relationships in this initial attempt
at modeling the different neural systems. These equations are
tentative; place-holders until research from the neurosciences
provide us with the basis for more accurate relationships.
Nonetheless, it appears that even with these heuristic approximations, a useful decision system with seemingly realistic behavior is possible.
As a first approximation, we based the equation we used to
model FEAR, on the distance to the threat. If the Manhattan
distance to the threat is greater than some threshold value, then
the prey takes no evasive action (shows no FEAR). Let this
threshold distance be X F . However once this threshold is
crossed, then the prey exhibits FEAR behavior (avoidance), and
we need a function to model this FEAR. Following from our
earlier discussion on neurons activating exponentially, we
choose an exponential function as an initial, approximate model
for FEARPac-Mans FEAR increases from zero at distance
[greater than or equal to] X F from a ghost, to infinity at a distance of zero from the ghost, as follows:

FEAR = e

XF
)
X

for X

< X F , otherwise FEAR = 0 (1)

Likewise, we use a similar approach for SEEKING. Starvation studies (Owen and Hanson 2004) indicate that humans
experience hunger pangs a few hours after a meal. These continue for the first three days without food, at which point, the
body switches to start consuming body fat. Once this happens,
the average human feels no further feelings of hunger for approximately five weeks. After this, with body fat consumed,
the system starts consuming muscle. The person then experiences extreme hunger that signals that they must find food or
else, their body will start consuming body muscle and brain
matter to survive. Rather than model this complex behavior in
detail, we again chose as a first approximation an approach
similar to that used with the FEAR dimension, except time
based. If the prey last consumed food at time TS , then the

SEEKING value for Pac-Man goes up exponentially for values of time T greater than TS , until T TS reaches another
value TD , at which time the prey dies from starvation. Like-

SEEKING varies exponentially in

proportion X D of the inverse distance X to the food, as:
wise, we assumed that

SEEKING = e

TD
)
T D ( T T S ) f

*e

XD
)
X

(2)

As before, we used Panksepps description of the RAGE

system to model RAGE as FEAR multiplied by a function for
entrapment. In the absence of more explicit rules, we simply
write RAGE as a function of FEAR and entrapment, with entrapment being a simple summation of the minimum distance of

the prey to boundaries in the x and y directions ( B X , BY respectively), and coming into play only when this value is below
some threshold value X E , i.e.:

RAGE = FEAR * entrapment

entrapment = e

XE
(min( B X ) + min( BY ))

(3)
for < X E

sponding

standard

deviation

and

mean

xi

for

TD , X F , X D , X E calculated from the population of surviving

autonomous agents. As is typically the case for mutation (Poli,
Langdon, and McPhee 2008.), we used = 0.2 . We also enforced a standard requirement that:
xi < 0 xi := 0 where, typically, 0 = 1 .

The preys affective state is hence contained in the dimensions of FEAR, SEEKING, RAGE and time, i.e., its affective
evaluation of the predator and its food, are represented by the
values of RAGE, FEAR, and SEEKING that result from its interacting with them and its own need for survival and wellbeing. At any given time, these values represent the preys semantic understanding of the predator. The overall affective
state of the prey at any given time is the net of the positive affect of SEEKING and the negative affects of FEAR and
RAGE:
PREYoverall _ affect = SEEKING FEAR RAGE (4)
This research (Joseph and Levkowitz 2012a) made a convincing case that adaptive behavior was possible with just the
simpler three emotions. Once we understand these better, we
plan to incorporate the more complex emotions not presently
considered. Hence, we view this model only as the start of a
program that will grow to include the more sophisticated emotions as knowledge of these become more available.
In this earlier work though, the values of TD , X F , X D , X E in
the equations were hard-coded. Since the results with this
hard coding were promising, we felt justified in going on to the
next stage and evolving these parameter values.
Figure 1: Setup used to evolve agent parameters

3. Evolving the parameters of the equation

The goal of Evolutionary Programming (EP) is to model evolution with learning. Given our underlying philosophy of constraining ourselves to empirically evidenced biology based approaches, EP appeared to be the natural choice to use over other
rational optimization methods in order to determine optimal affect values with which to replace the simple hard-coded values
used in our previous work. There are several approaches to
evolutionary programming (see for example [34]) and we used
the approach known as mutation. Since the focus of this research was not evolutionary programming, we simply used a
standard approach known as meta-EP, which uses selfadaptation of the parameters with a real-valued representation.
Mutation changes a chromosome represented as
< x1,..., xn , 1 ,..., n > into < x1,..., xn , 1,..., n > where:

i = i (1 + .N (0,1))
xi = xi + i.N (0,1)

(5)

and where N (0,1) is the result of a random selection from a

Gaussian distribution with a zero mean and a standard deviation
of 1. In our case, each instance of the autonomous agent was
treated as a chromosome, with each of the affect parameters
TD , X F , X D , X E corresponding to the xi , and the corre-

The general concept was to start with a certain number of

autonomous agents (prey) and predators. When a predator
killed an instance of prey, we regenerated a replacement
autonomous agent according to the mutation scheme above.
We tried various configurations of environment and numbers of
autonomous agents and predators, starting with some initial
seed values for the affect parameters. Figure 1 shows one
such typical environmentthe yellow dots are the autonomous
agents (prey), the other colors are the predators.
Two well-known issues that can arise in this type of actionselection problem are dithering and choices that take the
agent into local minima. In dithering, the prey moves back and
forth between the same two positions, and in local minima,
prey become paralyzed even if the predator is no where near.
We used two approaches to minimize either possibility. First,
in this version of the model, the predator has a degree of randomness built into it each time it chooses the direction of the
next step to take. Second, when presented with two equally optimal alternatives, the prey randomly chooses between either.
Together these two approaches seemed sufficient to guard
against both dithering and local minima. Further, none of the
tracks used by the prey for any given run ever fully matched
that from any other run. In future work we plan to provide the
prey with the ability to actively target and hunt down the prey
with no random behavior.

Results were as expected from standard concepts of evolutionary programming: affect parameters changed and evolved
depending on the number of predators and prey, different space
and food configurations of the environment, and with differing
degrees of rates of convergence and stability, and depending on
the population size. In short, as expected, we learned nothing
new about evolution per se. Rather, we found that the agents
were able to use this scheme to adapt to their environment, as
was to be expected. The basic finding was that EP is a useful
technique to use, allowing autonomous agents to learn and
adapt to their environment. We found that average computing
time on a low end 2.3GHz Intel CPU for 1000 runs of the
model was 7.08 seconds, in line with our LeDoux based heuristic of 10 ms., i.e., the model is fast enough for use in real-time
applications.

fect state so disposes it. In other words, a prey is observed to

move towards the predator if the preys SEEK value was high
enough to override its FEAR and RAGE, and there was food in
the vicinity of the predator.

140

120
FEAR Factor

Affect Value

100

80

SEEK Factor

60
FOOD Factor

Figure 3: Setup for a Turing test for affect

40

RAGE Factor

20

0
1

2001

4001

6001

8001

10001

12001

14001

16001

18001

20001

22001

24001

26001

28001

30001

Iteration Number

Figure 2: Stability of evolved affect parameters

Figure 2 shows typical evolution of the parameters, in this case,
for the example in Figure 1. Here, initial values chosen based
on the results of previous EP experiments showed stability. As
shown, parameters did not significantly change after about
30,000 iterations, i.e., the system had stabilized.

4. A Turing-like test for affect

The well-known Turing test [41] opens with the words: "I
propose to consider the question, 'Can machines think?'" Turing
goes on to say that, "thinking" is difficult to define and so
chooses to "replace the question by another, which is closely
related to it and is expressed in relatively unambiguous words.
His question was, "Are there imaginable digital computers
which would do well in the imitation game? Turing felt that it
was possible to answer this question with an empirical testthe
now famous Turing Test.
Likewise, the question for our purposes is not whether machines can feel emotions but rather, whether they behave as if
they feel emotions and are able to use these behaviors to survive. The mechanistic approach presented here via the equations implies that consciousness or subjective feels are not
required to leverage the ability of emotions to form the basis of
a decision system such as the one described above. Further,
following Turings approach of comparing empirical behavior
of the system to real life behavior, in theory the model should
allow for non-intuitive behavior observed in real life by ethologiststhat of a prey animal moving towards a predator if its af-

Figure 3 shows a setup we used to test the ability of our affective systems model to demonstrate this kind of behavior. We
programmed a predator (blue) to march only horizontally,
choosing at random whether to go to the left or to the right, and
thereby creating a guarded boundary. To pass through this
guarded boundary, the prey must expose itself to considerable
danger. We programmed the prey (yellow) with the emotional
system. All the food is on the side of the predator away from
the prey and the prey has to cross the predators line of marching to get to the food.
Initially, we see that the prey stays as far away from the
predator as per its value of FEAR. However, as the time since
it has last eaten increases, its SEEK value goes up and it begins
to move to the food, despite the presence of the predator. As
the predator comes closer to the food, the SEEK value also
goes up and directs it towards the food. At some point the
SEEK overrides the FEAR and the prey moves towards and
crosses the line of action of the predator. The position of the
second piece of food is in the corner: SEEK must exceed
RAGE (a function of closeness to enclosing walls) before the
prey will consume this food.
We found that the values of the parameters as evolved in
the environment of (say) Figure 1 were far from suitable for
such a task. The prey was never able to summon up enough of
SEEK to overcome its FEAR. We had to evolve these values
specifically for the Turing test environment before we saw the
expected outcome of the prey overriding its FEAR, making it to
its food, and thereby surviving. The evolved value of FEAR
for this more aggressive situation where Pac-Man had to pass
the ghost was about 25 times less than in the previous case.
RAGE (a measure of the prey being trapped by the boundaries)
was about five times smaller, signifying that Pac-Man had
evolved more aggressive behavior and so less likely to not approach Pac-Man or to stay away from the boundaries of its environment. Likewise, the evolved values of the prey indicated

NET AFFECT
10000
NET AFFECT (affect units)

as much as an order of magnitude reduction in its SEEK factor,

and a five fold reduction in its FOOD factor, both reductions
indicating a hardier profile, with reduced dependence on food.
This is as expectedparameter values evolved in one particular environment must be different from those evolved in a
significantly different (more hostile) environment. The conclusion was that the evolutionary process did work as expected;
the model could evolve to exhibit the wide range of behavior
found in nature: a very fearful prey, or a hardier, less fearful
prey, depending on the environment it evolved in.
The predator uses a random algorithm to determine the direction (left or right) of its next step. Consequently, while for any
given iteration number, the specific values from the model
across many thousands of runs are different, the general pattern
of affect is the same for each run.
For the initial part of the run the prey remained where FEAR
and RAGE were lowest. As time since eating increased, SEEK
increased; the prey began to explore wider regions of its domain despite increases in both RAGE and FEAR. Finally, as
time (SEEK) increased, the total of SEEK exceeded the sum of
RAGE and FEAR and the prey crossed the boundary patrolled by the predator, and despite increasing RAGE, approached the boundary, and ate the food. Figure 4 shows this
switch in NET affect for one Turing test in which there were
at least 10 reversals of affect (and hence direction) before PacMan crossed the predator.
Heider and Simmel (1944) studied the reactions of human
observers watching animations of moving triangles and
spheres. These observers invariably projected life like behavior
onto these abstract moving objects, concocting elaborate stories, including romantic interactions, between triangles and
squares. Consequently, rather than rely on subjective interpretations by human observers to judge the realism of the predator/prey behavior we chose to determine this with more objective criteria, that is available for this kind of a test
specifically, whether Pac-Man demonstrated behaviors typically observed in predator-prey interactions (see [8] for concept). Specifically, these were 1) the actual survival rate, i.e.,
the measure of the overall success of the prey behavior 2) retreat/advance in the face of the prey 3) lateral avoidance
(dodging) behavior to get past the prey and 4) freezing behaviors when movement by the prey would put it in the immediate path of the predator (not the same as the paralysis in the
absence of the predator described earlier).
On average, over 1,000 runs, we found that the success rate
of the evolved prey in getting past the predator was above 80%.
The prey would move retreat/advance with respect to the predator a minimum of three times per run, coming closer to the
predator in each subsequent approach, before finally moving
past the predator. Also demonstrated in the runs were lateral
avoidance behaviors (once every three runs), and freezing behaviors (once ever run). Based on these data we felt that the
model was able to mimic real life prey behavior in a similar
situation with reasonable fidelity.

5000
0
1

7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73

-5000
-10000
-15000
-20000
-25000
Prey Step Number

Figure 4: Variation of NET affect for sample Turing test

The results indicate that EP used in conjunction with the affect model appears to produce reasonable affect-based behavior. The evolved factors are specific to the environment and to
the nature of the predator, as they are in nature; for example,
the deer respond differently to different predators in different
environments. Future work will likewise develop affect factors
specific to different environments and type of objects interacted
with. In short, the affect model appears generalizable to situations that are more complex in terms of environment and types
of danger. The results also suggest that there is an important
place for similar Turing-like tests for affect. Next, we compare our approach with alternate approaches.

5. Comparison with other affect models

Picard [33] wrote the first book on affective computing where
she used the terms affective computing systems and emotion-oriented computing to refer to systems that considered
emotions. Following Picard, numerous researchers have done
similar work. Space considerations permit us to address only
the two major groupings of this researchappraisal theory
based approaches and biologically based approaches.

5.1. Appraisal Theory Based Approaches

Appraisal theory by far is the basis of the largest amount of
work in computational theories of emotion. In appraisal theory,
emotions result from cognitive assessments of events in relation
to ones beliefs, desires, and intentions [5]. The central process is cognitive appraisal and much work has focused on the
structural relationship between appraisal variables and specific
emotion variables and cognitive responses. Applications include human-computer-interaction, psychology, and AI and described in [14, 36, 40, 35, 9, 29] among others. Marinier [26]
used appraisal theory as a source of intrinsic reward in Eaters
a maze much like Pac-Man, but lacking a predator. In general,
research in appraisal theory has been active for several decades;
see Scherer et al. 2010 for a good summary.
The key difference between appraisal theory and the biological approach we propose is that in appraisal theory, the selection of appraisal variables is unconstrained by the need for empirical evidence in underlying biology. Instead, appraisal theory based approaches use if-then rules specified from outside
the system. Consequently, we suggest a complementary approach; one completely constrained by the underlying biology

of affect, and with rules generated internally based on the interaction of the objects with the general goal of maximizing agent
well-being. As more information of the biological underpinnings of affect becomes available, we surmise that biological
based approaches, presently very much in their infancy, will
eventually match the abilities of current, sophisticated, appraisal theory based approaches.

5.2. Biologically Based Approaches

Biologically based theories (including ours), try to model the
biological systems that underlie affect. For this reason, we
provide a more detailed review of other such research.
Velasquez [42] modeled important aspects of emotional processing, for use as a biasing signal when making decisions. He
identified six primary emotions based primarily on ideas from
Ekman [12] and used associative networks comparable to
Minskys K-lines [28] in which salient external stimuli are
connected to primary emotions. The main draw back of his
system is that the designer explicitly specifies reward/punishment values associated with specific interactions
with specific objects, and in that sense is arbitrary. This contrasts with our approach where there is no explicit specification
of the reward/punishment value for each interactions and where
the agent makes its own affect-based evaluations of objects.
Drawing on the work of Pfeifer [31, 32] , Canamero [6, 7]
used emotions as the mechanism for action selection. She also
suggested novel artificial emotions not found in nature, but
which could be useful. While Pfeifer place basic emotions as
primary motivators, Canamero considered them second-order
modifiers. We do not see the need for artificial emotions and
view the basic emotions as primary motivators. Canamero
modeled six basic emotions in a neural net. We do not believe
that global neural network implementations account for the
modular nature of the brains affect handling systems.
Delgado-Mata et al. [10] used an affective system as part of
an ethologically-inspired action-selection mechanism for virtual
animals, to show that integrating emotion resulted in emergent
behavior of flocking. They did not explore the use of agents interacting with the objects to develop semantic understandings
of them.
Blumberg et al. [4] used an ethologically inspired computational model for action selection and learning in an autonomous
animated creature. A Behavior System computes the right
control signals to send to the agents motor system. Their approach is conceptually very similar to the one we propose here
except their decision algorithm arbitrates among different behaviors rather than the basic emotions that drive these behaviors.
Wright [43] suggested that the assignment of value to an
emotion is a self-monitored process of credit assignment, i.e.,
is assigned by the agent itself. He pointed to the importance of
a domain-independent representation of utility for adaptive architectures, but did not provide any implementation details.
We agree strongly with both his suggestions: in our approach,
agents independently develop affect-based evaluations of ob-

jects interacted with, based on the domain independent criteria

of maximizing their own well-being.

6. Conclusion
We aim our future work towards generalizing our approach to
handle more complex situations than the Pac-Man game. To do
this, we plan to explore analytical methods such as reinforcement learning to replace the current greedy optimization
strategy with a more efficient approach; introduce the concept
of learning during an agents lifetime based on the localized
damage (health) of a single agent; introduce memory and analytical reasoning; include more emotions such as PANIC so as
to model social situations, such as the teenage boy on his first
date; enrich environment diversity by introducing other affectenabled agents with different characteristics; improve the equations used in the affect model by continuing to study the underlying empirical evidence as it becomes available from mainstream neuroscience.
Rodney Brooks (2002) proposed a subsumption architecture where each layer subsumes the functionality of the layer
below it. In our system, there seems an obvious place for a like
approach: an affect layer subsumes the layer that manages the
agents body, and with a cognitive layer subsuming the affect
layer. This three-layer, triune architecture offers the possibility of a new kind of autonomous agent, one that because of its
ability to develop affect-based understandings of objects based
on its interactions with them is capable of developing semantic
understandings of these objects and hence demonstrates humanlike behavior in social or other complex emotional situations.

7. References
1. Armony, J. L., Servan-Schreiber, D., Cohen, J. D., and
LeDoux, J. E. 1997. Computational modeling of emotion: explorations through anatomy and physiology of fear conditioning. Trends in Cognitive Science, 1, 28-34
2. Barrett, L. 2006. Are emotions natural kinds? Perspectives on Psychological Science 1, 28-58
3. Barrett, L., Mesquita, B., Ochsner, K. N., and Gross, J. J.
2007. The experience of emotion. Annual Review of Psychology, 58, 373-403
4. Blumberg, B., Todd, P., and Maes, P. 1996. No bad dogs:
ethological lessons for learning. In: Proceedings of SAB96.
Cambridge, MA: The MIT Press
5. Bratman, M. E. 1987. Intention, Plans and Practical Reason Stanford, CA: CSLI Publications
6. Canamero, D. 1997. Modeling motivations and emotions
as a basis for intelligent behavior. In W. Lewis Johnson, ed.,
Proceedings fo the First International Conference on Autonomous Agents, New York, NY: ACM Press, 148-155
7. Canamero, D. 1998. Issues in the design of emotional
agents. In emotional and intelligent: the tangled knot of cognition. Papers from the 1998 AAAI Fall Symposium, Technical
Report FS-98-03, Meno Park, CA: AAAI Press, 49-54
8. Combes, S. A., Rundle, D. E., Iwasaki, J. M., and Crall, J.
D. 2012. Linking biomechanics and ecology through predator

prey interactions: flight performance of dragonflies and their

prey. The Journal of Experimental Biology, 215, 903-913
9. Connat, C. and Maclaren, H. 2004. Evaluating a probabilistic model of student affect. 7th International Conference on
Intelligent Tutoring Systems. Maceio, Brazil.
10. Delgado-Mata, C, Ibanez, J. and Aylett, R.S. 2003. Lets
run for it: consepcific emotional flocking triggered by virtual
pheremones. Proc. Smart Graphics 2003, LNCS 2733, Springer-Verlag
11. DeNero, J. and Klein, D. 2010. Teaching introductory artificial intelligence with Pac-Man. Symposium on Educational
Advances in Artificial Intelligence
12. Ekman, P. 1992. An argument for basic emotions. In:
Stein, N. L., and Oatley, K. eds. Basic Emotions, 160-200.
Hove, UK: Lawrence Erlbaum
13. Fogel, L. 1999. Intelligence through simulated evolution:
forty years of evolutionary programming.
New York,
NY:Wiley-Interscience
14. Gratch, J., Marsella, S., Wang, N. and Stankoic, B. 2009.
Assessing the validity of appraisal-based models of emotion.
International Conference on Affective Computing and Intelligent Interaction. Amsterdam, IEEE.
15. Greene, J. D. 2004. Why are VMPFC patients more utilitarian? A dual-process theory of moral judgment explains. Department of Psychology, Harvard University, Cambridge,
Mass.
16. Greenfield, S. 2000. The private life of the brain. New
York, NY: John Wiley & Sons.
17. Heider F. and Simmel, M. 1944. An experimental study
of apparent behaviour. American Journal of Psychology, Volume 13
18. Joseph, P. G. and Levkowitz, H. 2011. Patterns of emotion driven by affect state and environment. The Third International Conference on Pervasive Patterns and Applications,
Rome,
Italy
(http://www.thinkmind.org/index.php?view=article&articleid=
patterns_2011_3_20_70039)
19. Joseph, P. G. and Levkowitz, H. 2012 a. An algorithmic
approach to affect driven stream of consciousness. The Sixteenth International Conference on Cognitive and Neural Systems, Boston, USA
20. Joseph, P. G. and Levkowitz, H. 2012 b. Computing affect in autonomous agents, IEEE Conference on Development
and LearningEpiRob, SanDiego, CA, USA, 2012
21. Kalat, J. 2008. Biological psychology. Florence,
KY:Wadsworth Publishing
22. Kim, J. 1996. Philosophy of mind. Boulder, CO: Westview
23. LeDoux, J. 2003. Synaptic self: how our brains become
who we are, New York, New York: Viking
24. Luan K. P., Wagner, T. D., Taylor, S. F., and Liberazon,
I. 2004. Functional Neuroimaging Studies of Human Emotions,
CNS Spectrums, Volume 9Number 4, 258-266
25. Mao, W. and Gratch, J. 2006. Evaluating a computational
model of social causality and responsibility. 5th International

Joint Conference on Autonomous Agents and Multiagent Systems. Hakodate, Japan.

26. Marinier, R. P. 2008. A computational unification of cognitive control, emotion, and learning. PhD dissertation, Computer Science. Ann Arbor, MI, University of Michigan.
27. Marinier, R. P., Laird, J. E., and Lewis, R. L. 2009.A
computational unification of cognitive behavior and emotion.
Cognitive Systems Research, 10, 48-69.
28. Minsky, M. 1986. The society of mind. New York, NY:
Simon & Schuster.
29. Paiva, A., Dias, J. and Aylett, R. 2005. Learning by Fealing: Evoking Empathy with Synthetic Characters. Applied Artificial Intelligence special issue on "Educational Agents Beyond Virtual Tutors", 19, 235-266.
30. Panksepp, J. 1998, Affective neuroscience: The Foundations of Human and Animal Emotions, New York, New York:
Oxford University Press
31. Pfeifer, R. 1991. A dynamic view of emotion with an application to the classification of emotional disorders. Vrije
Universiteit Brussel, AI Memo 91-8.
32. Pfeifer, R. 1993. Studying emotions: fungus eaters. Proceedings of the Second European Conference on Artificial
Life, ECAL93, ULB, Brussels, Belgium 916-927
33. Picard, R. 1997. Affective computing, Cambridge, USA:
The MIT Press
34. Poli, R., Langdon, W. and McPhee, N. 2008. A field
guide to genetic programming. Lulu.com
35. Purves, D., Augustine, G. A., Fitzpatrick, D., Hall, W. C.,
LaMantia, A., White, L. E., (2011), Neuroscience, 5th Edition,
Sinaeur Associates, 858 pp.
36. Reilly, W. S. 1996. Believable Social and Emotional
Agents. Pittsburgh, PA, Carnegie Mellon University.
37. Russell, S. and Norvig, P. 2009. Artificial intelligence, a
modern approach, New Jersey: Prentice Hall
38. Scherer, K.R., Bnziger, T., & Roesch, E. (Eds.) 2010. A
blueprint for affective computing. Oxford: Oxford University
Press
39. Sterelny, K. The representational theory of mind (Oxford:
Blackwell, 1990).
40. Traum, D., Rickel, J., and Marsella, S. 2003. Negotiation
over tasks in hybrid human-agent teams for simulation-based
training. International Conference on Autonomous Agents and
Multiagent Systems. Melbourne, Australia.
41. Turing, A. 1950. Computing machinery and intelligence,
Mind, 59, 433-460
42. Velasquez, J. D. 1998. Modeling emotion-based decisionmaking. In Emotional and Intelligent: The Tangled Knot of
Cognitions, Papers from the 1998 AAAI Fall Symposium.
Technical Report FS-98-03. Menlo Park, CA: AAAI Press,
164-169
43. Wright, I. 1996. Reinforcement learning and animat
emotions. In: Proceedings of SAB96, Cambridge, MA: The
MIT Press