Constructing Complex NPC
Behavior via MultiObjective Neuroevolution
Jacob Schrum – schrum2@cs.utexas.edu
Risto Miikkulainen – risto@cs.utexas.edu
University of Texas at Austin
Department of Computer Sciences
Introduction


Goal: discover NPC behavior automatically
Benefits





Challenges





Save production time/effort
Learn counterintuitive behaviors
Find weaknesses in static scripts
Tailor behavior to human players
Games are complex
Multi-modal behavior is required
Success often depends on multiple objectives
RL & Evolution popular approaches
Current methods do not explicitly consider multiple objectives!
Multi-Objective Optimization


Exceptional performance in any objective is important
Imagine game with two
High health but did not deal much damage
objectives:




Damage Dealt
Health Remaining
Tradeoff between objectives
A dominates B iff A is
strictly better in one
objective and at least
as good in others
Population of points
not dominated are best:
Pareto Front
Dealt lot of damage,
but lost lots of health
NSGA-II


Evolution: natural approach for finding optimal population
Non-Dominated Sorting Genetic Algorithm II*




Population P with size N; Evaluate P
Use mutation to get P´ size N; Evaluate P´
Calculate non-dominated fronts of {P P´} size 2N
New population size N from highest fronts of {P P´}
*K. Deb et al. 2000
Neuroevolution




Genetic Algorithms + Artificial Neural Networks
NNs good at generating behavior
GA creates new nets, evaluates them
Three types of mutations (no crossover used)
Perturb Weight
Add Connection
Add Node
Battle Domain
15 NPCs
attack player
Player
Bat
Infinite Plane
NPC Fitness Objectives

NPCs have several goals
Inflict damage as a group → Attack Assist Bonus
 Avoid damage individually → Damage Received
 Stay alive individually → Time Alive


Objectives are contradictory and distinct
NPCs take damage from bat
Player is knocked back by NPC
Experiment

Multi-objective vs. Single-objective


Single-objective uses weighted sum of z-scores
Evolve each method against static player strategies
Spinning strategy
 Alternating strategy
 Chasing strategy


NPCs must beat
previous player
strategy to progress
Results
Differences for Alternating and Chasing significant with p < .05
Evolved NPC Behaviors

Single-Objective
Cannot find balance between
attack and defense
 Behavior is ineffectual


Multi-Objective

Baiting Behavior
One agent acts as bait
 Others attack player on blind side


Charging Behavior
Try to dodge bat while rushing in
 Maximizes damage dealt

Discussion

Benefits
Multi-modal behavior
 Exploits dynamics of
environment
 Individuals can have different
behaviors


Future Work
Evolve against humans
 Learn many behaviors in
advance
 More complex games:
Unreal Tournament

Conclusion


Multi-objective encourages multi-modal behavior
Multi-objective better than single-objective
Deals with contradictory objectives
 Leads to several different but effective solutions


Lead to multi-modal behavior in future games
Questions?
See movies at: http://nn.cs.utexas.edu/?multiNPCs
E-mail: schrum2@cs.utexas.edu