Particle Swarm Optimization with

Area Extension (AEPSO):

A Macroscopic model of PSO in
Robotic Swarm

Adham Atyabi
Supervisor: Dr. Somnuk Phon-Amnuaisuk
Co-supervisor: Dr. Chin Kuan Ho
 Order of the slides
 1. Particle Swarm Optimization (Basic PSO)

 2. Particle Swarm Optimization with Area

Extension (AEPSO)

 3. Simulations and Scenarios

 4. Experimental Results
Problem Statement
 The problem is Hostile robotic scenario based on
cooperative robots trying to navigate bombs
location and disarm them.

 The robots have limited knowledge about the

bombs location (only know the likelihood of bombs
in the area).

 The likelihood information is uncertain (because of

noise and Illusion effects).
Objectives
 To identify design and evaluate strategies for implementing a
new Particle Swarm Optimization (PSO) for robot navigation
in hazard scenarios and hostile situations.

 To solve the uncertainty in the perception level of the

robots/agents in cooperative learning scenario.

 To reduce the proportion of involved robots in the navigation

tasks with the aim of reducing costs.

 To solve the initial location dependency in navigation

scenarios.
 Particle Swarm Optimization
 PPSO
SO is
isan
anEvo
Evolu lutio
tionnary
aryA Algo
lgorith
rithm m ins
inspired
pired frfroomm an animal
imalssocial
ocial behaviors
behaviors..(K(Kenennene dy ,, 19
dy 1995,
95, Rib
Ribeir eirooaa nd
nd Sc
Sc hlan
hlansskker,,2200
er 005;
5; Ch
Chang
ang etet al.,
al., 20
2004
04;;PuPugh
gh and
and MM ar artino
tinoli,li, 200
2006;6;SSou
oussaa et
et al.,
al., 200
2003;
3; No
Nomu
murara,,200
2007)
7)
 PPSO
SO ououtp
tperf
erfor
ormed
med other
otherEvo
Evolu lutio
tionary
naryAlgo
Algorithms
rithms su such
ch as
as G GA Ain in so
someme pr prob
oblemslems(Ves(Vestters
ers trom
trom anandd Rig
Riget,et, 20
2002;
02; Ra
Ra tna
tna we
we era
era et
et al.
al.,,20
2004
04;;Pa
Pa ssuupul
puleti
eti and
and BB atti
attiti,2
ti,2006
006).).
 PPSO
SO isisan
an op
optimizatio
timizationn techn
techniq
iqueue wwhich
hich mod
models els aassetet ooffppotential
otentialpprrob oblemlem ssolutio
olutionnss as asaa sw
swarm arm ooffppararticlesticles momovvin inggabo
about ut inin aavvirirtual
tualssear
earch ch sp space. ace. (KeKe
( nnedy
nnedy,, 1995
1995 ))
 The
The metho
methoddw wasasininsspir
pired
ed by
by th
thee mov
movemen
ementtoofffflo lockckin inggbbird
irdss anandd their
theirinteraction
interactionssw with
ith ththeireirnneigh
eighbo bors rs in
in th
theeggro roup
up.. (Ke (Ke nne
nne dy,, 19
dy 1995
95 ))
 PPSO
SO ach
achieves
ieves ooptimizatio
ptimizationnuusinsingg ththree
reepprimar
rimaryy pr prinincip les:: ((Pug
ciples Pugh,
h, MM arti
artinoli
noli,,20
2007
07))

1)
1) EEva
valuat
luation,
ion, whe
where
re quanti
quantita
tative
tive fitne
fitness
ss ca
cann be
be de
dete
termine
rminedd forfor some
some pa partrticl
iclee loc
locat
ation;
ion;
2)
2) Comparison,
Comparison, where
where the
the best
best performe
performerr outout of
of multi
multipl plee partic
particle
less ca
cann be
be selselec
ecteted;
d;
3)
3) Imit
Imitat
ation,
ion, where
where the
the qua
qualit
litie
iess of
of bet
better
ter parti
particl
cles
es aare
re mi
mimic
mickekedd by
by othe
others.rs.
 Velocity: (1)

 Maximum Velocity (2)

 Inertia Weight (2-1)

 Solutions (positions): (3)

 Personal best and Global Best (4)

 Problems and Solutions in
Basic PSO
 Problems • Solutions
 Premature convergence • Improvement on neighborhood
topology, velocity equation,
 Parameter control global best and personal best.

 Diversity (exploration, exploitation)

• Niching PSO, Mutation,
 Dynamic domains Parallelism, Reinitialization,
Clearing memory, Using Sub-
 Real Time domains Swarms

(Brits, Engelbrecht, and Van Den Bergh, 2002,2003; Yoshida, et al.,2001; Stacey, Jancic and
Grundy,2003;Chang, et al., 2005; Vestestrom, Riget, 2002; Qin et al., 2004; Pasupuleti and
Battiti, 2006; Ratnaweera et al., 2004;Peram et al., 2003; Parsopoulos and Vrahatis, 2002)
Various Modeling of PSO
Literature suggested two models for implementing robot controller
in robotic swarm. These models are known as i) Macroscopic and
ii) Microscopic models.

 Microscopic Modeling: refer to implementations in which each

agent represent a swarm by itself and particles of that swarm,
represent the possible acts for that individual agent as the next
move.
 Macroscopic Modeling: refer to implementations in which each
agent represent a particle of the swarm therefore, the whole team
represent the swarm.
 (Agassounon et al., 2004, Chang et al., 2005, Kazadi et al., 2002, Lerman et al., 2001,
Martinoli et al., 2007, Pugh and Martinoli 2007a, Li et al., 2004)
Advantages of Macroscopic modeling

 Macroscopic modeling of the swarm helps to reduce

the amount of swarm population (swarm size)
which decrease the amount of computations.

 Macroscopic modeling helps to use PSO as a

controller for robotic swarms while in Microscopic
modeling, robots’ controller is mostly an Artificial
Neural Network (ANN)
Particle Swarm Optimization with Area
Extension (AEPSO)

 New velocity heuristics which improved the

performance.
 Environment reduction heuristics which solved
the premature and hill climbing problems.
 Limited communications range conditions.
 Credit Assignment heuristic, which solved the
stuck behind problem.
 Boundary Condition heuristic, which solved the
stuck behind problem.
 New velocity equation

 (5)
 Environment Reduction Heuristic
 The idea is based on dividing the environment to sub
virtual fixed areas with various credits.
 Areas credit defined the proportion of survivors and
obstacles positioned in the area.
 particles know the credit of first and second layer of its
current neighborhood.
 2 Deserting Policies are used (DP1, DP2)
Communication Methodology and Help Request
Signal Heuristics

 Agents can only communicate with those who are in

their communication range.
 Various communication ranges were used (500, 250,
187, 125, 5 pixels).
 This heuristic has major effect on the sub swarm
size.
 Help request signal can provide a chain of
connections.
Credit Assignment and Boundary
Heuristics
 Reward and punishment
 Suspend factor
Uncertainty (Illusion effect)

Random noise
(distributed uniform )

Illusion Effect
Cooperative AEPSO (CAEPSO)

 CAEPSO is designed to handle high level of

uncertainty and skill variation between
agents.

 CAEPSO employed two additional heuristics

(Speculation mechanism and Leave force
heuristics)
 CAEPSO use Past-knowledge gained during
the training
 Past-Knowledge
 Past knowledge refers to overall knowledge gathered during training phases
from various trials/executions.
 In the testing phase, agents might be experiencing same or new random
initializations.
Speculation mechanism and Leave
Force
Speculation mechanism is based on using an extra memory in
agents called Mask.
Masks can take values by:
 Illusion effect.
 Agent’s self observation.
 Self speculation.
 Neighbor’s observation.
 Neighbors speculation.
Leave Force is an extra punishment which will force agents to
decrease 10% of their current area after certain iteration.
Simulations and scenarios
 Simulation 1: variation of Basic PSO (parameter adjustment,
swarm size and population density)

 Simulation 2: AEPSO in Static and Dynamic environments

 Simulation 3: AEPSO in Static and Dynamic environments.

(Time Dependency, Area Deserting Policy and Uncertainty)

 Simulation 4: homogeneity and heterogeneity effects on

CAEPSO (Past Knowledge, Time Dependency, Uncertainty,
Skill Variation)
Simulation 1
Simulation 1- Parameter Adjustment
Simulation 1- Swarm Size
Simulation 1- Population Density
Simulation 2- Static and Dynamic Environments
Simulation 2- Static and Dynamic Environments
Simulation 3- Uncertainty and
Communication range
Simulation 3- Deserting Policy
Simulation 3
Simulation 3- AEPSO VS Basic-PSO1
Simulation 4-1- Effect of Past-
Knowledge
 4 different scenarios for past-knowledge is
designed (A1, A2, A3, A4)
 A1: same initialization as Training phase
 A2: new initialization for Testing phase
 A3: using 4 individual tables each referring
to a quarter of the environment.
 A4: using specific table for each execution
Simulation 4-1- Past knowledge and
Initialization

 Scenarios: A1, A2, A3, A4

Simulation 4-1- Comparison of AEPSO
and CAEPSO
Simulation 4-1- AEPSO’s Trajectory
Trace
Simulation 4-1- CAEPSO’s Trajectory
Trace
Simulation 4-1- A3 and A4 scenarios
Simulation 4-2
Simulation 4-2- survivors elimination
in various phases
Simulation 4-2- AEPSO’s Trajectory
Trace
Simulation 4-2- CAEPSO’s Trajectory Trace
Contribution
 AEPSO performed better local search compare with
other techniques (Basic PSO, Random Search, Linear
Search).

 In CAEPSO, more accurate Past-Knowledge favors in

faster performance achievements.

 AEPSO and CAEPSO are robust to Noise and Time

dependency.

 Cooperation between agents allowed CAEPSO to

perform well.
Conclusion and Future Work
• In this study, we introduced AEPSO and CAEPSO as a new modified version of
Basic PSO and we also investigated its effectiveness on static, dynamic, real-
time, multi dimension, and multi objective problem domains.

• It is necessary to mentioned that the small number of particles (only 5 robots)

gave a great advantage to AEPSO and CAEPSO (due to being able to reduce the
costs).

• Robots were able to solve problems with high level of complexities based on
using poor level of knowledge (training knowledge) and high level of
cooperation and experience sharing.

• We are going to compare CAEPSO results with a behaviour-based version of q-

learning in a Cooperative Learning scenario with Heterogeneous robots.
Publications
1. "Particle Swarm Optimization with Area Extension (AEPSO)", conf CEC2007, IEEE Congress on
Evolutionary Computation, Stanford university of Singapore, accepted in 15 July 2007.
2. "Effects of Communication range, Noise and Help request Signal on Particle Swarm Optimization
with Area Extension (AEPSO)", conf WIC / IAT, IEEE / ACM International Conference on
Intelligent Agent Technology, Stanford University, USA, 25-28 September 2007.
3. "Particle Swarm Optimizations: A Critical Review", conf IKT07, Third conference of Information
and Knowledge Technology, Ferdowsi University, Iran, submitted in 21 July 2007.
4. “Effectiveness of a Cooperative Learning version of AEPSO in Homogeneous and Heterogeneous
Multi Robot Learning Scenario”, Conf, IEEE World Congress on Computational Intelligence
(WCCI 2008, CEC08), Hong Kong, Accept March 2008.
5. "Applying Area Extension PSO in Robotic Swarm", Journal paper, Evolutionary Computation,
MIT-Press journal, submitted in June 2008).
6. “Robotic Navigation with PSO”, Soft Computing journal, Elsevier, Submitted in April 2008.
Reference
Ellips, M. and Davoud, S. “Classic and Heuristic Approaches in Robot Motion Planning
– A Chronological Review”, Proc. World Academy of Science, Engineering and
Technology, Vol 23 AUG 2007 ISSN 1307-6884.
Li W.; Yushu L.; Hongbin D. and Yuanqing X.; "Obstacle-avoidance Path Planning for
Soccer Robots Using Particle Swarm Optimization", Proc. IEEE Int. Conf. on Rob.
and Biomimetics (ROBIO '06). (2006) pp. 1233- 1238.
Saska, M.; Macas, M.; Preucil, L. and Lhotska, L. "Robot Path Planning using Particle
Swarm Optimization of Ferguson Splines", Proc. IEEE/ETFA '06, (2006) pp. 833-
839.
Xin C. and Yangmin L.; "Smooth Path Planning of a Mobile Robot Using Stochastic
Particle Swarm Optimization" Proc. IEEE on Mechatronics and Aut., (2006) pp.
1722-1727.
Yuan-Qing Q.; De-Bao S.; Ning L. and Yi-Gang C.; Path planning for mobile robot
using the particle swarm optimization with mutation operator Proc. Int. Conf. on
Machine Learning and Cybernetics, (2004) pp. 2473 – 2478.
Siegwart, R., Nourbakhsh, I. R., “Introduction to Autonomous Mobile Robots”, book,
Springer, Chapter 5, 2004.
Reference
Hettiarachchi, S. (2006). Distributed online evolution for swarm robotics.
Autonomous Agents and Multi Agent Systems.
J. Kennedy, and R. C. Eberhart, Particle swarm optimization,
Proceedings of the 1995 IEEE International Conference on Neural Networks,
vol. 4, IEEE Press, (1995): 1942-1948.
Kennedy, J. and Mendes, R. (2002). Population structure and particle swarm
performance. Proceedings of the 2002 Congress on Evolutionary
Computation (CEC ’02).
Krink, T., Vesterstrom, J. S., and Riget, J. (2002). Particle swarm optimization with
spatial particle extension. Congress on Evolutionary Computation (CEC),2002
IEEE World Congress on Computational Intelligence.
Lee, C., Kim, M., and kazadi, S. (2005). Robot clustering. Systems, Man and
Cybernetics, 2005 IEEE International Conference on.
Luke, S., Sullivan, K., Balan, G. C., and Panait, L. (2005). Tunably decentralized
algorithms for cooperative target. Fourth International Joint Conference on
Autonomous Agents and Multi-Agent Systems (AAMAS 2005).
Reference
Werfel, J., Yaneer, B. Y., and Negpal, R. (2005). Building patterned structures with
robot swarms. Nineteenth International Joint Conference on Artificial
Intelligence (IJCAI’05).
Yamaguchi, T., Tanaka, Y., and Yachida, M. (1997). Speed up reinforcment
learning between two agents with adaptive mimetism. IEEE/RSJ Intelligent
Robots and Systems.
Yang, C. and Simon, D. (2005). A new particle swarm optimization technique. 18 th
International Conference on Systems Engineering (ICSEng 2005).
Zavala, A.E.M., Aguirro, A.H. and Diharce, E.R.V. (2005). Constrained
Optimization via Particle Evolutionary Swarm Optimization Algorithm (PESO).
Proceedings of the 2005 conference on Genetic and Evolutionary
Computation.
Zhang,W. and Xie, X. (2003). Depso: Hybrid particle swarm with differential
evolution operator. IEEE, Systems, Man and Cybernetics.
Zhao, Y. and Zheng, J. (2004). Particle swarm optimization algorithm in signal
detection and blind extraction. IEEE, Parallel Architectures, Algorithms and
Networks.
Thanks