Swarm Algorithms
Akshay Narayan
Shruti Tople
Abha Belorkar
Pratik Shah
Shweta Shinde
Wang Shengyi
Ratul Saha
(A0095686)
(A0109720)
(A0120126)
(A0107576)
(A0109685)
(A0120125)
(A0110031)
1
Introduction
AKSHAY
3
4
What is Swarm Intelligence?
•
Emergent collective intelligence of groups of
simple agents
◦ Foraging in insect colonies
◦ Flocking of birds
◦ Nest building
• Why is it interesting to us?
◦ Distributed systems of agents
◦ Optimization & robustness
◦ Self organized control
5
Mechanisms
• Stigmergy
• Self organization
6
Stigmergy
• Stigma = mark/sign; ergon = work/action
• Indirect agent interaction
◦ Through the environment
• Environment modification
◦ Work state memory
• Work not agent specific
7
Self Organization
• Dynamic mechanism
• Interaction at lower level entities
• Structure appear at global level
• Characteristics
◦ Positive feedback
◦ Stabilization
◦ Numerous iterations
◦ Multistability
8
What to Expect Today?
• Ant colony optimization
◦ Framework
◦ Convergence
◦ Application to TSP
• Particle swarm optimization
◦ Framework
◦ Application to TSP
• Applications of swarm algorithms
9
Ant Colony Optimization
• Ants ↔ Agents
• Individual behaviour
• Mark path to food
◦ Environment modification
• Other ants follow the trail
◦ Indirect interaction
• Decision = Probabilistic selection
10
Particle Swarm Optimization
• Particles ↔ Birds/Fish
• Each particle has life
• Cognitive ability
◦ Evaluate the quality of its own experience
• Social knowledge
◦ Perceive knowledge of neighbours
• Decision = Cognition + Social factor
11
Ant Colony
Optimization
SHRUTI
Ant Behavior
• Ants release “pheromones” on their path
◦ Pheromones evaporate after some time
◦ Other ants follow the trail with maximum pheromone
•
Stigmery : Trail-laying and trail-following
behaviour
• Ants find the shortest path from nest to food
13
The Double Bridge Experiment
•
Marco Dorigo (1991)
•
Two bridges, one short and other long
•
Randomly choose one of the paths
•
Pheromone concentration increases on
shorter path
14
15
Toward Artificial Ants
•
Goal: Define algorithms to solve shortest
path problems
•
Real ants behaviour does not scale up
•
Ant colony optimization
◦ Artificial stigmergy
◦ Artificial ants
16
Artificial Ants : Features
•
Forward pheromone trail updating
mechanism
◦ Introduces loops
◦ Remove forward pheromone
• Limited memory of Ants
◦ Store forward path
◦ Cost of each edge
Food
Nest
17
Basic ACO Mechanisms
• Probabilistic forward
◦ Store edge in ant’s memory
• Deterministic backward
◦ Loop removal
4
Nest1
2
9
5
3
7
6
Food
8
Original path : 1 2 3 4 5 6 7 3 4 5 8 9
1 2 3 4 5 8 9
New path :
18
Basic ACO Mechanisms
• Pheromone updates
◦ Based on solution quality
• Pheromone evaporation rate
◦ Effects the solution
◦ Path exploration
• Visibility
Food
Nest
◦ Reciprocal of distance
19
Comparison
Nature
ACO Algorithm
Natural habitation
Graph (nodes and edges)
Nest and food
Source and Destination nodes
Ants
Agents, Artificial Ants
Visibility
Reciprocal of distance (h )
Pheromones
Artificial Pheromones (t )
20
ACO Algorithm
initialize: pheromones (t ) , visibility (h )
foreach iteration:
foreach ant:
while (destination not reached):
Choose next state probabilistically
Add edge to ant’s memory
Remove loops from ant’s memory
Evaluate solution
Update pheromones – backward trail
if (local best better than global solution)
Update the global solution
21
Metaheuristic &
Convergence
ABHA
Metaheuristic
• Heuristic
◦ Experience based methods to obtain approximately
optimal solutions
• Metaheuristic
◦ A framework to generate problem-specific heuristic
23
Combinatorial Optimization
• Combinatorial optimization problem: (S, f,W)
◦ S : set of candidate solutions
◦ f : objective function that assigns cost f (s) "s Î S
◦ W : set of constraints
• Goal: Find an optimal feasible solution s*
24
Problem Representation
• C : set of components
◦ C = {c1,c2,...,cN } where N C = total number of components
C
• L : set of connections between components
• GC =(C,L): construction graph
• X : set of states
◦ "x Î X, x = ci , c j ,..., ch ,... , a finite sequence over C
25
Problem Representation
• S : set of candidate solutions
• X : set of feasible states
• S* : set of optimal solutions
• f (s) : cost function over S
26
Mapping to ACO
C
L
GC = (C, L)
x
S
X
S*
f (s)
Variables
Components
Connections
ACO
Nodes
Edges
Construction graph
State
Candidate solutions
Feasible solutions
Input graph
Partial/complete path
Tours
Tours satisfying constraints
Optimal solutions
Cost function
Tour(s) with least cost
Distance/other cost
27
ACO Metaheuristic
Procedures:
• ConstructAntsSolutions
◦ Concurrent and asynchronous ant movement
• UpdatePheromones
◦ Modification of pheromone trails: deposition and
evaporation
• DaemonActions (optional)
◦ Centralized actions which cannot be performed by single
ants
28
Convergence
• Does ACO ever find an optimal solution?
• Optimal solution is found at least once
◦ For a sufficiently large q , P*(q ) ®1
• Assumptions
◦ Topology and costs of GC are static
◦ In case of multiple optimal solutions, we are satisfied with
any one
29
Some Notations
Notation
Nik
T
sq
sbs
x
Meaning
Feasible neighbourhood of ant k at
node i
Vector of pheromone trails t
Best tour in iteration q
Best tour so far
Path
ij
30
Algorithm Structure
• In each iteration q :
◦ ConstructAntSolutions: for all ants, select the next edge
probabilistically
◦ UpdatePheromone: Evaporation and deposition
31
ConstructAntSolutions
• Selecting the next node:
PT(ch+1 = j|xh)=
å
0
t ija
a
(i,l )ÎNik
t il
if (i, j) Î Nik
otherwise
◦ Simplification: No heuristic (h)
◦ T : vector containing pheromone values t ij
32
AntSolutionConstruction
AntSolutionConstruction:
select a start node c
1
x¬ c1
while( x ÏS and N k ¹f )
i
j ¬SelectNextNode (x, T)
if
x ÎS
x®xÅ j
return x
33
ACOPheromoneUpdate
• Quality function q f (s) :
◦ A non-increasing function with respect to f
f (s1)> f (s2)Þ qf (s1)£ qf (s2)
•Best-so-far update:
◦ Pheromone updated only on the best path so far
• We have t min > 0
34
ACOPheromoneUpdate
ACOPheromoneUpdate:
foreach (i, j) Î L do
t ij ¬ (1- r )t ij
if f (sq )< f (sbs) then
sbs ¬ sq
foreach (i, j) Î sbs do
t ij ¬ t ij +q f (sbs )
foreach (i, j) Î L do
Evaporation
Update best-so-far
Deposition
t ij ¬ max(t min, t ij )
35
Bounds on τ
• Lower bound on t : t > 0
min
• Upper bound on t :   q ( s*) / 
max
f
t ij (1) £ (1- r )t 0 + q f (s* )
t ij (2) £ (1- r )2 t 0 +(1- r )q f (s* )+ q f (s* )
q
t ij (q ) £ (1- r ) t 0 + å(1-r )q -i- q f (s* )
q
i=1
0< r £1 : The sum converges toq ( s*) / 
f
36
Convergence
• P*(q ) : Probability of finding an optimal
solution in first q iterations.
• Given:
◦ Arbitrarily small e > 0
◦ Sufficiently large q
• The following holds:
◦ P* (q ) ³1- e
◦ limq ®¥ P* (q ) =1
37
Proof
• pmin : min probability of selecting the next node
t
• p̂min =
and p  p̂
(N -1)t + t
a
min
C
a
a
max
min
min
min
n
• P ( s* generated in an iteration): p̂³ p̂min
>0
\ P ( s* not generated in an iteration): 1- p̂
\ P ( s* not generated in q iterations): (1  pˆ )
• P ( )  1  (1  pˆ )
*

 lim  
P ( ) 1
*
38
A stronger result
• What happens if we reach the solution and
keep iterating?
• It can be proved that the algorithm reaches a
state which keeps generating an optimal
solution
39
ACO applied to
Travelling Salesman
Problem
PRATIK
Travelling Salesman Problem
• Given:
◦ A list of cities
◦ Pairwise distances between them
  {a , b, c , d , e, a}
f (p )=11
• To find:
◦ Least cost tour
◦ Visiting each city exactly once
f ( ) d (i) (i 1) d(n)(1)
n1
i1

d
2
2
2
3
a
e
4
3
f (p ) = cost function = C
p = Permutations of cities visited b
dij = distance between node i & j
5
5
1
c
41
Glimpse of ACO
• Construction Graph
◦ Nodes = Cities
◦ Arcs = Connecting Roads between cities
◦ Weight = Distance
• Constraints (W)
◦ Each city to be visited exactly once
42
Glimpse of ACO
• Pheromone Trails (t )
ij
◦ Pheromone trails: desirability of visiting city j after i
• Heuristic Information (h)
◦ Visibility: 1/dij
• Solution Construction
◦ Initial selection : Random
◦ Termination : Definition is satisfied
43

Tour Construction
• Initialization:
◦ M Ants, N Cities
• Probabilistic Action


[

]
[

]
ij
ij
pijk 


[

]
[

]
 il
il
d
c
a
b
l  Nik
• Memory M
k
◦ To check feasible neighborhood Nk
◦ To compute the length of the tour
◦ To deposit pheromones
M1:{ a, b, c, d, a }
M2:{ b, d, a, c, b }
44
Pheromone Updation
• Pheromone Evaporation
d
 ij  (1 ) ij , (i, j)L
• Pheromone Deposition

c
a
 ij   ij   ijk , (i, j)L
m
b
k1


 ijk 
 0,
1 Ck,
if arc ( i, j) belongs to T k ;
otherwise


45
ACO Algorithm for TSP
time=t
d
c
a
b
↓
path1:{a, b, c, d, a }
anta(t)=1
antb(t)=2
antc(t)=3
antd(t)=0
path 1 (1)=a
path 1 (2)=b
path 1 (3)=c
path 1 (4)=d
• τ ij (t) = pheromones on edge i
to j at time t
• path = list of cities travelled
by ant
• anti (t) = number of ants at
node i at time t
• pij (t) = probability function
from node i to j at time t
• Ck = cost of path travelled by
at k
46
Algorithm & Complexity
1.Initialization
set t= 0
for all edge(i,j)
set τij(t)
set Δτij(t,t+n) := 0
for all node i
place anti(t)
set path_index := 1
for i=1 to n do
for k=1 to anti(t) do
pathk(path_index) := i
O(n2)
O(m)
O(m)
O(n2  m)
47
Algorithm & Complexity
2. repeat till path list is not full O(n)
set path_index := path_index+1
for all i=1 to n do
O(m)
for k=1 to anti(t) do
choose next node with prob pij(t) O(n)
move the kth ant to jth node
insert a node j in pathk(path_index)
O(n2m)
48
Algorithm & Complexity
3. for k=1 to m do
O(m)
k
compute C from path list
for path_index =1 to n-1 do
O(n)
set (h,l) := (pathk(path_index),
pathk(path_index+1))
Δτij(t,t+n):= Δτij(t,t+n) + 1/Ck
O(nm)
49
Algorithm & Complexity
4. for all edge (i,j)
τij(t+n):= (1-ρ) τij(t) + Δτij(t,t+n)
set time:= t+n
for all edge (i,j)
set Δτij(t,t+n):= 0
O(n2)
O(n2)
O(n2)
50
Algorithm & Complexity
5.Memorize the shortest path
if (iterations < iterationsMAX)
then
O(m)
for k=1 to m
for i=1 to n
O(n)
empty pathk (i)
set path_index:=1
for i=1 to n do
O(m)
for k=1 to anti (t) do
pathk (path_index):= i
Go to step 2
else
O(nm)
print shortest path
stop
51
Complexity Analysis
STEP
1
2
3
4
5
Overall Complexity
/ iterations
COMPLEXITY
O(n2 + m)
O(n2 m)
O(nm)
O(n2 )
O(nm)
O(n2 m)
52
How it works?
pde (t)
d
a
pad (t)
pae (t)
pab (t)
τab (t)
τ ea (t)
pac (t)
b
τ de (t)
pbd (t)
τ bc (t)
pbc (t)
e
τpcdcd(t)
(t)
c
pce (t)
53
Particle Swarm
Optimization
SHWETA
en.wikipedia.org/wiki/Swarm_behaviour
55
Analogy
•
Birds searching for food
•
Only know how far the food is
•
Particles – Agents that fly through the search
space
•
Record and communicate the solution
56
Basic Idea
• Each particle
◦ Is searching for the optimal
◦ Is moving and hence has a velocity
◦ Remembers the position where it had personal best result
so far
• All particles in the swarm
◦ Co-operate
◦ Exchange information (what they discovered in the places
they visited)
57
Search Space Terrain
58
Particle Swarm Optimization
• Particle has
◦ Position
◦ Velocity
◦ Neighborhood
◦ Fitnesses in its neighborhood
◦ Position of best fitness
• In each iteration, particles
◦ Communicate the best solution so far
◦ Update their positions and velocities
59
PSO: Initialization
60
PSO: Update
• In each time step
◦ Particle moves to a new position and adjusts velocity
◦ Current velocity + fraction of personal best + fraction of
neighborhood best
• New position = Old position + New velocity
61
PSO: Equation
Vid : Velocity of each particle
 : Inertia Weight
Ci : Constants
Xid : Current position of each particle
Pid : Best position of each particle
Pgd : Best position of swarm
62
PSO: Algorithm
initialize: position, velocity
foreach iteration:
foreach particle:
Calculate fitness value
if (fitness better than personal best):
set current value as the new Pid
Choose Pgd  best fitness value of all
foreach particle:
Calculate particle velocity
Update particle position
while (ending criteria)
63
Tuning the Parameters
• V max
◦ too low – too slow
◦ too high – too unstable
• C 1,C 2
◦ Can also influence the optimization process
64
PSO in Action
65
Selecting Parameters
66
PSO applied to
Travelling Salesman
Problem
SHENGYI
Particle
68
Particle
<5, 3, 2, 1, 4>
69
Particle
Encode tour using location labels: <5, 2, 1, 3, 4>
70
Swap Operators
<5, 3, 2, 1, 4> + (1, 5) = <4, 3, 2, 1, 5>
71
Swap Sequence
<3, 2, 5, 4, 1> + {(1, 2), (3, 4)}
= (<3, 2, 5, 4, 1> + (1, 2)) + (3, 4)
= <2, 3, 5, 4, 1> + (3, 4)
= <2, 3, 4, 5, 1>
72
Sequence Join
{(1, 2), (3, 4)} ⨁ {(3, 4), (1, 4)}
= {(1, 2), (3, 4), (3, 4), (1, 4)}
73
Path Difference
A: <1, 2, 3, 4, 5>
B: <2, 3, 1, 4, 5>
B+?=A
74
Path Difference
A: <1, 2, 3, 4, 5>
B: <2, 3, 1, 4, 5>
B + (A - B) = A
75
Path Difference
A: <1, 2, 3, 4, 5>
B: <2, 3, 1, 4, 5>
A[1] = B[3] = 1
76
Path Difference
A: <1, 2, 3, 4, 5>
B: <2, 3, 1, 4, 5>
B’ = <2, 3, 1, 4, 5> + (1, 3) = <1, 3, 2, 4, 5>
77
Path Difference
A: <1, 2, 3, 4, 5>
B’: <1, 3, 2, 4, 5>
A[2] = B’[3] = 2
78
Path Difference
A: <1, 2, 3, 4, 5>
B’: <2, 3, 1, 4, 5>
B’’ = <1, 3, 2, 4, 5> + (2, 3) = <1, 2, 3, 4, 5>
79
Path Difference
A: <1, 2, 3, 4, 5>
B: <2, 3, 1, 4, 5>
A - B = {(1, 3), (2, 3)}
80
Probability Factor
 , ,  : parameters between 0 to 1
81
Demo
82
Demo
•
Search Space: 8! = 40320
•
Number of Particles: 100
•
Iterations: 24
•
2400/40320 = 0.0595238
83
Demo
84
Applications
RATUL
Swarm Algorithms Applications
•
Wide range of applications to NP-hard
problems
•
TSP using ACO and PSO – covered already
•
TSP itself has many real world applications
Numerous other applications of ACO and PSO
86
Applications of ACO
Anything that fits the ACO-metaheuristic
•
Example: Sequence Ordering Problem (SOP)
•
It is an example of routing problem
•
Very similar to asymmetric TSP
•
We will solve SOP using ACO
87
SOP: Real-World Applications
Freight Transportation
88
SOP: Statement
•
Nodes = customers
•
Pairwise asymmetric
distances
89
SOP: Statement
•
Nodes = customers
•
Pairwise asymmetric
distances
•
Precedence
90
SOP: Statement
•
Input: Given a set of nodes + asymmetric
distances + precedence set
•
Output: Shortest tour subject to the
precedence relation
91
SOP: ACO Based Solution
Hybrid Ant System – SOP
•
Similar to ACO for TSP
•
The neighbour set of an ant is constrained by
the precedence relation
•
Innovative local search to handle multiple
precedence constraints
92
Applications of PSO
Anything that can be modelled as a search
space for the particles
•
Example: Optimizing neural network weights
•
The XOR function: If the two inputs are
different, output 1, otherwise 0
•
Model XOR as a neural network
93
Neural Network Weights
•
Weight – associated with each vector
•
Bias – associated with each internal node and
output
94
Neural Network Weights
•
Optimize the weights and biases
•
Particles move in nine-dimensional space
95
Take away
•
Framework inspired by nature
•
Simple agents
•
Optimal global behaviour
•
Swarm algorithms not limited to ACO & PSO
96
References
•
Kennedy, J. F., Kennedy, J., & Eberhart, R. C. (2001). Swarm Intelligence. Morgan Kaufmann.
•
Bonabeau, E., Dorigo, M., & Theraulaz, G. (1999). Swarm Intelligence. Oxford.
•
Dorigo, M., Stützle, T., (2004). Ant Colony Optimization. The MIT Press
•
Huang, L., Wang, K. P., Zhou, C. G., PANG, W., DONG, L. J., & PENG, L. (2003). Particle Swarm
Optimization for Traveling Salesman Problems [J].
•
C. Jacob and N. Khemka. Particle Swarm Optimization in Mathematica: An Exploration Kit for
Evolutionary Optimization. IMS'04: Proceedings of the Sixth International Mathematica
Symposium.
•
HAS-SOP: An Ant Colony System Hybridized with a New Local Search for the Sequential Ordering
Problem, Gambardella L.M, Dorigo M.
•
Perna, A., Granovskiy, B., Garnier, S., Nicolis, S. C., Labédan, M., Theraulaz, G., & Sumpter, D. J.
(2012). Individual rules for trail pattern formation in Argentine ants (Linepithema humile). PLoS
computational biology, 8(7).
•
https://www.youtube.com/watch?v=tAe3PQdSqzg
•
https://www.youtube.com/watch?v=yoiyR8o2ca0
97
Thank You