Low-level Teamwork Hybridization (LTH) for

P-metaheuristics : A Review and Comparative

S.Masrom1, Siti Z.Z. Abidin1, P.N.Hashimah1, A.S.Abd.Rahman2
1
Faculty of Computing and Mathematical Sciences
Universiti Teknologi MARA, Malaysia
{suray078, phashi655}@perak.uitm.edu.my
sitizaleha533@salam.uitm.edu.my

Computer and Information Science Department

2

Universiti Teknologi PETRONAS, Malaysia

sanirahman@petronas.com.my

Abstract. Inspired by nature, many types of Population based

metaheuristics or P-metaheuristics is cropping out of research labs to
help solve real life problems. Since every metaheuristics has its own
strength and weaknesses, hybridizing the algorithms can sometimes
produce better results. Among hybridization techniques available we
are particularly interested in the Low-level Teamwork Hybridization
(LTH). LTH operates at the internal structure of algorithms, thus
allowing researchers to finely select the components from the
algorithms to be hybridized. However, LTH technique is quite
complicated since it requires the restructuring of the original
algorithms. This paper identifies and describes the internal structures
of P-metaheuristics that are suitable candidates for hybridization
using LTH. The review and comparative study of several
implementations of this technique is also presented.

Keywords: P-metaheuristics;internal structures;hybridization; LTH.

1 Introduction
Metaheuristics have been considered among the best method for many types of
application especially in the optimization problems such as scheduling, routing,
resource allocating and time tabling. To date, a variety of metaheuristics have been
introduced with different and specific characteristics. Depends on the characteristics,
a metaheuristic can be grouped either as single based(S-metaheuristics) or population
based (P-metaheuristics). Genetic Algorithm (GA) [1], Ant Colony Optimization
(ACO) [2], Particle Swarm Optimization (PSO) [3], Bee Colony (BC) [4] and
Differential Evolution (DE) [5] are a few examples of P-metaheuristics. As for the
single based, the common metaheuristics are Simulated Annealing (SA) [6] ,Tabu
Search (TS) [7] and Variable Neighborhood Search (VNS)[8].
 Over the last years, many researchers have agreed that relying on sole
metaheuristics was quite restricted in achieving best solution for optimization
problem. Moreover, the “no free lunch theorem” [9] has explained that no single
optimization strategy will be always better than any other. Therefore, hybrid
metaheuristics have been widely accepted as an effective approach compared to the
single implementation.
The numerous implementations of hybrid metaheuristics have attracted many
researchers to cluster hybrid metaheuristics techniques into several classifications
and introduced different taxonomy [10]. The classification schemes however, are not
specifically grouped with regards on the internal structures of P-metaheuristics.
Identification of the internal structures that are suitable for hybridization has been
given in this paper.
The remaining content of this paper is organized as follows. Section 2 gives a
brief background about different classification schemes and taxonomies that have
been proposed for hybrid metaheuristics. Then the different hybrid classification is
presented in section 3. Section 4 and 5 provide review and comparison study on
several implementations of LTH respectively. Finally, in section 6, we conclude the
paper with a short summary.

2 Related Works

In this section, the focus is directed towards other research works that tackles relevant
issues in classification, methodology and taxonomy of metaheuristics hybridization.
As introduced in [10], the hybrid metaheurisitics scheme can be divided into three
general forms. There are component exchange among metaheuristics, cooperative
search from different metaheuristics and integration with others methods. The book
however, does not provide detailed explanation on the first category, which mostly
related to metaheuristics internal structure. Many of the implementation examples in
the rest of chapter are rather focused on the third category only.
A more detailed classification can be seen in [11]. The authors primarily
distinguish hybrid metaheuristics according to four criteria namely the kinds of
algorithms, the level of hybridization, the order of execution, and the control strategy.
Although level of hybridization was considered in the classification scheme, it is
however did not involved the elements of internal structure.
Mohammed El-Abd and Mohamed Kamel have introduced new taxomomy for
hybrid metaheuristics [12]. The taxonomy is created according to the algorithms
involved in hybrid and space decomposition. Nevertheless, it is just focus on parallel
implementation of multiple metaheuristics that is not required for internal structure
modifications. On the contrary, a classification scheme that was concentrated with
hybrid evoloutionary algorithms (EAs) has been described in [13]. In this work,
different methodologies and architectures of hybrid EAs has been illustrated with
variation of implementations but all of them were mostly associated with general
combination of algorithms.
In a more detailed view, E. G. Talbi in the Journal of Heuristics [14] have
introduced another taxonomy for hybrid metaheuristics techniques. The classification
has been grouped into hierarchical and flat scheme. Low-level Relay Hybridization
(LRH), Low-level Teamwork Hybridization (LTH), High-level Relay Hybridization
(HRH) and High-level Teamwork hybridization (HTH) are the four classifications
regarding to the hierarchical scheme. Each class from the hierarchy scheme can be
associated to the certain criteria from the flat scheme such homogeneous versus
heterogeneous and global versus partial.
We have gained a lot of inspirations from the Talbi classification scheme
especially to the LTH technique. LTH is a hybrid technique that embedded other
metaheuristics in P-metahaeuristics. It has been shown from a comparative study that
LTH method has sparked the most interest among researchers compared to other
schemes [14]. In term of the hybrid effectiveness, LTH has been proved to
outperform LRH and HRH[15].
The implementation of LTH can be considered as quite complicated. It is
involved directly to the modification of internal structure of the hybridized
algorithms. In fact, choosing a suitable metaheuristics combination at the right
internal structures will contribute to some significant impacts to the hybridization
effectiveness[11].
As to concern with the difficulties, a study should be done to identify and
describes the internal structures of P-metaheuristics that are suitable candidates for
hybridization using LTH. This study can provide some guidance for the LTH
implementation thus make new classification is possible.

3 Internal Structure of P-metaheuristics

In P-metaheuristics, the internal structure that is suitable for LTH can be classified as
population initialization (PI) and population generation (PG).
PI method has some significant influence in the effectiveness of the
metaheuristics. Nevertheless, the determination of PI is often overlooked during the
design phase [16]. In P-metaheuristics, the initial populations are naturally diversified
which benefits in reducing premature convergence problem of P-metaheuristics.
Unfortunately, in some cases, a sole P-metaheuristic are not enough diversified. This
is why hybridization is essential at the population initialization.
In P-metaheuristics, there are two categories of PG strategy have been formed.
One is related to evolution process, involved selection and reproduction using
variation operators, namely as crossover and mutation. Evolutionary algorithms (EAs)
[1] are the most well-known P-metaheuristics in this type. The second category is
associated on utilizing shared memory mechanism which is used as input information
for the next population generation. ACO for example, uses pheromone matrix
information in each population generation while PSO updates particles velocity,
location, personal and global best solutions.
4 A Review of LTH

A review on several LTH has been done based on the internal structure of P-
metaheuristics.

4.1 Population Initialization (PI)

A remarkable result has been achieved from PI hybridization technique that combines
single-parent evolution into GA population initialization [17]. The hybridization has
shown an improvement when applied to the route optimization process for wireless
sensor network. Similarly, an insertion of differential evolution (DE) for PI in genetic
algorithm (GA) has been proved to improve the solution of job shop scheduling
problem [18]. Hybridization between GA, Multiple Phase Neighbourhood Search
GRASP (MPNS-GRASP) and PSO [19] has also produced significant results when
applied to vehicle routing problem (VRP). In this research, the MPNS-GRASP has
been used for the population initialization. Alternatively, ACO has also been applied
with EAs based population initialization like GA [20] as to get high population
quality. Efficiency and effectiveness of the hybridization of tabu search into ACO
has also been achieved when applied to the open vehicle routing problem [21].

4.2 Population Generation (PG)

Hybridization of PG is the most popular method for P-metaheuristics. The variants of

EA for example, have different crossover and mutation operators. Any of these
operators are possible to be used in any single p-metaheuristics among the EAs. The
hybridization technique has been demonstrated with some empirical experiments. All
the results show that hybrid EAs outperform the individual ones in all of the
benchmark problems[22].
A similar work that hybrid EA with estimation distribution algorithm (EDA) also
proved to give better results than the individual GA and EDA[23]. However, in this
work, it has introduced several participation function (PF) which describes a ratio of
how many individual has been selected from each EA and EDA. All the PF factors in
the hybridization produced successful results when tested on the satisfiability (SAT)
problem [24].
In some techniques, S-metaheuristics strategy has been used as a selection method
in the EAs for example GA with local search [25], GA with simulated annealing[26],
EAs with taboo search [27] and GA with Variable Neighbourhood Search (VNS)
[18].
On the other hand, there is a way for researcher to hybrid memory based P-
metaheuristics before the stage of selection process in the evolution based P-
metaheuristics. For example, PSO strategy can be used to evolve each individual of
GA population after the crossover and mutation operations[19, 28]. This is to ensure
that only the fitness individuals who are successful in adapting to their environment
will have a better chance of surviving and reproducing.
 Experiment test on some optimization benchmark problems have shown that the
performance of particle swarm optimization algorithm could be improved with the
integration of Velocity Propelled Averaged Crossover (VPAC) operator [29]. VPAC
operator has been created based on the ideas of GA operators but this new parameter
has been used to determine the population proportion before proceeding to the
reproduction processes.
With respect to increase the particle swarm synergy, a mechanism of mutation
from the EA paradigm could be adapted before the process of population update in
PSO[30]. This technique has been tested on some benchmark problems and the output
was better than the standard PSO. Combining PSO and DE operator [31] is an another
successful work in which the results outperformed the two hybridization methods of
[29] and [32].
As to improve the pheromone updating rules of ant colony optimization (ACO)
algorithm, an employment of PSO mechanism into ACO population generation
proved to improve the performance of the algorithm [33]. Through the convergence
analysis, the algorithm with the hybridization method has achieved better
convergence than the single PSO and ACO. In other way, an introduction to GA
operators to ACO population generation has produced an improved result to the single
ACO[21].
Hybridization between ACO with local search procedure is one of the examples
that combine S-metaheuristics and P-metaheuristics. This technique demonstrates
excellent convergence and robustness in uncovering low risk paths or routes in a
sparse graph [34]. Hybridization between ACO with simulated annealing and ACO
with tabu search with related to the course timetabling problem has produced good
solutions compared to others methodologies[35].
Bee colony (BC) is another type of swarm intelligence technique like ACO and
PSO. As a P-metaheuristics type, BC population needs for hybridization method for
performance improvement. Combination of BC with greedy heuristic and local
search strategy for population generation can produce significant and satisfactory
results for the optimization problem [36]. Similarly, after testing to the Knapsack
Problem, a better result has been achieved from the hybridization of BC with
PSO[37].

5 A Comparative Study

Table 1 illustrates the comparison of LTH according to three different aspects: the
metaheuristics, internal structure and problem applied in the experiment.
It is shown from comparison table that majority of P-metaheuristics hybridization
techniques have been implemented at the population generation (PG). In most cases,
researchers prefer to use benchmark optimization problems such as Ackley, Rastrigin,
Ellipse, Rosenbrock, Sphere, Schaffer, Griewank, Ellipse and Ellipsoida. However, it
has been revealed from various studies that LTH implementation provides effective
and improved solution to the real optimization problems like network design and
course time-tabling.
 Table 1. A comparison of LTH

Research works Metaheuristics Internal Problem

structure
L. Guo, et al[17] GA with single population PI Network routing
G. Zobolas, et al. [18] GA with DE PI Scheduling
GA with VNS PG
Y. Marinakis and M. Marinaki GA with MPNS-GRASP, PI VRP
[19] GA with PSO PG
H. Guangdong, et al [20] ACO with GA PG Scheduling
X.-Y. Li, et al. [21] ACO with tabu search PI VRP
PG
Antonio LaTorre et al. [23] EA variants(GA,ES) and PG Benchmark
EDA
J. M. Peña, et al. [24] GA with DE PG SAT
V. Robels et al. [38] GA with EDA PG Bench-mark
M. Gen, et al. [25] GA with local search PG VRP
Y. K. Ahn, et al. [26] GA with SA PG Nonlinear
squeeze film
damper
J. P. Caldeira, et al. [27] EA with tabu search PG Scheduling
F. Juang [28] GA with PSO PG Network design
S. Matthew and S. Terence PSO with GA PG Benchmark
[29]
Z. You and T. Ying [30] PSO with GA PG Benchmark
X. Wei and G. Xingsheng [31] PSO with EAs PG Benchmark
B. Shuang, et al. [33] ACO with PSO PG TSP
K. K. Lim, et al. [34] ACO with local search PG Path planning
M. Ayob and G. Jaradat[35] ACO with SA PG Course time-
ACO with tabu search tabling
S. Pulikanti and A. Singh [36] BC with greedy heuristic PG Knapsack
M. Aurelio, et al. [37] BC with PSO PG Knapsack

6 Conclusion

Low-level Teamwork Hybridization is a technique that is used to combine P-

metaheuristics with other metaheuristics. It involves restructuring original algorithms
through modification of P-metaheuristics internal structure, i.e. the essential elements
that determine the traits and characteristics of original algorithms. The main internal
structures of P-metaheuristics that can be embedded with other metaheuristics are
population initialization and population generation. Based on our review and
comparative study, hybridizations at the population generation is preferred by many
researchers. We have also found that the use of GA is very common when
implementing LTH based hybridization.

Acknowledgement

The authors would like to thank Universiti Teknologi MARA for its financial
support to this project.

References

1. Affenzeller, M., et al., Genetic Algorithms and Genetic Programming - Modern Concepts
and Practical Applications. 2009: CRC Press.
2. Dorigo, M. and T. Stutzle, Ant Colony Optimization. 2004: MIT Press Ltd.
3. Clerc, M., Particle Swarm Optimization. 2006: ISTE.
4. Snodgrass, R.E., Anatomy of Honey Bee. 1953, Ithaca, NY: Comstock Publishing
Associates.
5. Storn, R. and K. Price, Differential evolution: A simple evolution strategy for fast
optimization. Dr. Dobb's Journal, 1997. 22(4): p. 18-24.
6. Laarhoven, P.J.M.V. and E.H.L. Aarts, Simulated Annealing: Theory and Applications
(Mathematics and Its Applications). 1988: Kluwer Academic Publishers Group.
7. Glover, F. and M. Laguna, Tabu Search. 1998, Boston.: Kluwer Academic.
8. Hansen, P., N. Mladenović, and J.A.M. Pérez, Variable neighbourhood search: methods
and applications. Annals of Operations Research, 2009. 175(1): p. 367-407.
9. Wolpert, D. and W. Macready, No free lunch theorems for optimization. EEE Transaction
on Evolutionary Computation, 1997. 1(1): p. 67-82.
10.Blum, C. and A. Roli, Hybrid Metaheuristics: An Introduction, in Hybrid Metaheuristics, C.
Blum, et al., Editors. 2008, Springer. p. 1-30.
11.Raidl, G.R., J. Puchinger, and C. Blum, Metaheuristic Hybrids, in Handbook of
Metaheuristics, Pardalos, et al., Editors. 2010, Springer New York. p. 305-335.
12.El-Abd, M. and M. Kamel, A Taxonomy of Coorperative Search Algorithm, in Hybrid
Metaheuristic 2005, M.J.B.e. al., Editor. 2005, Springer-Verlag Berlin Heidelberg. p. 32-41.
13.Grosan, C. and A. Abraham, Hybrid Evolutionary Algorithms: Methodologies,
Architectures, and Reviews. Studies in Computation Intelligence (SCI), 2007. 75: p. 1-17.
14.Talbi, E.G., A Taxonomy of Hybrid Metaheuristics. Jounal of Heuristics, 2002. 8: p. 541-
564.
15.Lau, H.C., et al., A software framework for fast prototyping of meta-heuristiccs
hybridization. International Transactions in Operational Research, 2007. 14(2): p. 123-141.
16.Talbi, E.-G., Metaheuristics: From Design to Implementation. 2009: Wiley. 586.
17.Guo, L., B. Wang, and Q. Tang, A Hybrid Genetic Routing Algorithm in Wireless Sensor
Networks, in Advances in Wireless Networks and Information Systems, Q. Luo, Editor. 2010,
Springer-Verlag: Berlin Heidelberg. p. 87–92.
18.Zobolas, G., C. Tarantilis, and G. Ioannou, A hybrid evolutionary algorithm for the job shop
scheduling problem. Journal of the Operational Research Society, 2009. 60: p. 221-235.
19.Marinakis, Y. and M. Marinaki, A hybrid genetic - Particle Swarm Optimization Algorithm
for the vehicle routing problem. Expert System with Applications, 2010. 37: p. 1446-1455.
20.Guangdong, H., L. Ping, and W. Qu. A Hybrid Metaheuristic ACO-GA with an Application
in Sports Competition Scheduling. in Eighth ACIS International Conference on Software
 Engineering, Artificial Intelligence, Networking, and Parallel/Distributed Computing
(SNPD 2007). 2007. Haier International Training Center, Qingdao, China IEEE.
21.Li, X.-Y., P.Tian, and S. Leung, An ant colony optimization metaheuristic hybridized with
tabu search for open vehicle routing problems. Journal of the Operational Research Society,
2009. 60: p. 1010-1025.
22.LaTorre, A., et al. Hybrid Evolutionary Algorithms for Large Scale Continuous Problems. in
GECCO’09, July 8–12, 2009. 2009. Montréal Québec, Canada.
23.Peña, J.M., et al., GA-EDA: Hybrid Evolutionary Algorithm Using Genetic and Estimation
of Distribution Algorithms, in IEA/AIE 2004, R.O.e. al., Editor. 2004, Springer-Verlag
Berlin Heidelberg. p. 361-371.
24.Rodriguez-Tello, E. and J.Torres-Jimenez. ERA: An algorithm for reducing the epistasis of
SAT problems. in Genetic and Evolutinary Computation Conference. 2003: Springer Verlag.
25.Gen, M., L. Lin, and J.-B. Jo, Hybrid Genetic Algorithm for Designing Logistics Network,
VRP and AGV Problems, in Intelligent and Evolutionary Systems. 2009, Springer-Verlag
Berlin Heidelberg p. 123-139.
26.Ahn, Y.K., et al., Optimal Design of Nonlinear Squeeze Film Damper Using Hybrid Global
Optimization Technique. Journal of Mechanical Science and Technology, 2006. 20(8): p.
1125-1138.
27.Caldeira, J.P., F. Melicio, and A. Rosa. Using a Hybrid Evolutionary-Taboo Algorithm to
solve Job Shop Problem. in ACM Symposium on Applied Computing. 2004. Nicosia, Cyprus:
ACM.
28.Juang, F., Hybrid of Genetic Algorithm and Particle Swarm Optimization for Recurrent
Network Design. IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—
PART B: CYBERNETICS, 2004. 34(2).
29.Matthew, S. and S. Terence. Breeding swarms: a GA/PSO hybrid. in Proceedings of the
2005 conference on Genetic and evolutionary computation. 2005. Washington DC, USA:
ACM.
30.You, Z. and T. Ying. Particle swarm optimization with triggered mutation and its
implementation based on GPU. in Proceedings of the 12th annual conference on Genetic
and evolutionary computation. 2010. Portland, Oregon, USA: ACM.
31.Wei, X. and G. Xingsheng. A hybrid particle swarm optimization approach with prior
crossover differential evolution. in Proceedings of the first ACM/SIGEVO Summit on
Genetic and Evolutionary Computation. 2009. Shanghai, China: ACM.
32.Yen, G.G. and L. Wen Fung, Dynamic Multiple Swarms in Multiobjective Particle Swarm
Optimization. Systems, Man and Cybernetics, Part A: Systems and Humans, IEEE
Transactions on, 2009. 39(4): p. 890-911.
33.Shuang, B., J. Chen, and Z. Li, Study on hybrid PS-ACO algorithm. Applied Intelligence,
2009.
34.Lim, K.K., et al., Hybrid ant colony algorithms for path planning in sparse graphs. Soft
Computing, .2007. 12: p. 981–994.
35.Ayob, M. and G. Jaradat. Hybrid Ant Colony systems for course timetabling problems. in
The Second Conference on Data Mining and Optimization, 2009. 2009. Kajang, Malaysia:
IEEE.
36.Pulikanti, S. and A. Singh, An Artificial Bee Colony Algorithm for the Quadratic Knapsack
Problem, in ICONIP 2009, C.S.Leung, M. Lee, and J.H. Chan, Editors. 2009, Springer-
Verlag Berlin Heidelberg. p. 196–205.
37.Aurelio, M., et al., Application of the Bee Swarm Optimization BSO to the Knapsack
Problem, in Soft Comp. for Recogn. Based on Biometrics,, P.M.e. al., Editor. 2010,
Springer-Verlag Berlin Heidelberg p. 191–206.
38.Robles, V., et al., Extending the GA-EDA Hybrid Algorithm to Study Diversification and
Intensification in GAs and EDAs, in IDA 2005, A.F.F.e. al, Editor. 2005, Springer-Verlag
Berlin Heidelberg p. 339-350.