1

SurvivaI of the FIexibIe: ExpIaining the Recent

PopuIarity of Nature-Inspired Optimization
within a RapidIy EvoIving WorId

James M. Whitacre
School of Computer Science. Universitv of Birmingham. Edgbaston. UK
Biophvsics and Bioinformatics department. National Universitv of Mongolia. Ulaanbaatar.
Mongolia
5hone: (44) 121 414 2215; Iax: (44) 121 414 2799; e-mail: jwhitacre79mgmail.com

bstract#esearchers oIten comment on the 5o5:larity and 5otential oI nat:re-ins5ired meta-
he:ristics (NIM), however there has been a 5a:city oI data to directly s:55ort the claim that NIM
are growing in 5rominence com5ared to other o5timization techniq:es. In a com5anion article
5:blished in this s5ecial iss:e, I re5orted evidence that the :se oI NIM is not only growing, b:t
indeed has s:r5assed mathematical o5timization techniq:es (MOT) and other metahe:ristics in
several metrics related to academic research activity (5:blication Ireq:ency) and commercial
activity (5atenting Ireq:ency). Motivated by these Iindings, this article reviews several theories oI
algorithm :tility and disc:sses why these arg:ments remain :nsatisIying. I arg:e that any
ex5lanation oI NIM 5o5:larity sho:ld directly acco:nt Ior the manner in which most NIM s:ccess
has act:ally been achieved: thro:gh hybridization and c:stomization to s5eciIic 5roblems. By
taking a 5roblem liIecycle 5ers5ective, this 5a5er 5rovides sim5le yet im5ortant insights into how
nat:re-ins5ired meta-he:ristics might derive :tility by being Ilexible. Given global trends in the
evol:tion oI b:siness 5rod:cts and services where o5timization algorithms are a55lied, I s5ec:late
that highly Ilexible algorithm Irameworks will become increasingly 5o5:lar within o:r ra5idly
changing world.

Kevwords decision theorv. evolutionarv algorithms. mathematical programming. nature-
inspired meta-heuristics. operations research. optimization

INTRODUCTION
One aim oI this article is to ex5lore the merits and limitations oI two general a55roaches toward
o5timization. The Iirst a55roach is labeled in this 5a5er as Mathematical O5timization Techniq:es
(MOT). MOT reIers to a diverse set oI highly s:ccessI:l 5rogramming techniq:es that are
designed based on known (or ass:med) mathematical relationshi5s within an o5timization
5roblem. Using well-gro:nded mathematical theory, MOT ex5loit these 5roblem characteristics to
search Ior sol:tions or constr:ct sol:tions, sometimes with a g:arantee oI o5timality. To be
:tilized, each class oI MOT algorithms req:ires a set oI corres5onding conditions be met (e.g.
linearity, convexity) or req:ires certain a priori knowledge abo:t the 5roblem to be solved.
A second a55roach to o5timization is labeled in this 5a5er as nat:re-ins5ired meta-he:ristics or
NIM. NIM algorithms have been develo5ed :sing ins5iration Irom nat:ral systems that dis5lay
5roblem-solving ca5abilities s:ch as ants, swarm behavior, the imm:ne system, genetic evol:tion,
and non-inherited learning. The most 5o5:lar NIM techniq:es have been a55lied to an extensive
2
set oI o5timization 5roblems incl:ding many that deIy characterization within MOT-established
5roblem classes. With the ins5iration Ior each NIM algorithm being :nrelated to 5roblem
characteristics, it is common Ior these algorithms to be modiIied s:bstantially when a55lied in
5ractice. Im5lementation oI these algorithms ty5ically involves the integration oI tools taken Irom
MOT or other meta-he:ristics as well as the integration oI domain knowledge Irom s:biect matter
ex5erts |1|. Witho:t a strong theoretical Io:ndation or a clearly s5eciIied class oI 5roblems, it is
not always clear why 5artic:lar NIM techniq:es are more 5o5:lar than others, altho:gh it is
im5licitly ass:med that the long-term 5o5:larity oI each algorithm reIlects some degree oI
algorithm :tility.
In |2|, I 5resented Iindings Irom a st:dy that eval:ated o5timization # intensity over the last
several decades. Most notably, I 5resented evidence within 5:blication and 5atent trends that
nat:re-ins5ired algorithms are now more Ireq:ently im5lemented than MOT algorithms. The
5rimary aim oI this article is to 5ro5ose and disc:ss contrib:ting Iactors Ior this recent growth in
NIM 5o5:larity.
In inter5reting the res:lting in |2|, it was concl:ded that historical bias cannot acco:nt Ior the
trends re5orted in that st:dy, while algorithm :tility is likely to be a signiIicant contrib:ting Iactor
in NIM :sage. However, this nat:rally raises the q:estion oI why NIM might be more :seI:l in
today`s 5roblems. The next section reviews ex5lanations oI NIM :tility Irom the literat:re. I then
ex5lore and ex5and :5on the hy5othesis that the Ilexibility oI an algorithm Iramework d:ring
hybridization and c:stomization is a signiIicant contrib:tor to the Ireq:ency oI algorithm :sage.
Section 0 reviews trends taking 5lace in ind:stry and society, which I :se to make tentative
5redictions abo:t how algorithm :tility will be eval:ated in I:t:re o5timization research. In
Section 0, I o:tline a theoretical basis Ior algorithm Ilexibility and disc:ss the relationshi5 between
these ideas and those develo5ed in com5lexity science and theoretical biology. Concl:sions are
given in Section 5.
Theories on AIgorithm UtiIity
Early theories oI NIM :tility Ioc:sed on Iitness landsca5e Ieat:res or theories related to the
o5eration oI genetic algorithms, e.g. schema theory |3| and the b:ilding block hy5othesis |4|. For
instance, genetic algorithms were oIten to:ted Ior their (relatively) low sensitivity to 5roblem
Ieat:res s:ch as discontin:ities in the Iitness landsca5e, non-Ga:ssian noise in obiective I:nction
3
meas:rements, non-stationarity oI a 5roblem, errors in determining obiective I:nction gradients,
and n:merical ro:nding errors in com5:ter calc:lations |5-7|. PerIormance within m:lti-obiective
and m:ltimodal 5roblems has also been commonly cited when listing the advantages oI NIM.
Many NIM involve 5o5:lation-based search and readily beneIits Irom the distrib:ted com5:ting
reso:rces increasingly available in ind:stry and academia. Some st:dies have investigated
conditions :nder which common NIM design Ieat:res s:ch as 5o5:lation-based search |8| and
recombination |9| are beneIicial, or have investigated the conditions when 5roblems are diIIic:lt
Ior a 5artic:lar ty5e oI search algorithm |10|. Still others have aimed to derive a better
:nderstanding oI NIM by investigating the search characteristics and convergence 5ro5erties oI an
algorithm |11-18|. Altho:gh many oI these st:dies have aIIorded theoretical insights, the st:dies
are oIten by necessity restricted to very s5eciIic algorithms and s5eciIic 5roblems. UnIort:nately
this limits their ex5lanatory 5ower when trying to :nderstand why NIM techniq:es are a55lied
across a diverse range oI 5roblems.
When considering the 5ractical challenges Iaced in a55lying an algorithm to a new 5roblem,
another narrative s:rro:nding the s:ccess oI NIM oIten arises based on the sim5le conce5t oI
algorithm Ilexibility. As is common knowledge by 5ractitioners within the Iield, an NIM`s s:ccess
or Iail:re de5ends :5on the designer`s ability to integrate domain knowledge into the algorithm
design and generally c:stomize the algorithm to handle the 5artic:lar needs oI the c:stomer and
5roblem. The im5ortance oI c:stomization to the s:ccess oI a GA has been thoro:ghly
doc:mented over the last 15 years within monogra5hs, reviews and individ:al st:dies |19-29|. In
the 5artic:lar case oI GA a55lied to ind:strial sched:ling 5roblems, reviewed in |2|, it is not
s:r5rising that most s:ccessI:l case st:dies involve a c:stom GA or a memetic algorithm.
The conce5t oI algorithm Ilexibility and its relation to :tility is conce5t:ally sim5le and is
ill:strated in Fig:re 1. In short, the :tility oI an algorithm, i.e. the ability to generate sol:tions that
are :seI:l to a client, is inIl:enced by the ability to ada5t the algorithm to the :niq:e attrib:tes oI a
5roblem and not sim5ly based on 'oII the shelI 5erIormance. Ada5tation in this context reIers to
the selective retention oI a seq:ence oI algorithm design changes that im5rove an algorithm`s
5erIormance (e.g. Iinal sol:tion q:ality) on a 5roblem. Flexibility then is a reIlection oI relaxed
design constraints that allow Ior the integration oI c:stomized ro:tines that ex5loit 5roblem
knowledge witho:t negatively im5acting on the algorithm`s ability to generate sol:tions.
4
SurvivaI of the fIexibIe: a key factor in aIgorithm
utiIity
The idea that algorithm design Ilexibility is im5ortant to NIM :tility is not new and in many
ways is a sim5le and int:itive conce5t (e.g. |25, 26, 29|). However, little eIIort has been devoted
to ex5loring its theoretical basis or its 5ractical im5lications Ior the Iield. For instance, there has
been little investigation oI the 5recise conditions in which design Ilexibility is most relevant to
algorithm :tility or the conseq:ences that an em5hasis on Ilexibility might have Ior I:t:re
algorithm research. As disc:ssed thro:gho:t this article, many oI these iss:es can be ill:minated
Irom the 5ers5ective oI a 5roblem`s liIecycle.
BeIore 5roceeding it is worth 5ointing o:t that iss:es related to Ilexibility also arise in the st:dy
oI dynamic o5timization, e.g. see |30, 31|. While there are theoretical similarities between these
to5ics, the c:rrent disc:ssion Ioc:ses on changes in a 5roblem (e.g. search s5ace dimensionality,
Ieasibility constraints) that occ:r d:ring its liIecycle and where s:IIicient algorithm modiIications
cannot (at 5resent) be a:tomated b:t instead req:ire h:man creativity and insight. In short, this
disc:ssion considers changes in a 5roblem deIinition that are more s:bstantial than what wo:ld
normally be considered as a non-stationary 5roblem.
ReIevant timescaIes in aIgorithm design adaptation
Stochastic search 5rocesses exhibit a trade-oII between sol:tion q:ality and com5:tational costs.
Similarly, the 5erIormance oI an algorithm Iramework is ex5ected to dis5lay a trade-oII between
sol:tion q:ality and the amo:nt oI time ex5ended on algorithm design ada5tation. To :nderstand
Ilexibility, it is th:s necessary to acco:nt Ior the eIIiciency and eIIicacy oI the design ada5tation
5rocess (Fig:re 1b). As elaborated on in this section, eIIiciency becomes relevant when
deadlines constrain algorithm develo5ment time and when the 5roblem being solved is
s:sce5tible to changes in deIinition within its liIecycle that necessitate changes in algorithm
design.
5

igure 1 Different perspectives for assessing algorithm utility. Panel a) At small time scales.
the utility of an algorithm is viewed from the perspective of a static algorithm design. Under
these conditions. classical explanations of algorithm utility can be divided into fitness
landscape (top) and computational resource (bottom) arguments. %45: Popular algorithms
are effective for problems with problem characteristics that are common. or are becoming
increasingly so. for so called ~real world problems. In the diagram. the distance between an
algorithm (box) and problem (circle) indicates the suitability of the pairing. 49942:
Algorithms often have different cost-benefit profiles. such that the preferred technique can
sometimes depend on the computational resources available. The blue arrow indicates
current trends in the availability of computational resources and provides a conceptual
illustration of how this could influence views of algorithm utility. Panel b) rom a broader
perspective. algorithm utility is influenced by an algorithm framework`s adaptability
towards different problems. %45: Illustration of how algorithm flexibility influences utility.
49942: Algorithm adaptation profiles which show hypothetical values of solution quality as
a function of the amount of time given to algorithm development. The blue arrow indicates
the general trend of smaller available algorithm development times (see Section 3.2).


To ill:strate how a 5roblem`s liIecycle inIl:ences the relevance oI design Ilexibility, I introd:ce
three timescales: (T1) algorithm runtime; the time needed to reach a sto55ing criteria d:ring a
search 5rocess, (T2) algorithm development time; the time allotted to design an algorithm Ior a
5roblem, and (T3) problem lifespan; the amo:nt oI time that a 5roblem is relevant to a client.
6
A 5artic:larly im5ortant 5oint that I want to make in this disc:ssion is that the initial
5erIormance oI an algorithm on a 5artic:lar 5roblem is not act:ally im5ortant so long as eIIective
design changes can be discovered within the allotted design time (T2). S:ch design ada5tation
ca5abilities will de5end in 5art on the variety oI design change o5tions that are available as well as
the n:mber oI design change attem5ts that can be considered (~T2/T1). The im5ortance oI
algorithm design ada5tation is inIl:enced by several Iactors. For instance, a short 5roblem liIes5an
(T3) can 5lace considerable 5ress:re on T2 while the extent that a new 5roblem Iorm:lation
deviates Irom 5revio:s Iorm:lations will hel5 to determine the im5ortance and meaning oI ra5id
and eIIective ada5tation. Each oI these Iactors s:bstantial changes in 5roblem Iorm:lation, small
T3, and large T2/T3 all Iavor algorithmic Irameworks that q:ickly ada5t to new conditions, e.g.
movement to the leIt in the bottom gra5h in Fig:re 1b.
Partic:lar details will nat:rally de5end on the ty5e oI 5roblem considered. For instance, the
meaning oI T3 will de5end on whether a 5roblem is solved once (e.g. most design 5roblems) or
many times, e.g. in sched:ling. II solved once and a sol:tion can be reached at any time d:ring T3,
then T3 will 5lace a straightIorward constraint on algorithm :tility, e.g. T2 T3. When 5roblems
are re5eatedly solved, algorithm :tility might be estimated by sol:tion q:ality (e.g. cost)
im5rovement over other algorithms m:lti5lied by the time it is im5lemented, e.g. Asol:tion
q:ality} x T3-T2}. However when T3 is small, ra5id im5lementation can be im5ortant d:e to the
val:e oI being Iirst to market or val:e Irom avoiding bottlenecks within a larger 5roiect. Under
these conditions, the ra5id design oI s:IIicient algorithms can tr:m5 a better 5erIorming algorithm
that takes a longer time to develo5 |1|.
Algorithm adaptation during and after development
The 5roblem liIecycle has th:s Iar been described as having a liIes5an over which 5roblems are
relevant and a time window when algorithm develo5ment takes 5lace. What is observed in 5ractice
is ty5ically more com5licated. By looking closely at the individ:al com5onents oI a 5roblem
liIecycle, additional advantages Irom algorithm Ilexibility can be revealed.
First, as5ects oI a 5roblem (e.g. constraints, 5roblem Iorm:lation, even obiectives) can change
over the co:rse oI an algorithm develo5ment 5roiect. The reasons Ior these changes are varied |1|.
Partic:larly Ior new 5roblems, it is common to learn more abo:t the :nderlying nat:re oI the
5roblem, and conseq:ently want to change the 5roblem deIinition, as one develo5s ways to solve
it. In addition, s:rrogate models oI a 5roblem can change iI a client decides that the original
7
5roblem deIinition is no longer satisIactory, e.g. d:e to changes in a market5lace, man:Iact:ring
Iacility, 5ersonnel, raw materials, etc. A client`s tr:e interests are not always ca5t:red by a well
deIined 5roblem Iorm:lation and can involve a network oI interde5endent s:b-5roblems and soIt
obiectives that exist as tacit domain knowledge. Altho:gh sometimes Ir:strating to o5timization
algorithm develo5ers, early s:ccess d:ring algorithm develo5ment can also breed a desire Ior
change, e.g. a desire to ex5and the sco5e oI the 5roblem. However, it is worth noting that a change
in the 5roblem deIinition does not necessarily reIlect 5oor 5lanning or 5oor :nderstanding by the
client. Instead, well-regarded insights Irom management science s:ggest that these 5roblem
changes are likely a conseq:ence oI intelligent yet boundedlv rational individ:als attem5ting to
make sense oI their dynamic and com5lex world (cI |32, 33|). These arg:ments s:ggest that
changes to a 5roblem d:ring algorithm develo5ment are not always 5reventable and are likely to
be a 5ersistent Ieat:re in I:t:re o5timization contexts.
Changes to a 5roblem also occ:r Ior reasons o:tside the control oI the client and may take 5lace
aIter an algorithm is already im5lemented, e.g. see |6|. Ty5ical reasons Ior this incl:de :nex5ected
changes in a market or in the internal o5erating conditions oI a Iirm |1|.
In s:mmary, 5roblems can change d:ring and aIter the time that might have been allocated to
algorithm develo5ment. When this occ:rs, an algorithm m:st eIIectively ada5t and do so q:ickly
eno:gh to kee5 :5 with changing req:irements, e.g. oI a client d:ring algorithm develo5ment or a
market d:ring algorithm im5lementation. Under these circ:mstances, an algorithmic Iramework
whose general s:itability de5ends greatly on the conditions set o:t in the original 5roblem
deIinition might Iind itselI less able to accommodate new :nex5ected conditions that arise.
A Dynamic and VoIatiIe MarketpIace
The accelerating 5ace oI technological and organizational change has been doc:mented :sing
several s:rrogate metrics Ior technological 5rogress, innovation, and man:Iact:ring activity |34-
42|. The rate oI change is oIten not steady however and analogies have been made between
organizational, technological, and market changes and the 5:nct:ated eq:ilibria oI biological
evol:tion, i.e. where 5eriods oI a55arent stasis are Iollowed by ra5id change |43, 44|.
Technological 5rogress and diversiIication is 5artic:larly ra5id in inIormation technology |37,
38|, energy sectors |36|, biochemical and 5etrochemical ind:stries |39|, and some man:Iact:ring
sectors |40-42|. Altho:gh ra5id ex5ansion is not :niversally observed across ind:stries, it is
5artic:larly ac:te in domains where o5timization algorithms are oIten a55lied, i.e. those sensitive
8
to innovations in ICT.
Technological innovations can drive growth, however they can also be disr:5tive. There is
considerable evidence that 5rod:ct liIecycles are shrinking and that this is res:lting in a
s:bseq:ent 5:sh Ior shorter 5rod:ct develo5ment times across many sectors |45-51|. In res5onse,
research on the com5etitive advantage oI Iirms 5laces considerable em5hasis on the val:e derived
Irom ra5id 5rod:ct develo5ment, innovation s5eed and ada5tation in changing markets. This
em5hasis is largely organized aro:nd the st:dy oI 'dynamic ca5abilities and time-based
com5etition |53-60|. The changing b:siness conditions i:st described are likely to have a direct
bearing on the :tility oI I:t:re algorithm Irameworks. An overview oI these global trends and
others that are relevant to o5timization research is given in Fig:re 2.

igure 2 Panel a) Trends and interactions between optimization timescales. The amount of
time needed for an algorithm to search for a solution (T
1
) is decreasing due to technological
progress. As a result. a smaller proportion of algorithm development time (T
2
) is spent with
the algorithm running and more is spent on algorithm design changes. Because a problem`s
lifespan (T
3
) is decreasing and problem definition volatility is increasing. the available time
to make algorithm design changes is becoming more constrained. Panel b) Major trends
that have a direct bearing on optimization research.

Given these trends, the n:mber oI new o5timization 5roblems is ex5ected to increase within
ind:stries that are growing and diversiIying in res5onse to technology and market changes.
Viewed as an ecosystem oI ex5anding and diversiIying reso:rces, the algorithms most oIten
im5lemented are likely to be those that can most q:ickly ex5loit these diverse reso:rces. In other
words, algorithmic 5aradigms that are the most Ilexible to new conditions and can contrib:te to an
organization`s time-based com5etitive advantage are more likely to be :tilized.
The second maior trend is one oI growing volatility in extant 5roblems: Ior many ind:stries the
antici5ation oI I:t:re conditions (e.g. in organizational ca5abilities, reso:rces, markets,
com5etitors) is becoming more :ncertain. In addition, the 5roblems that a Iirm wants solved today
9
might only 5artially resemble the relevant 5roblems oI the I:t:re. In s:ch volatile environments,
the :tility oI an algorithm Iramework will not be derived Irom the ability to solve a static 5roblem.
Instead it will be the ability to ada5t to changing 5roblem conditions that is likely to deIine the
s:ccess or Iail:re Ior many o5timization a55lications.
Design FIexibiIity in NIM and MOT
MOT research decom5oses the world oI o5timization 5roblems into mathematically tractable
domains involving 5recise ass:m5tions and well-deIined 5roblem classes. The rationale Ior this
decom5osition is straightIorward; an algorithm can be a55licable to many 5roblems meeting the
req:isite conditions. It sho:ld be stressed that when s:ch conditions are satisIied, MOT oIten
generate s:5erior sol:tions com5ared with NIM algorithms. However, these constraints on
algorithm-5roblem s:itability take on new meaning Ior 5roblems that change s:bstantially d:ring
their liIecycle. MOT s:itability will req:ire both c:rrent and I:t:re 5roblem Iorm:lations to meet
s5eciIic conditions and iI changes to a 5roblem Iorm:lation eliminate mathematical reg:larities
ex5loited by the MOT, then this will limit the MOT`s :tility.
On the other hand, beca:se NIM algorithm design modiIication is common5lace, an algorithm
Iramework`s 5o5:larity m:st rely on m:lti5le s:ccesses in diIIerent contexts involving diIIerent
algorithm variants. Under these circ:mstances, long-term NIM algorithm 5o5:larity is less likely
to reIlect the 5erIormance oI the canonical algorithm and instead more likely reIlects s:ccess in
algorithm design modiIication across 5roblem contexts (see Fig:re 1b). Based on these arg:ments,
it is 5la:sible that s:ccessI:l NIM are readily ada5ted to the :niq:e attrib:tes oI diIIerent
5roblems.
Towards a theory of aIgorithm fIexibiIity
Altho:gh the conce5t oI algorithm Ilexibility is straightIorward, the conditions that determine
whether an algorithm is Ilexible are m:ch less obvio:s and need to be ex5lored in greater detail.
Along these lines, I Ieel it is im5ortant to make 5rogress in answering the Iollowing (related)
q:estions:
Adaptive plasticity to environmental (problem) context: What general conditions make
it easy/diIIic:lt to incor5orate domain knowledge into an algorithm.
#obustness to internal (algorithm) context: Are there general conditions where the
incl:sion oI a 5artic:lar o5erator or a design change has a catastro5hic im5act on other
10
im5ortant search characteristics oI the algorithm?
rigins of design innovation (Exaptation): When is it 5ossible to im5lement algorithm
'b:ilding blocks in new ways to achieve a more eIIective search 5rocess Ior a s5eciIic
5roblem?

Below I 5ro5ose some q:alitative attrib:tes that one might ex5ect in an algorithm that is
ada5table to diIIerent o5timization contexts.
robust yet adaptable behavior: Partic:lar search characteristics can be achieved thro:gh
n:mero:s distinct algorithm Iorm:lations (rob:st, many-to-one ma55ing) yet at the same
time these search characteristics can be changed and Iine-t:ned when needed (ada5table,
one-to-many ma55ing).
modularity and loose coupling: There are Iew req:irements that one Ieat:re oI the
algorithm design 5laces on other design Ieat:res or on the 5roblem deIinition. This might
be viewed as the 5resence oI I:nctional enca5s:lation and virt:al 5rotocols within
algorithm search behavior.
#esponsive: Algorithm changes are easy to make and easy to test. Learning by doing is
ra5id s:ch that the time needed to ada5t the algorithm to a local context is Iast eno:gh to
make learning by doing a viable a55roach. Also, design im5rovements can be discovered
witho:t :nderstanding the mechanics oI the algorithm.
eedback: UseI:l Ieedback inIormation is available that 5rovides g:idance abo:t what
as5ects oI the algorithm design may need to change.
essons from nature
The q:alities oI an ada5tive algorithm Iramework listed above describe several Ieat:res that are
relevant to ada5tation in biology. For instance, in a review by Kirschner and Gerhart |62|, they
highlight mod:larity, a loose reg:latory co:5ling, 5rotocols, I:nctional versatility, and ex5loratory
behavior as being highly relevant to the evolvability oI biological systems. A rob:st yet Ilexible
algorithmic core that can broadly ada5t to diIIerent 5roblem conditions 5rovides the basis oI o:r
conce5t:al :nderstanding oI algorithm Ilexibility (Fig:re 5b) and shares many similarities with
observations oI biological evol:tion |62|. For instance, altho:gh individ:al s5ecies are highly
so5histicated s5ecialists o5erating within :niq:e habitats, most s5ecies share a set oI conserved
core 5rocesses |62, 63|. Trait diIIerences between many com5lex s5ecies can largely be attrib:ted
11
to diIIerences in the time and 5lace in which I:nctions are exec:ted while there are relatively
Iewer diIIerences in the obiects (enzymes, com5artments, cells) 5erIorming the 5artic:lar
I:nctions.
Finally, it is worth mentioning that there has been some 5rogress in :nderstanding the
relationshi5 between rob:stness, Ilexibility, and evolvability in the context oI biological evol:tion
and artiIicial liIe |64-67|. These develo5ments might event:ally 5rovide new design 5rinci5les that
can contrib:te to the develo5ment oI Ilexible and rob:st algorithms. For instance, recent st:dies
have indicated that degeneracy is an im5ortant contrib:tor to the rob:stness and evolvability oI
biological systems |66, 67|. egeneracy reIers to conditions in which m:lti-I:nctional
com5onents within a system can be I:nctionally interchangeable in certain contexts, yet 5rovide
:niq:e I:nctions in other contexts. In other words, when degeneracy is 5resent there are many
diIIerent ways to achieve a 5artic:lar I:nctional o:tcome, however these same 5rocesses can
a55ear to have highly diverse o:tcomes within new environments, th:s 5roviding a basis Ior
selective diIIerences that are revealed by 5artic:lar environmental contexts |68|. I have 5resented
arg:ments and sim:lations to demonstrate how these conce5ts can be a55lied in man:Iact:ring
assembly systems |69|, strategic 5lanning Ior land vehicle Ileets |70|, and in certain ty5es oI
dynamic o5timization 5roblems involving m:lti-agent systems |71|. While it is not yet clear how
these conce5ts can be a55lied to im5rove :5on algorithm Irameworks, these theoretical
develo5ments might :ltimately 5rove relevant to I:t:re algorithm research, 5artic:larly iI the
5ro5erties that Iacilitate evolvability in biological systems are widely a55licable to other com5lex
ada5tive systems.
ConcIusions
O5timization 5roblems are not traditionally tho:ght oI as having an ex5iration date. However,
waning are the days when a commercial o5timization 5roblem can be deIined and st:died Ior
many years witho:t the 5roblem changing. More and more in today`s com5anies, new 5roblems
ra5idly come into existence and existing 5roblems :nex5ectedly change. Under these conditions,
sol:tion q:ality de5ends on ra5id algorithm develo5ment and an algorithm`s ca5acity to
accommodate new inIormation.
In this 5a5er I 5ro5osed that the dominance oI nat:re-ins5ired meta-he:ristics is 5artly d:e to
their ca5acity to be eIIiciently and eIIectively modiIied to Iit the characteristics oI a 5roblem. In a
12
volatile and dynamic world, the 5o5:larity oI these algorithms might have less to do with the
eIIicacy oI a 5artic:lar algorithm on a 5artic:lar set oI 5roblems and more to do with the ability to
incor5orate domain knowledge q:ickly and to be advantageo:sly combined with other methods.
Under these volatile conditions, it is im5ortant Ior :s to gain a dee5er :nderstanding oI the basic
5rinci5les that allow Ior ada5tation in com5lex systems. #ecent insights in com5lexity science
and systems biology may 5rovide g:idance in the develo5ment oI algorithms with more Ilexible
designs Ior tackling the growing n:mber oI 5roblems with a short and volatile liIecycle.

References

|1| Z. Michalewicz and . B. Fogel, ow to solve it. modern heuristics: S5ringer-Verlag
New York 2004.
|2| J. M. Whitacre, "#ecent Trends Indicate #a5idly Growing ominance oI Nat:re-Ins5ired
O5timization in Academia and Ind:stry," Computing. (in 5ress).
|3| J. Holland, daptation in natural and artificial svstems: MIT 5ress Cambridge, MA,
1992.
|4| . E. Goldberg, Genetic algorithms in search. optimization and machine learning:
Addison-Wesley Longman P:blishing Co., Inc. Boston, MA, USA, 1989.
|5| Q. T. Pham, "EIIect oI N:merical Errors on the PerIormance oI O5timization Methods,"
5resented at the Proceedings oI Chemeca, Brisbane, A:stralia, 2005.
|6| . B. Fogel, "Introd:ction to evol:tionary com5:tation," Modern euristic Optimization
Techniques. Theorv and pplications to Power Svstems. 5. 1, 2007.
|7| Y. Jin and J. Branke, "Evol:tionary o5timization in :ncertain environments-a s:rvey,"
IEEE Transactions on Evolutionarv Computation. vol. 9, 55. 303-317, 2005.
|8| J. He and X. Yao, "From an individ:al to a 5o5:lation: An analysis oI the Iirst hitting
time oI 5o5:lation-based evol:tionary algorithms," IEEE Transactions on Evolutionarv
Computation. vol. 6, 55. 495-511, 2002.
|9| P. K. Lehre and X. Yao, "Crossover can be constr:ctive when com5:ting :niq:e in5:t
o:t5:t seq:ences," 2008, 55. 595604.
|10| J. He. et al., "A note on 5roblem diIIic:lty meas:res in black-box o5timization:
ClassiIication, realizations and 5redictability," Evolutionarv Computation. vol. 15, 55. 435-443,
2007.
|11| T. Blickle, "Theory oI Evol:tionary Algorithms and A55lication to System Synthesis,"
Swiss Federal Instit:te oI Technology, 1996.
|12| W. Wieczorek and Z. J. Czech, "Selection Schemes in Evol:tionary Algorithms,"
Proceedings of the Svmposium on Intelligent Information Svstems (IIS2002 ). 55. 185-194, 2002.
|13| E. Van Nimwegen and J. P. Cr:tchIield, "O5timizing E5ochal Evol:tionary Search:
Po5:lation-Size e5endent Theory," Machine Learning. vol. 45, 55. 77-114, 2001.
|14| T. Smith. et al., "Local evolvability oI statistically ne:tral GasNet robot controllers,"
Biosvstems. vol. 69, 55. 223-243, 2003.
|15| S. Niissen and T. Back, "An analysis oI the behavior oI sim5liIied evol:tionary
algorithms on tra5 I:nctions," IEEE Transactions on Evolutionarv Computation. vol. 7, 55. 11-22,
2003.
|16| . E. Goldberg and K. eb, "A Com5arative Analysis oI Selection Schemes Used in
Genetic Algorithms," Urbana. vol. 51, 55. 61801-2996.
|17| J. M. Whitacre. et al., "Making and breaking 5ower laws in evol:tionary algorithm
5o5:lation dynamics," Memetic Computing. vol. 1, 5. 125, 2009.
|18| F. Herrera. et al., "Tackling #eal-Coded Genetic Algorithms: O5erators and Tools Ior
Behavio:ral Analysis," rtificial Intelligence Review. vol. 12, 55. 265-319, 1998.
|19| A. Eiben and J. Smith, Introduction to evolutionarv computing: S5ringer Verlag, 2003.
|20| K. e Jong, Evolutionarv computation. a unified approach: The MIT Press, 2006.
|21| L. avis, andbook of Genetic lgorithms: Van Nostrand #einhold New York, 1991.
13
|22| . E. Goldberg and S. Voessner, "O5timizing global-local search hybrids," Urbana. vol.
51, 5. 61801, 1999.
|23| P. Merz and B. Freisleben, "A Com5arison oI Memetic Algorithms, Tab: Search, and Ant
Colonies Ior the Q:adratic Assignment Problem," in congress on evolutionarv computation, 1999,
55. 2063-2070.
|24| K. A. e Jong. et al., "Using Markov chains to analyze GAFOs," Foundations of genetic
algorithms. vol. 3, 55. 115-137, 1995.
|25| T. Back. et al., "Evol:tionary com5:tation: comments on the history and c:rrent state,"
IEEE Transactions on Evolutionarv Computation. vol. 1, 55. 3-17, 1997.
|26| Z. Michalewicz, "A hierarchy oI evol:tion 5rograms: An ex5erimental st:dy,"
Evolutionarv Computation. vol. 1, 55. 51-76, 1993.
|27| Z. Michalewicz, Genetic algorithms data structures evolution programs: S5ringer,
1996.
|28| P. P. Bonissone. et al., "Evol:tionary Algorithms omain Knowledge #eal-World
Evol:tionary Com5:tation," IEEE Transactions on Evolutionarv Computation. vol. 10, 5. 256,
2006.
|29| Z. Michalewicz, Genetic algorithms data structures: S5ringer, 1996.
|30| K. e Jong, "Evolving in a changing world," Lecture notes in computer science. 55. 512-
519, 1999.
|31| J. Branke and . C. MattIeld, "Antici5ation and Ilexibility in dynamic sched:ling,"
International Journal of Production Research. vol. 43, 55. 3103-3129, 2005.
|32| H. A. Simon, Behavioral Model of Rational Choice. Santa Monica: #and Cor5, 1953.
|33| K. E. Weick. et al., "Organizing and the 5rocess oI sensemaking," Organization Science.
vol. 16, 5. 409, 2005.
|34| #. Fre. et al., "Prod:ctivity growth, technical 5rogress, and eIIiciency change in
ind:strialized co:ntries," The merican Economic Review. 55. 66-83, 1994.
|35| #. K:rzweil, "The law oI accelerating ret:rns," KuzweilI. net. Retrieved Nov. vol. 24, 5.
2008, 2001.
|36| H. Koh and C. L. Magee, "A I:nctional a55roach Ior st:dying technological 5rogress:
extension to energy technology," Technological Forecasting and Social Change. vol. 75, 55. 735-
758, 2008.
|37| J. B. Waldner, Nanocomputers and swarm intelligence: Wiley-ISTE, 2008.
|38| C. Walter, "Kryder's law," Scientific merican. vol. 293, 5. 32, 2005.
|39| B. Achilladelis. et al., "The dynamics oI technological innovation: The case oI the
chemical ind:stry* 1," Research Policv. vol. 19, 55. 1-34, 1990.
|40| L. Argote and . E55le, "Learning c:rves in man:Iact:ring," Science. vol. 247, 55. 920-
924, 1990.
|41| S. Alberth, "Forecasting technology costs via the ex5erience c:rveMyth or magic?,"
Technological Forecasting & Social Change. vol. 75, 55. 952-983, 2008.
|42| C. Harmon, "Ex5erience c:rves oI 5hotovoltaic technology," Laxenburg. IIS. pp. vol.
17, 2000.
|43| A. Sood and G. J. Tellis, "Technological evol:tion and radical innovation," Journal of
Marketing. vol. 69, 55. 152-168, 2005.
|44| C. J. G. Gersick, "#evol:tionary change theories: A m:ltilevel ex5loration oI the
5:nct:ated eq:ilibri:m 5aradigm," The academv of management review. vol. 16, 55. 10-36, 1991.
|45| A. GriIIin, "Metrics Ior meas:ring 5rod:ct develo5ment cycle time," Journal of Product
Innovation Management. vol. 10, 55. 112-125, 1993.
|46| M. . #osena: Jr, "S5eeding yo:r new 5rod:ct to market," Journal of consumer
Marketing. vol. 5, 55. 23-36, 1988.
|47| W. Q:alls. et al., "Shortening oI the PLC: an em5irical test," The Journal of Marketing.
vol. 45, 55. 76-80, 1981.
|48| #. A. Bettis and M. A. Hitt, "The new com5etitive landsca5e," Strategic Management
Journal. vol. 16, 55. 7-19, 1995.
|49| M. CrawIord, "The hidden costs oI accelerated 5rod:ct develo5ment," Journal of Product
Innovation Management. vol. 9, 55. 188-199, 1992.
|50| M. #. Millson. et al., "A s:rvey oI maior a55roaches Ior accelerating new 5rod:ct
develo5ment," Journal of Product Innovation Management. vol. 9, 55. 53-69, 1992.
|51| A. L. Page, "Assessing new 5rod:ct develo5ment 5ractices and 5erIormance: establishing
cr:cial norms," Journal of Product Innovation Management. vol. 10, 55. 273-290, 1993.
|52| B. L. Bay:s, "Are 5rod:ct liIe cycles really getting shorter?," Journal of Product
Innovation Management. vol. 11, 55. 300-308, 1994.
14
|53| G. Stalk, "Timethe next so:rce oI com5etitive advantage," arvard Business Review.
vol. 66, 55. 41-51, 1988.
|54| G. Stalk and T. M. Ho:t, Competing against time: Free 5ress New York, 1990.
|55| E. H. Kessler and A. K. Chakrabarti, "Innovation s5eed: a conce5t:al model oI context,
antecedents, and o:tcomes," The cademv of Management Review. vol. 21, 55. 1143-1191, 1996.
|56| S. A. Zahra. et al., "Entre5rene:rshi5 and dynamic ca5abilities: a review, model and
research agenda," JOURNL OF MNGEMENT STUDIES-OXFORD-. vol. 43, 5. 917, 2006.
|57| C. E. HelIat and M. A. PeteraI, "The dynamic reso:rce-based view: Ca5ability
liIecycles," Strategic Management Journal. vol. 24, 55. 997-1010, 2003.
|58| . J. Teece, "Ex5licating dynamic ca5abilities: the nat:re and microIo:ndations oI
(s:stainable) enter5rise 5erIormance," Business Week. 5. 64, 2004.
|59| K. M. Eisenhardt and J. A. Martin, "ynamic ca5abilities: what are they?," Strategic
management iournal. 55. 1105-1121, 2000.
|60| K. Eisenhardt and B. N. Tabrizi, "Accelerating Ada5tive Processes: Prod:ct Innovation in
the Global Com5:ter Ind:stry," dministrative Science Quarterlv. vol. 40, 1995.
|61| A. E. Eiben and M. Jelasity, "A critical note on ex5erimental research methodology in
EC," 2002, 55. 582587.
|62| M. Kirschner and J. Gerhart, "Evolvability," Proceedings of the National cademv of
Sciences. US. vol. 95, 55. 8420-8427, 1998.
|63| J. Gerhart and M. Kirschner, "The theory oI Iacilitated variation," Proceedings of the
National cademv of Sciences. vol. 104, 5. 8582, 2007.
|64| S. Ciliberti. et al., "Innovation and rob:stness in com5lex reg:latory gene networks,"
Proceedings of the National cademv of Sciences. US. vol. 104, 55. 13591-13596, 2007.
|65| A. Wagner, "#ob:stness and evolvability: a 5aradox resolved," Proceedings of the Roval
Societv of London. Series B. Biological Sciences. vol. 275, 55. 91-100, 2008.
|66| J. M. Whitacre and A. Bender, "egeneracy: a design 5rinci5le Ior achieving rob:stness
and evolvability," Journal of Theoretical Biologv. vol. 263, 55. 143-53, Mar 7 2010.
|67| J. M. Whitacre and A. Bender, "Networked b:IIering: a basic mechanism Ior distrib:ted
rob:stness in com5lex ada5tive systems," Theoretical Biologv and Medical Modelling vol. 7, 15
J:ne 2010 2010.
|68| G. M. Edelman and J. A. Gally, "egeneracy and com5lexity in biological systems,"
Proceedings of the National cademv of Sciences. US. vol. 98, 55. 13763-13768, 2001.
|69| #. Frei and J. M. Whitacre, "egeneracy and Networked B:IIering: 5rinci5les Ior
s:55orting emergent evolvability in agile man:Iact:ring systems," Journal of Natural Computing -
Special Issue on Emergent Engineering. (in 5ress).
|70| J. M. Whitacre. et al., "Evol:tionary Mechanics: new engineering 5rinci5les Ior the
emergence oI Ilexibility in a dynamic and :ncertain world
(htt5://www.box.net/shared/l56kcd62:k)," Natural Computing. (in 5ress).
|71| J. M. Whitacre. et al., "The role oI degenerate rob:stness in the evolvability oI m:lti-
agent systems in dynamic environments," in PPSN XI, Krakow, Poland, 2010, 55. 284-293.