Appl Intell (2008) 28: 101–119
DOI 10.1007/s10489-007-0044-4
A hybrid case adaptation approach for case-based
reasoning
Claudio A. Policastro · André C.P.L.F. Carvalho ·
Alexandre C.B. Delbem
Received: 14 July 2006 / Accepted: 19 March 2007 / Published online: 18 May 2007
© Springer Science+Business Media, LLC 2007
Abstract Case-Based Reasoning is a methodology for
problem solving based on past experiences. This methodol-
ogy tries to solve a new problem by retrieving and adapting
previously known solutions of similar problems. However,
retrieved solutions, in general, require adaptations in order
to be applied to new contexts. One of the major challenges
in Case-Based Reasoning is the development of an efficient
methodology for case adaptation. The most widely used
form of adaptation employs hand coded adaptation rules,
which demands a significant knowledge acquisition and en-
gineering effort. An alternative to overcome the difficulties
associated with the acquisition of knowledge for case adap-
tation has been the use of hybrid approaches and automatic
learning algorithms for the acquisition of the knowledge
used for the adaptation. We investigate the use of hybrid ap-
proaches for case adaptation employing Machine Learning
algorithms. The approaches investigated how to automati-
cally learn adaptation knowledge from a case base and apply
it to adapt retrieved solutions. In order to verify the potential
of the proposed approaches, they are experimentally com-
pared with individual Machine Learning techniques. The re-
sults obtained indicate the potential of these approaches as
an efficient approach for acquiring case adaptation knowl-
edge. They show that the combination of Instance-Based
Learning and Inductive Learning paradigms and the use of
C.A. Policastro () · A.C.P.L.F. Carvalho · A.C.B. Delbem
University of São Paulo, São Paulo, Brazil
e-mail: capoli@icmc.usp.br
A.C.P.L.F. Carvalho
e-mail: andre@icmc.usp.br
A.C.B. Delbem
e-mail: acbd@icmc.usp.br
a data set of adaptation patterns yield adaptations of the re-
trieved solutions with high predictive accuracy.
Keywords Case-based reasoning · Case adaptation ·
Knowledge acquisition · Hybrid systems
1 Introduction
Case-Based Reasoning (CBR) is a methodology for problem
solving based on past experiences. This methodology tries to
solve a new problem by retrieving and adapting previously
known solutions of similar problems. Research in CBR is
directed by two main motivations [23]: The goal to model
human behaviors and the intent to develop new techniques
and construct more effective Artificial Intelligent systems.
When CBR systems are applied for the resolution of
problems, the retrieved solutions can rarely be directly used
as a solution for a new problem. They need to be adapted in
order to fit the requirements of the new situations. Therefore,
case adaptation is a desirable characteristic in CBR systems.
Many CBR systems avoid using adaptation at all and the
most widely used form of adaptation employs hand coded
adaptation rules, which demands a significant knowledge ac-
quisition effort for case adaptation, presenting several diffi-
culties [15, 16].
An alternative to overcome the difficulties in manually
acquiring adaptation knowledge has been the use of auto-
matic learning. Until now, there has been little work related
to automatically learning adaptation knowledge reported in
the literature. They can be found in [16, 24, 42, 43]. In these
publications, the authors propose different approaches for
extracting adaptation knowledge from a Case Base (CB).
This paper extends them by investigating a new algorithm
for automatic learning of adaptation knowledge from a CB
102
and by exploring the use of committees [9, 10, 37] of Ma-
chine Learning (ML) algorithms to perform automatic case
adaptation.
The proposed approaches, which are hybrid CBR sys-
tems, address the Parameter Adjustment adaptation process,
a strategy of substitutional adaptation [20], one of the most
frequently employed adaptation strategies for CBR systems.
In this article, we focus on numerical solution adaptation.
In order to show that automatic case adaptation may re-
sult in considerable gains in the prediction accuracy for a so-
lution attribute, we carry out a set of experiments comparing
the performance of the proposed hybrid CBR systems with a
CBR system employing the K-nearestneighbor (Knn) tech-
nique as a simple case adaptation mechanism. The experi-
ments carried out also compare the performance of the pro-
posed approach with several individual estimators generated
by the same ML algorithms employed in the proposed case
adaptation mechanism (CAE). Additionally, we show that
further performance gains can be obtained by substituting,
in the case adaptation mechanism, a single ML algorithm by
a committee of ML algorithms. The experiments carried out
also investigate the effect of different retrieval approaches in
the final adaptation result and the influence of the number of
similar cases considered in the generation of adaptation pat-
terns. The empirical evaluation suggests the potential of the
proposed approach by presenting experimental results where
it outperforms the individual ML techniques and CBR sys-
tems using Knn for case adaptation.
This paper is organized as follows: Sect. 2 briefly intro-
duces the CBR paradigm. Section 3 discusses related work.
Section 4 presents our hybrid system architecture and de-
scribes the experiments performed. Section 5 describes the
experimental results obtained. Finally, Sect. 6 presents our
conclusions.
2 Case-based reasoning
CBR is a methodology for problem solving based on past
experiences. It tries to solve a new problem by employing a
process of retrieval and adaptation of known solutions previ-
ously applied to similar problems. CBR systems are usually
described by a reasoning cycle (also named the CBR Cycle),
which has four main phases [1]:
1. Retrieval: according to a new problem provided by the
user, the CBR system retrieves, from a Case Base (CB),
previous cases similar to the new problem;
2. Reuse: the CBR system adapts a solution from a retrieved
case in order to fit the requirements of the new problem;
3. Revision: either the CBR system or the user revises the
solution generated by the reuse phase;
C.A. Policastro et al.
4. Retention: the CBR system may learn the new case by
incorporating it into the CB, which is named case learn-
ing. The fourth phase can be divided into the following
procedures: selection of relevant information to create a
new case, index composition for this case, and case in-
corporation into the CB.
CBR is not a technology developed for specific purposes,
but rather is a general methodology of reasoning and learn-
ing [1, 18, 41], differing in important aspects from other AI
paradigms [1]:
– CBR can use specific knowledge from previous problems;
– The reasoning from previous problems is a powerful strat-
egy for problem solving, inspired by human reasoning.
The CBR paradigm is supported by two main principles
[23]. The first principle says that the world is regular: simi-
lar problems have similar solutions. Consequently, solutions
from similar problems are a good starting point to solve new
problems. The second principle states that problems tend to
repeat. Therefore, new problems tend to be similar to previ-
ous problems.
Cases can be represented using a number of formalisms.
Depending on the attributes to be represented, a convenient
structure can easily be selected. The goal of a CBR sys-
tem strongly influences the content of its cases [11]. A case
may represent different types of knowledge and assume a
large variety of representations. Additionally, a case should
contain a certain information richness and complexity in its
structure [1]. In most CBR systems the cases are usually rep-
resented as an attribute-value vector, divided into two parts:
problem and solution. Figure 1 shows two examples for case
representation.
3 Case adaptation
When CBR systems are applied to real-world problems, the
retrieved solutions can rarely be directly used as suitable so-
lutions for a new problem. Retrieved solutions usually re-
quire a set of adaptations in order to be applied to new con-
texts. An adaptation process may be either as simple as the
substitution of a component (in this work, the case solution
attributes are named components) from the retrieved solu-
tion or as complex as a complete modification of the solution
structure. The adaptation can occur by inclusion, removal,
substitution or transformation of the components of a previ-
ous solution [20].
Several strategies for case adaptation have been proposed
in the literature [20, 35]. They can be classified in three main
groups: substitutional adaptation, transformational adapta-
tion and generative adaptation. Strategies for substitutional
adaptation exchange solution attribute values of the retrieved
solution with appropriate values, producing a new solution.
A hybrid case adaptation approach for case-based reasoning
Fig. 1 Two examples for case
representation: an
attribute-value vector and an
object-oriented case structure
Strategies for transformational adaptation modify the solu-
tion structure by including or removing components of the
retrieved solution in order to satisfy the requirements of the
new problem. Strategies for generative adaptation construct
a new solution from problem data using a predefined proce-
dure. When a new problem is presented, an adapted solution
is directly obtained by applying the procedure.
Case adaptation is one of the major challenges in CBR
[16, 43]. Several CBR systems avoid using adaptation due
to its complexity. The most widely used form of adaptation
employs hand coded adaptation rules, which demands a sig-
nificant effort of knowledge acquisition for case adaptation
[15, 16].
For example, in [22–24] a CBR system with a small set
of adaptation rules and a method for memory search is pro-
posed. When a new problem is presented to this system, it
retrieves a similar case and sends it to the adaptation mecha-
nism. The adaptation mechanism, in turn, selects an adapta-
tion rule and starts a search in the memory for components
able to substitute parts of the retrieved solution. These adap-
tation rules are hand coded knowledge packages acquired
specifically for a particular application domain.
Case adaptation knowledge is hard to acquire and de-
mands a significant knowledge engineering effort. An alter-
native to overcome these limitations in manually acquiring
adaptation knowledge has been the use of automatic learn-
ing, where case adaptation knowledge is extracted from pre-
viously obtained knowledge: the CB itself. Wilke and his
colleagues [42] present a framework for learning adaptation
knowledge with knowledge light approaches. This approach
uses knowledge already acquired and represented inside a
Case-Based Reasoning System. The knowledge source con-
tainers are: vocabulary, similarity measure, the case base,
and the adaptation container. The data from the knowledge
containers may be pre-processed into a suitable representa-
tion and then used by an inductive algorithm to produce the
adaptation knowledge.
103
Hanney [16] proposed an algorithm that automatically
acquires adaptation knowledge from a CB and represents
this knowledge as a set of adaptation rules. When a new
problem is presented to the CBR system, a case is retrieved
from the CB and sent to the adaptation mechanism. This
mechanism, in turn, extracts the differences between the
retrieved case and the new problem description. Next, it
searches in the adaptation rules set for proper rules for these
differences. Finally, the adaptation mechanism generalizes
the selected rules and applies them to the retrieved solution,
in order to obtain a new solution.
This approach employs domain-specific mechanisms due
the encoding employed for the adaptation rules and the ex-
traction of differences between description attributes and be-
tween solution component attributes. Additionally, unlike
the approach proposed that we present in this article, it does
not use existing ML algorithms for case adaptation.
Another related work, [28], presented an adaptation
heuristic, named the difference heuristic, for case-based es-
timation that does not rely on domain-specific rules. The
approach was implemented in a CBR system in which the
concept of case dominance plays an important role in check-
ing estimates based on the adaptation heuristic and in the
maintenance of consistency in the case library. The concept
of case dominance is adapted from multiple-criteria deci-
sion making, which eliminates dominated alternatives in the
screening of decision alternatives. To estimate the value of
a target case C1, the adaptation mechanism retrieves an ex-
isting case C2 which differs from C1 only in the value of
a single case attribute, named the distinguishing attribute.
Next, the adaptation mechanism retrieves two other cases,
C3, and C4, which also differ only in the value for the dis-
tinguishing attribute, such that C3 shares the distinguishing
attribute value with C1 and C4 with C2. Afterwards, an es-
timate for the value of the target case is given by: est(C1) =
est(C2) + est(C3) − est(C4), where est(C2), est(C3) and
104
est(C4) are the estimated values for C2, C3 and C4 in the
case library, respectively.
Although this approach does not rely on domain-specific
rules, it may be applied only under some circumstances or
assumptions.
In [43], the authors propose an inductive method for case
adaptation knowledge acquisition that employs the CB as a
knowledge source. The method consists of an algorithm that
executes comparisons between pairs of cases. For each pair,
it extracts the difference between the problem parts and the
difference between the solution parts. This algorithm uses
the extracted differences of the pairs of cases, together with
the description of the problem of the stored case as the cur-
rent problem, to construct a data set of standards that will
be used to train an inductive learning algorithm. The boost-
ing algorithm [14] is used to train a committee of Rise al-
gorithms [12]—a hybrid algorithm that combines induction
and learning based on instances—using the data set gener-
ated. A CBR system uses the committee of Rise algorithms
in the phase of case adaptation. When a new problem is
presented to the system, a previous case is retrieved and
the CBR system uses the committee of algorithms to de-
termine what adaptation actions to perform on the retrieved
case. This method is specific for design domains, due to the
specific encoding employed for the adaptation training pat-
terns and the extraction of differences between description
attributes and between solution component attributes.
In another paper, [8], a Radial Basis Function (RBF)
neural network [31] is employed for case adaptation. When
a new problem is presented to the CBR system, a set of simi-
lar previous cases are retrieved. These cases are used to train
the RBF network. After training, when a new problem de-
scription is presented to the network, it maps the problem
description to a solution.
These previous investigations explore the use inductive
learning to acquire general adaptation knowledge from ex-
amples and apply the acquired knowledge to automatic case
adaptation. In this paper, we instead describe a new, domain-
independent, approach for adaptation knowledge extraction
and automatic case adaptation based on the induction of
adaptation models by different ML algorithms.
4 Hybrid case-based reasoning system
The main contribution of this paper is a new methodology
for adaptation knowledge acquisition from a CB and the ex-
ploration of hybrid committees composed by estimators and
a combiner generated by different ML techniques to adapt
retrieved solutions.
4.1 Committee of machine learning algorithms
We investigates the use of a committee of ML algorithms
to perform automatic case adaptation. The committees in-
C.A. Policastro et al.
vestigated are composed of ML algorithms, based on dif-
ferent paradigms. The algorithms investigated are classical
ML algorithms and there is a large number of successful
experiments with them reported in the literature [21]. Each
algorithm (estimators) receives the same input, describing
a problem, and produces an individual output, describing a
solution. Additionally, a ML algorithm (combiner) receives
the output of the estimators and produces the final output
of the committee. The estimators and the combiner may be
used to accomplish adaptations in domains with either nu-
meric or nominal components of problems and solutions.
However, the experiments carried out in this paper employ
only domains with numeric solution attributes. The commit-
tees are composed of the following ML algorithms:
– Estimators—The Multi Layer Perceptron (MLP) neural
network [17]; the M5 regression tree learner [39]; the
Support Vector Machine (SVM) technique [38];
– Combiners—These same three ML algorithms were in-
vestigated for the Combiner.
In every case, the committee is always composed of three
estimators (MLP, SVM, and M5) and one combiner (MLP,
SVM, or M5). Although most of the approaches of com-
mittees proposed in the literature produce the final output
by voting or simple majority [9, 10, 37], the use of a ML
algorithm as a combiner may improve the accuracy of the
system by achieving a better partition of the decision space
produced by the estimators. The alternatives for the investi-
gated committees are illustrated in Fig. 2.
A MLP neural network is a common model for pattern
recognition. A MLP network includes one or more hidden
layers with nonlinear activation functions that carry out suc-
cessive nonlinear transformations on input patterns. Thus,
the intermediate layers can transform nonlinearly separable
problems into linearly separable ones [17].
M5 is a learning algorithm that generates models in the
form of regression trees combined with regression equations
(Model Tree) [39]. This model works similarly to a classi-
fication tree. However, the leaves contain linear expressions
instead of predicted values or classes. The Model Tree is
constructed by a divide-and-conquer approach that recur-
sively creates new nodes [33].
SVMs represent a family of learning algorithms based on
statistical learning theory [38]. They combine generalization
control with a technique that deals with the dimensionality
problem.1 This technique uses hyperplanes as decision sur-
faces and maximizes the separation margins between posi-
tive and negative classes. In order to achieve large margins,
1 Machine Learning algorithms can have poor performance when work-
ing on data sets with a high number of attributes. Attribute selection
techniques are frequently employed to reduce the dimensionality of
the original data set. A SVM is able to construct a good generalization
even for data sets with a large number of attributes.
A hybrid case adaptation approach for case-based reasoning
Fig. 2 Architecture of the
investigated committees
SVMs employ a statistical principle named structural risk
minimization [38]. Another key aspect of SVMs is the use of
kernels to build support vectors from the training set. These
support vectors are instances with equal distance from the
optimal hyperplane that divides the decision space.
4.2 Case retrieval approach
In this paper, to investigate the influence of the retrieval
mechanism on case adaptation performance, we compared
three different classical case retrieval approaches: one that
uses the KdTree structure [13], the Knn technique [27] and a
hybrid mechanism [32] employing an ART2 neural network
[7] and the Knn technique. In our experiments, all these re-
trieval approaches use the same weights for the attributes.
Thus, all attributes are considered to be equally important.
KdTree has been used for case retrieval in the CBR liter-
ature [23, 40]. It is a multi-dimensional generalization of the
one-dimension trees used for binary search, which facilitates
the storage and the search on groups of data of k dimensions.
This technique allows the integration of CBR with Induc-
tive Learning Algorithms, providing a retrieval mechanism
based on Decision Trees [4]. A KdTree recursively partitions
the data space into disjunct subspaces. Each node of the tree
represents a subspace of the original data space, including
the instances belonging to that subspace. The approach used
for the partitioning is the determination of the dimension for
which the separation margin is maximized. The subspaces
are recursively divided until the remaining subspaces con-
tain only one instance [4].
The K-nearest neighbor (Knn) algorithm [27] is the most
frequently used case retrieval mechanism in CBR systems
[23, 40]. Instance-based learning techniques are conceptu-
ally simple algorithms. The learning process employed by
Knn consists of the simple storage of the training instances
in a memory. When a new instance is presented to the sys-
tem, a group of k similar instances is retrieved from mem-
ory. The original version of the Knn algorithm assumes that
all stored instances correspond to points in n . For each new
instance, its closest neighbors are defined according to a dis-
tance metric, usually Euclidean distance, which is the metric
adopted in this paper.
The hybrid mechanism combines the ART2 neural net-
work with the Knn technique for case memory clustering
[32]. As a result, it speeds up the retrieval process. The
105
ART2 network performs incremental clustering by creating
new output nodes whenever the current output nodes cannot
properly represent a new example. The hybrid approach of
case indexing and retrieval contains two levels of memory
organization. The first level is formed by the output layer of
an ART2 network and is employed to organize clusters of
similar cases and to indicate the most similar cluster to the
problem currently presented, reducing the search space and
the retrieval time. The second level is a simple flat memory
that stores the instances of the cases. The cases stored in the
second level (flat memory) are linked to the clusters of the
first level (each case belongs to only one cluster).
4.3 Hybrid case adaptation approaches
Our proposed approach for case adaptation employs two
modules. The first module (ADAPTATION PATTERN
GENERATOR) produces a data set of adaptation rules. This
data set is used by the second module (CASE ADAPTA-
TION ENGINE) to train a committee of ML algorithms to
automatically perform case adaptation.
The first module construct the adaptation patterns data
set in the following way. Let x be a case stored in the CB
and yi , i = 1, . . . , k, one of the cases retrieved by the CBR
retrieval mechanism when the problem description associ-
ated with x is presented. As illustrated later, the attributes in
the problem description and solution components of x are
concatenated to the components of the problem description
of yi to compose the input attributes of the adaptation pat-
tern. The output attributes are the solution components of yi .
Next, the adaptation patterns are used to train the ML algo-
rithms that will produce the adaptation mechanism for the
second module. After training, the adaptation mechanism is
used to estimate the new component value of the retrieved
solution.
This approach assumes that a CB is representative [36],
i.e., the CB is a good representative sample of the target
problem space and the problem space does not change over
time. Therefore, no re-training of the adaptation mechanism
is required when the system creates new cases during the
reasoning process.
4.3.1 Adaptation pattern generator
The ADAPTATION PATTERN GENERATOR module can
extract implicit knowledge from a CB (see Fig. 3).
106
Fig. 3 The Adaptation Pattern
Generator algorithm
Initially, the function TestCaseExtract extracts a case
from the training set and uses it as a new problem (TestCase)
to be presented to the CBR system. The remaining cases
comprise a new CB without the test case (NewCB). Next,
the function DescriptionExtract extracts, from the test case,
the attributes of the problem (TestDescrpt) and the function
SolutionExtract extracts a component (indicated by Compo-
nent) of the solution (TestComponent). Next, the function
Retrieve, which may follow any of the retrieval approaches
presented in Sect. 4.2) returns the k most similar cases from
the NewCB (RetrievedCases), where k is a predefined value.
For each retrieved case, the attributes of the problem (RetDe-
scrpt) and a component of the corresponding solution (indi-
cated by Component) are extracted (RetComponent). Next,
the algorithm generates the adaptation patterns, as shown in
Fig. 5, using the function MakeAdaptationPattern, combin-
ing the following components of the cases:
– Input attributes: the problem description stored in the test
case; the problem description stored in the retrieved case;
one solution component stored in the retrieved case.
– Output attribute: one solution component stored in the
previous test case, related to the solution component in
the retrieved case.
Finally, the adaptation data sets produced are used to train
the committee of ML algorithms, which are employed as
adaptation mechanisms.
In the experiments carried out for each pattern adaptation
data set, initially, each of the three techniques (MLP, SVM
and M5) is individually trained. The trained techniques be-
come the estimators of the committees. Next, for each in-
stance, the output values produced by the estimators are
combined, creating a data set to be used by the combiners.
The adaptation mechanisms adapt the value of just one out-
put attribute (a single solution component). Therefore, one
independent pattern adaptation data set must be produced
for each solution component. This strategy preserves the in-
dependence of the approach from the structure of the cases.
Figure 4 shows the algorithm for the ADAPTATION
PATTERN GENERATOR, which is organized into blocks
C.A. Policastro et al.
of operations in order to illustrate time complexity calcu-
lus. In block A, the main operations are the extraction of
the problem description and solution (O(m)), and the con-
struction of the adaptation pattern (O(m)), where m is the
size of a case in the case base. Therefore, block A is O(m).
Block B performs all the inner operations of block A for all
k retrieved cases. Therefore, block B is k.O(m) = O(k.m).
The main steps from block C are: (1) the extraction of a test
case from the original case base; (2) the extraction of the
problem description and solution of the test case; (3) the re-
trieval of k similar cases from the case base; (4) operations
of Block B. In step 1, the algorithm accesses each case in the
order the cases are disposed in the case base, thus, each ac-
cess is O(1). Additionally, the extraction itself is O(m). As
a consequence, Step 1 is O(m). Step 2 obviously is O(m).
Step 3 is O(x), where x is the time complexity required for
the retrieval mechanism (knn, kd − tree, Hybrid). Step 4 is
O(k.m). Therefore, block C is O(m) + O(m) + O(x) +
O(k.m) = O(x + k.m). Since the algorithm performs all op-
erations of the block C for all cases in the original case base
(see block D), it is O(n).O(x + k.m) = O(x + k.m.n) in
the worst-case analysis. Thus, assuming that k and m are
constants for a CBR system, the time complexity of the
algorithm is O(x + n). Therefore, the time complexity of
our algorithm is limited by the adopted retrieval mechanism
(O(n) for Knn, and O(log n) for KdTree, for search and in-
sertion [29]).
Figure 5 shows an example of an adaptation pattern pro-
duced using one of the data sets employed in this article and
the training of the hybrid committee. This example shows
one adaptation pattern generated for the AG1 solution com-
ponent of the European data set.
4.3.2 Case adaptation engine
The case adaptation approaches investigated in this paper
learn the modifications that need to be performed to the val-
ues of the components of a retrieved solution in order to
achieve an adequate solution for a new problem. The most
important feature of the adaptation mechanism is the em-
ployment of implicit knowledge obtained from the CB with
A hybrid case adaptation approach for case-based reasoning
Fig. 4 ADAPTATION
PATTERN GENERATOR
algorithm. The blocks A, B, C,
and D represent blocks of
operations
a minimum knowledge acquisition effort. The case adapta-
tion process is described in Fig. 6.
When a new problem is presented to the CBR system,
the most similar case stored in the CB is retrieved by the re-
trieval mechanism. A retrieved case (RetrievedCase) is sent
to the adaptation engine together with the new problem de-
scription (Description). The adaptation engine, in turn, ex-
tracts the attributes from the new problem description (Ret-
Description). Next, the adaptation engine extracts the so-
lution component (RetSolution), indicated by Component,
and the function MakeInputPattern generates an adaptation
pattern for this component. Next, the function Adaptation-
Mechanism computes the new value of the component of the
retrieved solution (NewComponent), which is returned to the
system. When the case solution has more than one compo-
nent (like in the European data set), the adaptation engine is
called for each component. As a result, a set of values is re-
turned and composed to create the complete solution of the
new problem.
The main operations of this algorithm are the extraction
of the attributes from the retrieved case, the construction of
the input pattern with the extracted attributes, and the sub-
mission of the input pattern to the adaptation mechanism
(committee of ML techniques).
The algorithm receives as input the retrieved case, to-
gether with the description and the solution of the new prob-
lem. It extracts each attribute from the problem description
and solution of the retrieved case, taking O(m) for these
operations, where m is the size of a case in the case base.
Next, the algorithm combines the attributes to form the input
pattern, taking O(m) for this operation. Then the algorithm
submits the input pattern to the adaptation mechanism, tak-
ing O(x) for this operation, where x is the computational
complexity of the ML estimators used in the committee.
Each estimator has a different complexity, which is influ-
enced by the parameters optimized. For example, SVMs
have complexity O(n3 ) [6], training of MLP networks is
O(MW) for each epoch [34] (where M is the number of in-
stances and W is the number of weights), and training the
107
M5 algorithm has complexity O(n2 log n) [26]. The retrieval
techniques also have different complexities, which are O(n)
for Knn, and O(log n) for KdTree, for search and insertion
[29].
Therefore, the main operations of this algorithm are: the
operations on the attributes of the retrieved case: O(m), the
operations on the attributes of the new problem description:
O(m), and the adaptation of the attribute solution O(x).
Therefore, the algorithm takes O(x) in the worst case, where
its computational complexity is proportional to the complex-
ity of the ML algorithms employed in the hybrid committee
and the search techniques.
Figure 7 shows an example of the problem solving
process employing a trained committee. The example is re-
lated to the AG1 component of the European data set.
Figure 8 shows the general architecture of the proposed
hybrid system, highlighting the role of the algorithms in the
CBR Cycle.
5 Evaluation
This section presents the main results from a set of experi-
ments carried out to evaluate the performance of automatic
case adaptation. First, the hybrid CBR systems using indi-
vidual ML techniques are empirically compared with indi-
vidual estimators produced by ML techniques. Next, in or-
der to show that committees of ML algorithms may lead to
a higher prediction accuracy, we compare hybrid case adap-
tation approaches, using committees of ML techniques and
individual ML techniques, with individual ML techniques.
The hybrid approaches are also compared with a stan-
dard CBR system employing Knn for case adaptation. In
this standard approach, the value of the solution component
is obtained by averaging the solution of the k most similar
cases retrieved by the Knn technique. Finally, the experi-
ments investigate the effect of using different retrieval ap-
proaches and of varying the number of similar cases recov-
ered in the final adaptation result.
108 C.A. Policastro et al.

Fig. 5 Example of adaptation pattern generation

Fig. 6 The CASE

ADAPTATION ENGINE
algorithm

The experimental tests followed the 10-fold-cross- was used as a test fold (a set of new problems to be presented
validation methodology [27]. The patterns were randomly to the system) and the remaining 9 folds made up a training
divided into 10 groups (folds) with similar size. One fold fold (a set of previously stored cases). For each one of the
A hybrid case adaptation approach for case-based reasoning
Fig. 7 Example of the problem solving process
10 folds, the training-fold (using the remaining 9 folds) was
used to train the ML algorithms employed alone and to gen-
erate the adaptation patterns used to train the proposed case
adaptation mechanism of the CBR systems. Afterwards, the
average absolute error rates (the absolute difference between
the expected and the produced output value) were calculated
using the test fold. Based on the error rates for the 10 folds,
their average and standard deviation for all folds were cal-
culated.
In the experiments, the prediction performances of the
individual ML estimators (MLP, M5, and SVM) trained with
the original data sets were compared with those obtained by
the CBR systems, where each individual ML estimator was
trained using the adaptation patterns, and the CBR system
using Knn. The training adaptation patterns were produced
from the same folds of the original training set, using the
CBR retrieval mechanism. Thus, for the comparisons, the
109
adaptation patterns were independently produced for each
fold.
5.1 Experiments setup
As previously mentioned, estimators and combiners were
generated by three ML techniques: MLP, M5, and SVM.
The topologies of the estimators generated by the MLP net-
work are shown in Table 1.
The MLP network and the M5 algorithm were simu-
lated using the WEKA library, version 3.2—which includes
a set of Machine Learning algorithms.2 The SVMs were
simulated using the LIBSVM tool.3 The MLP networks
2 Available at http://www.cs.waikato.ac.nz/ml/weka/index.htm.
3 Available
at http://www.csie.ntu.edu.tw/~cjlin/libsvm.
110 C.A. Policastro et al.
Fig. 8 General architecture of
the CBR Cycle with the
proposed case adaptation
approach. APG is the
Adaptation Pattern Generator
Algorithm (see Fig. 3) and CAE
is the CASE ADAPTATION
ENGINE Algorithm (see Fig. 6)

Table 1 Topology of the estimators generated by the MLP network CBR(RA − AA − k). In this notation, RA indicates the ap-
Data set Input Hidden layer Output proach used for case retrieval (KdTree, Knn or Hybrid), AA
indicates the approach used for case adaptation (MLP, M5,
Servo 25 13 1 SVM, CMLP, CM5. or CSVM), and k indicates the number
Imports 162 82 1 of cases similar to the test case considered during adaptation
Housing 27 14 1 patterns generation. For example, CBR(KdTree − CSVM −
Michigan 27 14 1 3) means a CBR system employing the KdTree approach for
European 37 20 1 case retrieval and a committee using the SVM technique as
a combiner for case adaptation, trained with adaptation pat-
terns generated using the 3 cases most similar to the test
were trained using the momentum backpropagation algo-
case. Additionally, the use of ∗ means any mechanism or
rithm [17] with the moment term set to 0.2 and learning rate
approach. For the standard CBR system employing the Knn
set to 0.3. The M5 algorithm was trained using the default
average value as the adaptation mechanism, the employed
parameters settings of the WEKA Library. The SVMs were
notation is the following: CBR(Knn − k), where k indicates
trained using the Radial Basis Function kernel and default
the number of similar cases considered.
parameters settings of the LIBSVM tool.
The experiments carried out employ five data sets from
different domains in order to show the generality of the pro- 5.2 Empirical results
posed approach. For a detailed description of the structure
of these data sets, see Appendix 1. Three different adapta-
This section presents the main results obtained from our ex-
tion pattern data sets were created by generating adaptation
periments. Table 2 shows the main results of the experiments
patterns using the k most similar cases, k = {1, 3, and 5}
(absolute error ± standard deviation) for the individual tech-
(see Sect. 4.3.1). The data preprocessing was performed
only when required by the retrieval and adaptation mech- niques employed alone and for the CBR systems employing
anisms. The cases were stored in the CB in their original automatic case adaptation approaches for the three different
format. The numeric values were normalized to the inter- retrieval methods (see Sect. 4.2) using different values of k,
val [0 . . . 1] using the minimum and the maximum values for each data set. Since the European data set has more than
already present in the CB (we are supposing that the CB is one solution, the average absolute error of all components
representative [36]). The input attributes with nominal val- was calculated for this data set. For the complete set of re-
ues were transformed into orthogonal vectors of binary val- sults of the experiments carried out, see Appendix 2.
ues. A first look at these results suggests the superiority of
The results reported in this paper employ the follow- the hybrid approach. In order to verify the statistical signif-
ing notation for the tests with the hybrid CBR models: icance of this superiority, we ran the two-tailed paired t test
A hybrid case adaptation approach for case-based reasoning 111
Table 2 Main results of the experiments 6 Discussion
Model Average absolute error
The experimental results presented in the previous section
Servo data set show that using committees of ML techniques as adapta-
CBR(Hybrid-M5-5) 0.024 ± 0.019 tion mechanisms generally reduces the system’s average ab-
CBR(Hybrid-CMLP-5) 0.024 ± 0.011 solute prediction error and its standard deviation. These re-
CBR(Knn-1) 0.552 ± 0.278 sults possibly occur due to the fact that the committee is
CSVM 0.532 ± 0.299 able to benefit from the different biases of the estimators.
Therefore, these results show the potential of using hybrid
Imports data set
approaches that combine inductive learning and instance-
CBR(Knn-CM5-3) $518.85 ± $83.05
based learning.
CBR(Knn-1) $1.323.56 ± 329.26
The hybrid committee of ML algorithms takes advan-
CSVM $1.051.31 ± $311.53 tage of using different techniques with different inductive
Housing data set biases for the search in the decision space. Also the use of
CBR(Knn-CSVM-5) 2.37 ± 0.46 a combiner usually leads to a better partition of the search
CBR(Knn-1) 3.16 ± 0.83 space.
CSVM 3.02 ± 0.12 These results suggest that the adaptation pattern data
set extracted from the CB contains a good representative
Michigan data set sample of the required adaptations for the solution compo-
CBR(Knn-CSVM-3) 4.51 ± 0.38 nents.
CBR(KdTree-CSVM-5) 4.49 ± 0.38 Finally, the results illustrate that the case retrieval mech-
CBR(Knn-1) 6.02 ± 1.74 anism used by the system can influence the final results
CSVM 7.98 ± 0.42 of the hybrid CBR system, indicating that using a more
accurate retrieval mechanisms can increase system accu-
European data set
racy.
CBR(Hybrid-CSVM-3) 2.76 ± 0.19
CBR(Hybrid-CSVM-5) 2.77 ± 0.09
CBR(Knn-1) 6.11 ± 1.58
7 Conclusions
CSVM 3.50 ± 0.22

This article investigated the use of a hybrid case adaptation

with 99% certainty [27, 30]. The statistical results are shown
in Table 3.
In contrast to the work presented by Hanney [16] and
Wiratunga et al. [43], we developed a generic automatic case
adaptation approach that may be used on domains that re-
quire parameter adaptation. The knowledge extraction algo-
rithm we proposed employs a simple encoding for adapta-
tion patterns. This is achieved by concatenating components
of the problem description and solution, instead of extract-
ing attribute differences and adjustments.
The number (k) of similar cases used to generate the
adaptation pattern mechanism may influence adaptation per-
formance. For large data sets and/or data sets with a large
number of continuous input attributes, an increase of k may
decrease the performance of the system and k = 1 is more
appropriate. This may be due to the increase in the vari-
ability and conflicting adaptation rules in the adaptation pat-
terns.
mechanism able to perform automatic case adaptation on
several domains. Our results suggest that the use of a hy-
brid committee of ML techniques increased prediction ac-
curacy.
According to the results, the CBR(∗ − CSVM − ∗) mod-
els produced, in general, the best prediction results. This is
probably due to the properties of the SVM algorithm. SVMs
use a generalization control (inductive bias) and kernel func-
tions that allow the construction of hyperplanes that create
more efficient decision surfaces.
The results indicate the potential of the hybrid approach.
Our analysis of the results also reveals that increasing
the number of similar cases considered during the gen-
eration of the adaptation patterns may influence the pre-
diction performance of case adaptation. Further investiga-
tions are required to determine more precisely the influence
of k.
Additionally, according to the results, in general, com-
mittees of ML algorithms increased system accuracy. The
average error rates and the standard deviations were, in gen-
eral, smaller than those obtained for hybrid CBR systems
using the ML algorithms individually for the case adapta-
tion.
112 C.A. Policastro et al.
Table 3 T test conclusions
Model Conclusion

Servo data set

CBR(Hybrid - M5 - 5) and CBR(Hybrid - CMLP - 5) similar performance
CBR(Hybrid - CMLP - 5) and CMLP CBR(Hybrid - CMLP - 5) is better
CBR(Hybrid - CMLP - 5) and CM5 CBR(Hybrid - CMLP - 5) is better
CBR(Hybrid - CMLP - 5) and CSVM CBR(Hybrid - CMLP - 5) is better
CBR(Hybrid - CMLP - 5) and CBR(Knn - 1) CBR(Hybrid - CMLP - 5) is better

Imports data set

CBR(Knn - CM5 - 3) and CM5 CBR(Knn - CM5 - 3) is better
CBR(Knn - CM5 - 3) and CMLP CBR(Knn - CM5 - 3) is better
CBR(Knn - CM5 - 3) and CSVM CBR(Knn - CM5 - 3) is better
CBR(Knn - CM5 - 3) and CBR(Knn - 1) CBR(Knn - CM5 - 3) is better

Housing data set

CBR(Knn - CSVM - 5) and CM5 CBR(Knn - CSVM - 5) is better
CBR(Knn - CSVM - 5) and CMLP CBR(Knn - CSVM - 5) is better
CBR(Knn - CSVM - 5) and CSVM CBR(Knn - CSVM - 5) is better
CBR(Knn - CSVM - 5) and CBR(Knn - 1) CBR(Knn - CSVM - 5) is better

Michigan data set

CBR(Knn - CSVM - 5) and CBR(KdTree - CSVM - 5) similar performance
CBR(KdTree - CSVM - 5) and CMLP CBR(KdTree - CSVM - 5) is better
CBR(KdTree - CSVM - 5) and CM5 CBR(KdTree - CSVM - 5) is better
CBR(KdTree - CSVM - 5) and CSVM CBR(KdTree - CSVM - 5) is better
CBR(KdTree - CSVM - 5) and CBR(Knn - 1) CBR(KdTree - CSVM - 5) is better

European data set

CBR(Hybrid - CSVM - 3) and CBR(Hybrid - CSVM - 5) similar performance
CBR(Hybrid - CSVM - 3) and CMLP CBR(Hybrid - CSVM - 3) is better
CBR(Hybrid - CSVM - 3) and CM5 CBR(Hybrid - CSVM - 3) is better
CBR(Hybrid - CSVM - 3) and CSVM CBR(Hybrid - CSVM - 3) is better
CBR(Hybrid - CSVM - 3) and CBR(Knn - 1) CBR(Hybrid - CSVM - 3) is better

The experiments with the European data set indicated
that the case adaptation approach can be easily extended to
domains where the solution of the cases has more than one
component, by creating one independent adaptation data set
and one independent ML algorithm or committee for each
component of the solution, handling these components as
distinct problems.
Our approach employs a process of adaptation pattern
generation that can reduce knowledge acquisition effort.
Moreover, the process employed to obtain an adaptation pat-
tern data set is fully integrated with the case retrieval mech-
anism and can benefit from standard retrieval techniques [4,
13, 25]. Finally, the results obtained suggest that the set of
adaptation rules extracted from the CB used can provide a
useful new approach for case adaptation, with a good poten-
tial to solve problems.
For future work, we plan to investigate the influence of
post-processing the generated adaptation patterns data set in
order to eliminate noisy, distant, conflicting, or duplicated
patterns. We will also investigate the influence of differ-
ent similarity functions and attributes weights during case
retrieval. Additionally, we believe that the investigation of
committees composed of a higher number of estimators can
further decrease error rates. Finally, we will compare the
proposed approaches with additional CBR and case adap-
tation approaches. Another future work task will be to com-
pare our approach versus using Locally Weighted Learning
[2] for case adaptation.
Acknowledgement The authors would like to thank CNPq and
FAPESP for support received.
Appendix 1
The hybrid CBR system described in this paper was eval-
uated using different data sets. These evaluations involved
A hybrid case adaptation approach for case-based reasoning 113

a large number of experiments, which analysed the poten- screw/nut, and a sliding carriage. The output value rep-
tial of the combination of ML algorithms for automatic resents the time required for the system to respond to
case adaptation. This appendix presents the main features a step change in a position set point. This data set has
of the five data sets employed in this paper are presented. been used by Hanney [16] to evaluate another case adap-
These data sets, except for the Michigan and European tation strategy. For a summary of this data set, see Ta-
data sets, come from the UCI Repository of ML Data- ble 4);
bases [5]. Imports This data set lists the predicted price of a car given
Servo This data set was extracted from the simulation of a a set of specifications (see Table 5);
servo system involving a servo amplifier, a motor, a lead Housing The Housing data set contains housing values in
suburbs of Boston. This data set can be used for the predic-
Table 4 Servo case structure tion of the median value of owner-occupied homes, given
the value of a set of attributes (see Table 6);
Attribute Values
Michigan This data set comes from a water quality study
Problem Motor A,B,C,D,E carried out in Michigan, USA [19]. The instances in this
Screw A,B,C,D,E data set were analysed for several physical and chemical
PGain 3,4,5,6 parameters, such as: depth, secch, color, turbidity, conduc-
VGain 1,2,3,4,5 tivity and pH. For each case, the first 13 attributes represent
physical and chemical measurements, considered relevant
Solution Class Continuous
to predict plankton concentration. The last value of each

Table 5 Imports case structure

Attribute Value

Problem risk −3, −2, −1, 0, 1, 2, 3

loss-normalize Continuous
manufacturer alfa-romero, audi, bmw, chevrolet, dodge, ...
fuel-type diesel, gas
aspirantion std, turbo
door-number four, two
cover-type hardtop, wagon, sedan, hatchback, convertible
traction 4wd, fwd, rwd
motor-location front, rear
wheelwork-relationship Continuous
width Continuous
length Continuous
height Continuous
weight-limit Continuous
motor-type dohc, dohcv, l, ohc, ohcf, ohcv, rotor
cylinder-number eight, five, four, six, three, twelve, two
motor-size Continuous
fuel-system 1bbl, 2bbl, 4bbl, idi, mfi, mpfi, spdi, spfi
cylinder-capacity Continuous
course Continuous
tax-compression Continuous
potency Continuous
maximum-rotation Continuous
consumption-city Continuous
consumption-highway Continuous

Solution Price Continuous

114 C.A. Policastro et al.

case represents the concentration of planktons in the water ples were taken from sites on different European rivers over
(see Table 7); a period of approximately one year. The instances were an-
European This data set was employed in the 1999 Compu- alyzed for various chemical substances, including: nitrogen
tational Intelligence and Learning (COIL) competition. It in the form of nitrates and ammonia, phosphate, pH, oxy-
can be downloaded from the UCI KDD Repository [3]. gen and chloride. In parallel, algae samples were collected
This data set comes from a water quality study where sam- to determine the algae population distributions. The first 11
attributes of each case represent the season, the river size,
Table 6 Housing case structure the fluid velocity and 8 chemical concentrations, consid-
ered relevant to predict the algae population distributions.
Attribute Value
The last 7 values of each case are the distribution of differ-
ent types of algae (see Table 8).
Problem per-capta-crime-tax Continuous
residential-lands-proportion Continuous Table 9 summarizes the main characteristics of these data
unnegotiated-land-proportion Continuous sets. The column  Instances shows the number of instances
charles-river-variable Continuous in each data set. The column  Attributes (Continuous, Sym-
nitric oxide concentration Continuous bolic) presents the total number of attributes, and the number
home-rooms-medium-number Continuous of numeric and nominal attributes. The column  Missing
owner-occupied-units-proportion Continuous Attributes lists the total number of missing attributes in the
five-centers-boston-ponderate-distance Continuous original data set. The column  Conflicting and Duplicated
radial-highways-accessibility-index Continuous Attributes shows the total number of conflicting instances
tax-rates-over-properties Continuous plus the total number of duplicated instances (the number of
learning-taxes Continuous cases with the same problem description, but different solu-
negro-proportion Continuous
tions and cases with the same problem description and the
same solution).
% low-income-population Continuous

Solution home-medium-value Continuous

Appendix 2

This appendix presents complete set of results obtained in

Table 7 Michigan case structure
the experiments carried out for this paper.
Attribute Value

Problem Depth, ft Continuous

Secchi, m Continuous Table 8 European case structure
Color Continuous Attribute Value
Turbidity Top Continuous
Turbidity Bot Continuous Problem Season summer, spring, autumn, winter
Conductivity Top Continuous Size small, medium, large
pH Top Continuous Velocity high, medium, low
pH Botton Continuous CC1 Continuous
Temp C Top Continuous ... ...
Temp C Botton Continuous CC8 Continuous
Dissolved Oxygen Top Continuous
Solution AG1 Continuous
Dissolved Oxygen Botton Continuous
... ...
Solution Plankton Continuous AG7 Continuous

Table 9 Data set characteristics

summary Data set Instances Attributes Attributes Duplicated or Solution
(Numeric, Nominal) Missing Conflicting Components

Servo 167 4(0,4) 0 0 1

A hybrid case adaptation approach for case-based reasoning 115
Table 10 Results of CBR
Systems for the Servo data set Model Average absolute error
k=1 k=3 k=5

CBR(KdTree-M5) 0.032 ± 0.017 0.027 ± 0.018 0.024 ± 0.018

CBR(Knn-M5) 0.032 ± 0.018 0.028 ± 0.019 0.026 ± 0.015
CBR(Hybrid-M5) 0.032 ± 0.017 0.026 ± 0.018 0.024 ± 0.019
CBR(KdTree-MLP) 0.041 ± 0.021 0.033 ± 0.020 0.032 ± 0.017
CBR(Knn-MLP) 0.053 ± 0.022 0.034 ± 0.019 0.028 ± 0.015
CBR(Hybrid-MLP) 0.033 ± 0.020 0.033 ± 0.020 0.034 ± 0.019
CBR(KdTree-SVM) 0.067 ± 0.014 0.076 ± 0.009 0.075 ± 0.009
CBR(Knn-SVM) 0.050 ± 0.020 0.076 ± 0.010 0.075 ± 0.009
CBR(Hybrid-SVM) 0.078 ± 0.010 0.077 ± 0.009 0.075 ± 0.010
CBR(KdTree-CM5) 0.032 ± 0.019 0.032 ± 0.018 0.032 ± 0.019
CBR(Knn-CM5) 0.032 ± 0.019 0.032 ± 0.018 0.032 ± 0.019
CBR(Hybrid-CM5) 0.032 ± 0.019 0.032 ± 0.018 0.032 ± 0.019
CBR(KdTree-CMLP) 0.038 ± 0.019 0.030 ± 0.019 0.025 ± 0.012
CBR(Knn-CMLP) 0.045 ± 0.020 0.028 ± 0.017 0.026 ± 0.014
CBR(Hybrid-CMLP) 0.033 ± 0.019 0.032 ± 0.020 0.024 ± 0.011
CBR(KdTree-CSVM) 0.039 ± 0.015 0.038 ± 0.016 0.040 ± 0.017
CBR(Knn-CSVM) 0.035 ± 0.015 0.039 ± 0.017 0.036 ± 0.015
CBR(Hybrid-CSVM) 0.042 ± 0.014 0.038 ± 0.015 0.043 ± 0.019
CBR(Knn) 0.55 ± 0.28 1.14 ± 0.30 1.24 ± 0.31

Table 11 Results of CBR

Systems for the Imports data set Model Average absolute error
k=1 k=3 k=5

CBR(KdTree-M5) $1.143.78 ± $278.49 $612.02 ± $78.22 $652.08 ± $76.67

CBR(Knn-M5) $1.127.69 ± $282.66 $614.06 ± $79.31 $652.06 ± $84.73
CBR(Hybrid-M5) $1.147.47 ± $275.31 $612.11 ± $81.29 $659.09 ± $74.51
CBR(KdTree-MLP) $1.882.84 ± $539.62 $1.129.41 ± $133.72 $992.35 ± $121.62
CBR(Knn-MLP) $1.940.78 ± $653.71 $1.093.41 ± $140.41 $103.31 ± $133.67
CBR(Hybrid-MLP) $1.841.76 ± $467.78 $1.044.36 ± $135.25 $988.35 ± $117.59
CBR(KdTree-SVM) $1.412.41 ± $226.41 $1.088.36 ± $109.33 $1.084.34 ± $71.87
CBR(Knn-SVM) $1.420.33 ± $234.26 $1.079.35 ± $96.63 $1.089.38 ± $68.72
CBR(Hybrid-SVM) $1.398.33 ± $221.79 $1.089.35 ± $111.34 $1.079.35 ± $69.67
CBR(KdTree-CM5) $1.018.42 ± $204.21 $577.85 ± $83.05 $731.07 ± $90.43
CBR(Knn-CM5) $1.029.42 ± $217.58 $518.85 ± $83.05 $738.07 ± $96.28
CBR(Hybrid-CM5) $1.008.42 ± $1987.30 $579.85 ± $85.05 $731.07 ± $88.23
CBR(KdTree-CMLP) $1.167.03 ± $178.45 $639.21 ± $85.30 $699.21 ± $94.94
CBR(Knn-CMLP) $1.160.05 ± $178.17 $627.19 ± $75.36 $688.19 ± $89.59
CBR(Hybrid-CMLP) $1.168.03 ± $171.41 $637.21 ± $87.93 $689.21 ± $94.77
CBR(KdTree-CSVM) $1.102.04 ± $124.09 $623.21 ± $69.77 $632.16 ± $61.12
CBR(Knn-CSVM) $1.109.04 ± $139.82 $618.23 ± $72.28 $639.17 ± $53.38
CBR(Hybrid-CSVM) $1.089.15 ± $116.18 $629.15 ± $66.05 $627.14 ± $63.10
CBR(Knn) 1.323.56 ± 329.26 1.582.36 ± 609.37 1.678.14 ± 499.11
116 C.A. Policastro et al.
Table 12 Results of CBR
Systems for the Housing data set Model Average absolute error
k=1 k=3 k=5

CBR(KdTree-M5) 3.44 ± 0.28 3.68 ± 0.45 3.93 ± 0.52

CBR(Knn-M5) 3.45 ± 0.28 3.78 ± 0.51 4.10 ± 0.64
CBR(Hybrid-M5) 3.43 ± 0.29 3.62 ± 0.41 3.82 ± 0.45
CBR(KdTree-MLP) 4.28 ± 0.61 3.98 ± 0.53 4.26 ± 0.48
CBR(Knn-MLP) 3.94 ± 0.52 3.96 ± 0.46 4.00 ± 0.38
CBR(Hybrid-MLP) 4.51 ± 0.66 4.00 ± 0.58 4.44 ± 0.54
CBR(KdTree-SVM) 3.45 ± 0.36 3.18 ± 0.30 3.12 ± 0.29
CBR(Knn-SVM) 3.46 ± 0.38 3.17 ± 0.31 3.11 ± 0.29
CBR(Hybrid-SVM) 3.45 ± 0.35 3.19 ± 0.29 3.12 ± 0.28
CBR(KdTree-CM5) 3.65 ± 0.42 3.54 ± 0.47 4.23 ± 1.06
CBR(Knn-CM5) 3.47 ± 0.36 3.53 ± 0.40 3.95 ± 0.90
CBR(Hybrid-CM5) 3.77 ± 0.45 3.55 ± 0.51 4.42 ± 1.17
CBR(KdTree-CMLP) 3.82 ± 0.38 3.53 ± 0.41 3.77 ± 0.42
CBR(Knn-CMLP) 3.76 ± 0.43 3.67 ± 0.36 3.63 ± 0.34
CBR(Hybrid-CMLP) 3.85 ± 0.35 3.44 ± 0.45 3.87 ± 0.48
CBR(KdTree-CSVM) 2.89 ± 0.72 2.55 ± 0.21 2.52 ± 0.29
CBR(Knn-CSVM) 3.07 ± 1.63 2.44 ± 0.35 2.37 ± 0.46
CBR(Hybrid-CSVM) 2.78 ± 0.12 2.63 ± 0.12 2.62 ± 0.17
CBR(Knn) 3.16 ± 0.83 3.65 ± 2.35 3.28 ± 2.27

Table 13 Results of CBR

Systems for the Michigan data Model Average absolute error
set k=1 k=3 k=5

CBR(KdTree-M5) 12.35 ± 2.09 14.42 ± 3.58 16.12 ± 3.53

CBR(Knn-M5) 12.53 ± 2.47 14.68 ± 4.06 16.33 ± 3.72
CBR(Hybrid-M5) 12.19 ± 2.89 15.15 ± 3.27 16.49 ± 3.59
CBR(KdTree-MLP) 22.98 ± 3.98 23.33 ± 5.31 23.97 ± 7.45
CBR(Knn-MLP) 23.25 ± 4.27 23.70 ± 6.02 24.20 ± 7.54
CBR(Hybrid-MLP) 24.78 ± 4.33 26.10 ± 5.58 24.66 ± 5.02
CBR(KdTree-SVM) 11.51 ± 0.61 12.98 ± 1.51 12.99 ± 1.72
CBR(Knn-SVM) 13.10 ± 1.37 13.16 ± 1.79 13.15 ± 1.87
CBR(Hybrid-SVM) 12.18 ± 2.85 13.10 ± 2.34 12.87 ± 2.16
CBR(KdTree-CM5) 20.97 ± 4.40 21.30 ± 6.24 23.10 ± 5.99
CBR(Knn-CM5) 21.29 ± 4.81 21.67 ± 7.12 23.36 ± 6.02
CBR(Hybrid-CM5) 22.21 ± 4.58 24.10 ± 5.78 24.72 ± 5.58
CBR(KdTree-CMLP) 20.98 ± 4.40 21.31 ± 6.33 23.11 ± 5.88
CBR(Knn-CMLP) 21.24 ± 4.81 21.69 ± 7.00 23.33 ± 6.20
CBR(Hybrid-CMLP) 20.09 ± 4.13 23.31 ± 5.60 24.63 ± 4.87
CBR(KdTree-CSVM) 6.43 ± 0.50 4.78 ± 0.30 4.49 ± 0.38
CBR(Knn-CSVM) 6.51 ± 0.70 4.85 ± 0.36 4.51 ± 0.38
CBR(Hybrid-CSVM) 5.78 ± 0.59 4.79 ± 0.29 4.68 ± 0.23
CBR(Knn) 6.02 ± 1.74 6.11 ± 1.90 6.40 ± 2.02
A hybrid case adaptation approach for case-based reasoning 117
Table 14 Prediction results of
the CBR Systems for the Model Average absolute error
European data set k=1 k=3 k=5

CBR(KdTree-M5) 4.45 ± 1.12 5.10 ± 1.60 5.83 ± 2.63

CBR(Knn-M5) 4.60 ± 1.15 5.12 ± 1.75 5.80 ± 3.05
CBR(Hybrid-M5) 5.31 ± 0.76 5.48 ± 0.57 5.99 ± 0.87
CBR(KdTree-MLP) 10.82 ± 2.77 9.75 ± 1.98 9.98 ± 1.93
CBR(Knn-MLP) 10.91 ± 2.70 9.97 ± 2.24 10.43 ± 2.54
CBR(Hybrid-MLP) 11.54 ± 0.32 10.24 ± 0.88 11.76 ± 0.80
CBR(KdTree-SVM) 5.55 ± 0.38 5.34 ± 0.33 5.31 ± 0.43
CBR(Knn-SVM) 5.61 ± 0.45 5.54 ± 0.49 5.65 ± 0.50
CBR(Hybrid-SVM) 5.90 ± 0.50 5.80 ± 0.40 5.87 ± 0.36
CBR(KdTree-CM5) 7.99 ± 2.55 7.24 ± 1.77 6.91 ± 2.02
CBR(Knn-CM5) 8.21 ± 2.79 7.52 ± 2.03 7.38 ± 2.51
CBR(Hybrid-CM5) 8.64 ± 0.42 7.84 ± 0.60 8.11 ± 0.69
CBR(KdTree-CMLP) 8.33 ± 2.47 8.11 ± 1.80 8.41 ± 2.16
CBR(Knn-CMLP) 8.57 ± 2.70 8.33 ± 2.11 8.71 ± 2.40
CBR(Hybrid-CMLP) 8.82 ± 0.50 8.70 ± 0.70 8.85 ± 0.56
CBR(KdTree-CSVM) 2.96 ± 0.32 3.49 ± 0.29 3.11 ± 0.22
CBR(Knn-CSVM) 3.07 ± 0.40 3.56 ± 0.36 3.23 ± 0.30
CBR(Hybrid-CSVM) 2.99 ± 0.03 2.76 ± 0.19 2.77 ± 0.09
CBR(Knn) 6.11 ± 1.58 6.13 ± 1.20 6.25 ± 1.19

Table 15 Results of ML Techniques for the Servo data set Table 17 Results of ML Techniques for the Housing data set

Model Average absolute error Model Average absolute error

M5 0.959 ± 0.431 M5 3.43 ± 0.29

MLP 0.976 ± 0.302 MLP 4.28 ± 0.94
SVM 0.832 ± 0.319 SVM 3.18 ± 0.33
CM5 0.559 ± 0.311 CM5 3.17 ± 0.22
CMLP 0.586 ± 0.202 CMLP 3.95 ± 0.34
CSVM 0.532 ± 0.299 CSVM 3.02 ± 0.12

Table 16 Results of ML Techniques for the Imports data set Table 18 Results of ML Techniques for the Michigan data set

Model Average absolute error Model Average absolute error

M5 $1.583.66 ± $321.32 M5 12.34 ± 2.05

MLP $1.751.31 ± $483.52 MLP 12.81 ± 1.45
SVM $1.651.31 ± $471.58 SVM 24.88 ± 6.59
CM5 $1.283.66 ± $221.31 CM5 11.91 ± 1.34
CMLP $1.251.31 ± $363.53 CMLP 12.74 ± 2.03
CSVM $1.051.31 ± $311.53 CSVM 7.98 ± 0.42
118
Table 19 Prediction results of the ML Techniques for the European
data set
Model Average absolute error
M5 5.17 ± 0.54
MLP 11.14 ± 0.96
SVM 5.74 ± 0.46
CM5 4.68 ± 0.31
CMLP 9.50 ± 0.63
CSVM 3.50 ± 0.22
References
1. Aamodt A, Plaza E (1994) Case-based reasoning: foundational
issues, methodological variations, and systems approaches. AI
Commun 7:39–59
2. Atkeson CG, Moore AW, Schaal S (1997) Locally weighted learn-
ing. Artif Intell Rev 11:11–73
3. Bailey T, Elkan C (1993) Estimating the accuracy of learned con-
cepts. In: Bajcsy R (ed) 13th international joint conference on ar-
tificial intelligence, Chambry, France. Kaufmann, San Francisco,
pp 895–901
4. Bentley J (1975) Multidimensional binary search tree used for as-
sociative searching. Commun ACM 18(9):509–517
5. Blake CL, Merz CJ (1998) UCI repository of machine learn-
ing databases. University of California, Irvine, Dept. of Infor-
mation and Computer Sciences. http://www.ics.uci.edu/~mlearn/
MLRepository.html
6. Burges CJC (1998) A tutorial on support vector machines for pat-
tern recognition. Data Min Knowl Discov 2(2):121–167
7. Carpenter G, Grossberg S (1987) ART 2: self-organization of sta-
ble category recognition codes for analog input patterns. Appl Opt
26(23):4919–4930
8. Corchado J, Lees B, Fyle C, Ress N, Aiken J (1998) Neuro-
adaptation method for a case-based reasoning system. Comput Inf
Syst J 5:15–20
9. Breiman L (1996) Bagging predictors. Mach Learn 24(2):123–
140
10. Dietterich G (1997) Machine learning research: four current direc-
tions. AI Mag 18(4):97–136
11. Main J, Dillom T, Shiu S (2001) A tutorial on case-based reason-
ing. In: Pal S, Dillon T, Yeung D (eds) Soft computing in case-
based reasoning. Captulo 1. Springer, Berlin
12. Domingos P (1996) Unifying instance-based and rule-based in-
duction. Mach Learn 24:141–168
13. Duda R, Hart P, Stork D (2001) Pattern classification. Wiley–
Interscience, New York
14. Freund Y, Schapire R (1996) Experiments with a new boosting
algorithm. In: Saitta L (ed) 13th international conference on ma-
chine learning, Bari, Italy. Kaufmann, San Francisco, pp 148–156
15. Hanney K, Keane MT, Smyth B, Cunningham P (1995) What
kind of adaptation do CBR systems need? A review of current
practice. In: Aha DW, Ram A (eds) AAAI 1995 fall sympo-
sium on adaptation of knowledge for reuse, MIT Campus, Cam-
bridge, Massachusetts, EUA. Available at http://www.aic.nrl.navy.
mil/aha/aaai95-fss/papers/hanney.ps.Z
16. Hanney K (1996) Learning adaptation rules from cases. Master’s
thesis, University College Dublin
17. Haykin S (1999) Neural networks: a comprehensive foundation.
Prentice Hall, New York
18. Hilario M (1997) An overview of strategies for neurosymbolic in-
tegration. Connectionist-symbolic integration: from unified to hy-
brid approaches. Lawrence Earlbaum Associates, Inc., Chap. 2
C.A. Policastro et al.
19. Jackson WG (2002) Water resources outreach education program
for schools and community groups. Annis Water Resources Insti-
tute. http://www.gvsu.edu/wri/education/waterdata.htm
20. Kolodner J Adaptation methods and strategies. Case-based reason-
ing. Kaufmann, San Francisco, Chap. 11
21. Lavrac N, Gamberger D, Todorovski L, Blockeel H (eds) (2003)
Machine learning: ECML 2003. In: Proceedings of the 14th Eu-
ropean conference on machine learning, Cavtat-Dubrovnik, Croa-
tia. Lecture notes in computer science, vol 2837. Springer, Berlin,
ISBN 3-540-20121-1
22. Leake D (1995) Becoming an expert case-based reasoner: learning
to adapt prior cases. In: 8th annual Florida artificial intelligence
research symposium, Melbourne, pp 1120–1160
23. Leake D (1996) CBR in context: the present and future. In:
Case-based reasoning: experiences, lessons and future directions.
AAAI/MIT, Menlo Park, pp 1–35
24. Leake D, Kinley A, Wilson D (1996) Acquiring case adaptation
knowledge: a hybrid approach. In: Burkhard D, Lenz M (eds) 30th
national conference on artificial intelligence and 8th innovative
applications of artificial intelligence conference, Portland, USA.
AAAI/MIT, Menlo Park, pp 684–689
25. Lenz M, Burkhard H-D (1996) Case retrieval nets: basic ideas and
extensions. In: Burkhard H-D, Lenz M (eds) 4th German work-
shop on case-based reasoning: system development and evalua-
tion, Berlin, Germany, pp 103–110
26. Malerba D, Appice A, Bellino A, Ceci M, Pallotta D (2001) Step-
wise induction of model trees. In: Esposito F (ed) AI*IA 2001:
advances in artificial intelligence. Lecture notes in artificial intel-
ligence, vol 2175. Springer, New York
27. Mitchell TM (1997) Machine learning. McGraw–Hill, New
York
28. McSherry D (1998) An adaptation heuristic for case-based estima-
tion. In: Smyth B, Cunningham P (eds) Proceedings of the Euro-
pean workshop on case-based reasoning. Springer, Berlin, pp 184–
195
29. Moore AW An introductory tutorial on Kd-trees. citeseer.ist.psu.
edu/140157.html
30. Moses LE (1986) Comparison of averages from two samples
and some related problems. Think and explain with statistics.
Addison–Wesley, Reading, Chap. 6
31. Orr M (1996) Introduction to radial basis function networks. Tech-
nical report, Centre for Cognitive Science, University of Edin-
burgh
32. Policastro C, Carvalho A, Delbem A (2003) Hybrid approaches
for cases retrieval and adaptation. In: Günter A, Kruse R, Neu-
mann B (eds) Proceedings of 26th German conference on artificial
intelligence, Hamburg, Germany. Lecture notes in artificial intel-
ligence, vol 2821. Springer, Berlin, pp 297–311
33. Quinlan R (1992) Learning with continuous classes. In: 5th Aus-
tralian joint conference on artificial intelligence, Hobart, Tasma-
nia. World Scientific, Singapore, pp 343–348
34. Russel R, Norvig P (1995) Artificial intelligence: a modern ap-
proach. Prentice Hall, Englewood Cliffs
35. Smyth B, Cunningham P (1993) Complexity of adaptation in real-
world case-based reasoning systems. In: 6th Irish conference on
artificial intelligence and cognitive science, Belfast, Ireland
36. Smyth B (1998) Case base maintenance. In: Mira J, Pobil A (eds)
12th international conference on industrial and engineering appli-
cations of artificial intelligence and expert systems, Cairo, Egypt.
Springer, Berlin, pp 507–516
37. Valentini GM, Masulli F (2002) Ensembles of learning machines.
In: Marinaro M, Tagliaferri R (eds) Neural nets WIRN Vietri-
02, Heidelberg, Germany. Lecture notes in computer sciences.
Springer, Berlin
38. Vapnik V (1998) Statistical learning theory. Wiley, New York
A hybrid case adaptation approach for case-based reasoning
39. Wang Y, Witten I (1997) Induction of model trees for predicting
continuous classes. In: Someren M, Widmer G (eds) 9th Euro-
pean conference on machine learning, Prague, Czech Republic.
Springer, Berlin, pp 128–137
40. Watson I (1997) Applying case-based reasoning: techniques for
enterprise systems. Kaufmann, Los Altos
41. Watson I (1999) CBR is a methodology not a technology.
Knowledge-Based Syst 12:303–308
119
42. Wilke W, Vollrath I, Althoff K-D, Bergmann R (1996) A frame-
work for learning adaptation knowledge based on knowledge light
approaches
43. Wiratunga N, Craw S, Rowe R (2002) Learning to adapt for case-
based design. In: Craw S, Preece A (eds) 6th European confer-
ence on case-based reasoning, Aberdeen, Scotland, UK. Springer,
Berlin, pp 421–435