ARTICLE IN PRESS

Journal of Biomechanics 43 (2010) 720726

Contents lists available at ScienceDirect

Journal of Biomechanics
journal homepage: www.elsevier.com/locate/jbiomech
www.JBiomech.com

Comparison among probabilistic neural network, support vector machine

and logistic regression for evaluating the effect of subthalamic stimulation in
Parkinson disease on ground reaction force during gait
A.M.S. Muniz a, H. Liu b, K.E. Lyons c, R. Pahwa c, W. Liu b, F.F. Nobre a, J. Nadal a,n
a
Biomedical Engineering Program, Federal University of Rio de Janeiro, COPPE, P.O. Box 68510, 21941-972, Rio de Janeiro, RJ, Brazil
b
Department of Physical Therapy and Rehabilitation Sciences, University of Kansas Medical Center, Kansas City, KS, USA
c
Department of Neurology, University of Kansas Medical Center, Kansas City, KS, USA

a r t i c l e in f o a b s t r a c t

Article history: Deep brain stimulation of the subthalamic nucleus (DBS-STN) is an approved treatment for advanced
Accepted 6 October 2009 Parkinson disease (PD) patients; however, there is a need to further evaluate its effect on gait. This study
compares logistic regression (LR), probabilistic neural network (PNN) and support vector machine
Keywords: (SVM) classiers for discriminating between normal and PD subjects in assessing the effects of DBS-STN
Parkinson disease on ground reaction force (GRF) with and without medication. Gait analysis of 45 subjects (30 normal
Deep brain stimulation and 15 PD subjects who underwent bilateral DBS-STN) was performed. PD subjects were assessed under
Logistic regression four test conditions: without treatment (mof-sof), with stimulation alone (mof-son), with medication
Probabilistic neural network alone (mon-sof), and with medication and stimulation (mon-son). Principal component (PC) analysis
Support vector machine
was applied to the three components of GRF separately, where six PC scores from vertical, one from
Gait analysis
anteriorposterior and one from mediallateral were chosen by the broken stick test. Stepwise LR
analysis employed the rst two and fth vertical PC scores as input variables. Using the bootstrap
approach to compare model performances for classifying GRF patterns from normal and untreated PD
subjects, the rst three and the fth vertical PCs were attained as SVM input variables, while the same
ones plus the rst anteriorposterior were selected as PNN input variables. PNN performed better than
LR and SVM according to area under the receiver operating characteristic curve and the negative
likelihood ratio. When evaluating treatment effects, the classiers indicated that DBS-STN alone was
more effective than medication alone, but the greatest improvements occurred with both treatments
together.
& 2009 Published by Elsevier Ltd.

1. Introduction 2008). Approaches that capture features of the entire waveform

Parkinson disease (PD) is a neurodegenerative disorder leading
to difculty in motor function, including gait and balance. Deep
brain stimulation of the subthalamic nucleus (DBS-STN) is a
treatment for advanced PD patients with disabling motor
uctuations, allowing a signicant reduction in dopaminergic
medications (Ferrarin et al., 2005). Various studies have evaluated
the effects of DBS-STN using clinical motor scores (Krack et al.,
2003; Ostergaard and Sundae, 2006), while only a few have
quantitatively assessed the gait of PD patients (Liu et al., 2005;
Ferrarin et al., 2005). Gait speed is shown to be the variable most
affected by the DBS-STN; however, it does not take into account
atypical waveforms and therefore does not provide enough
information about the gait pattern (Schwartz and Rozumalski,
n
Corresponding author. Tel.: +55 21 25628577; fax: + 55 21 25628591.
E-mail address: jn@peb.ufrj.br (J. Nadal).
instead of a few parameters may improve the effectiveness of the
analysis (Chester et al., 2007). Additionally, the correlations
among variables must be considered to accurately evaluate the
extent of gait abnormalities and to assess the changes resulting
from a specic treatment (Schutte et al., 2000).
A clinical challenge is to understand the disease process as well
as outcomes of potential interventions. Logistic regression (LR) is
commonly used as a linear predictive model for diagnostic and
prognostic tasks. Recently, computational intelligence techniques
such as articial neural networks (ANN) and support vector
machines (SVM) have played an important role in gait classica-
tion and the diagnosis of diseases (Lai et al., 2009). Studies have
compared the predictive ability of LR and ANN (Dreiseitl and
Ohno-Machado, 2002; Song et al., 2005). ANN modeling has been
used in gait analysis focusing on pattern recognition (Hahn et al.,
2005), as well as for classifying normal and pathological patterns
(Lafuente et al., 1997; Su and Wu, 2000). SVM has recently been
used for automated identication of gait pathologies (Begg et al.,

0021-9290/$ - see front matter & 2009 Published by Elsevier Ltd.

doi:10.1016/j.jbiomech.2009.10.018
 ARTICLE IN PRESS
A.M.S. Muniz et al. / Journal of Biomechanics 43 (2010) 720726
2005; Lai et al., 2009). However, none of the past studies
compared LR, probabilistic neural network (PNN) and SVM in
classifying gait patterns or evaluated the effect of therapeutic
interventions on ground reaction force (GRF) of PD patients.
This study evaluated LR, PNN and SVM models for discriminat-
ing between normal and PD subjects using principal components
derived from the GRF as input variables. For performance
evaluation, the accuracies (ACC) and the areas under the receiver
operating characteristic (ROC) curves (AUC) based on 1000
bootstrap runs of the classiers were compared. The effects of
DBS-STN on GRF with and without medication were also
evaluated with both the models.
2. Materials and methods
2.1. Studied groups
The subjects (n= 45) consisted of 30 healthy controls (20 women) and 15 PD
patients (three women). The healthy subjects, recruiting by advertisements, were
without neurological illness, degenerative conditions or any general disease that
might interfere with gait (Table 1). A screening questionnaire was completed to
guarantee that subjects were suitable for the study. The PD subjects were recruited
from the Parkinson Disease and Movement Disorder Center of the University of
Kansas Medical Center. All PD subjects had undergone bilateral DBS-STN and were
stable when the study was conducted. Each subject signed an informed consent
approved by the local Institutional Review Board.
2.2. Experimental protocol and signal processing
For each PD subject, quantitative gait measurement was obtained on two
different days. In the rst session, the subject had taken the usual dose of PD
medication and stimulators were turned on. The gait assessment was rst
conducted with both medication and stimulation (mon-son). After turning the
stimulators off for 30 min, the measurements were repeated (mon-sof). In the second
session, the subjects were without medication for at least 12 h. Gait analysis was rst
conducted with stimulation (mof-son), and repeated after 30 min without stimula-
tion (mof-sof). Due to technical problems, some subjects did not complete all tests.
Therefore, 13 subjects were evaluated in mof-sof, 12 in mof-son, 14 in mon-sof and 11
in mon-son conditions. Subjects from the control group were evaluated only once.
The quantitative analysis for the controls and PD subjects in the mof-sof condition
was used to develop the classier models. The other three PD conditions were
included in the developed models to evaluate the DBS-STN effect in PD treatment.
Two force platforms (AMTI, USA) were mounted in series at the middle of a
walkway. All subjects practiced the walking trial ve times before the experiment.
The subjects walked barefoot at their self-selected speed and repeated the walking
trial ve times. The GRF from both force platforms were collected for 10 s at a
sampling frequency of 100 Hz, ltered using a low-pass Butterworth lter, with a
cut-off frequency of 30 Hz, and normalized by subject body weight.
The averaged vertical, anteriorposterior and mediallateral components of
GRF from ve walking trials were interpolated with cubic splines and re-sampled
with 101 sample points according to the stance phase duration of each foot. Thus,
202 GRF samples were analyzed for each GRF component separately.
Each vertical, anteriorposterior and mediallateral GRF waveform was stored
in a matrix E with 43 rows (# of subjects) and 202 columns (# of GRF samples for
both right and left limbs). Principal component analysis (PCA) was applied to the
covariance matrices S (202  202) from each E, separately (Jollife, 2002).
2.3. Logistic regression
LR is a statistical modeling technique that estimates the probability of a
dichotomous outcome event being related to a set of explanatory variables
Table 1
Subjects characteristics.
Control group PD patients
Age (years) 50.17 7.8 56.4 7 8.3
Mass (kg) 90.56 7 15.7 50.52 7 8.02
Height (m) 1.73 70.08 1.67 7 0.09
Duration of PD (years) 12.27 4.3 N.A.
Time since surgery (months) 15.17 9.5 N.A.
721
(Schumacher et al., 1996):
1
Px Pn 1
1 ebo i 1
bi Pci
where bo is the intercept and bi is the coefcient associated with the explanatory
variable PCi. The maximum likelihood estimation method is used to estimate the b
coefcients. For subjects classication, the PC scores were selected as the
independent variables for computing the natural logarithm of the odds ratio; the
ratio between the probabilities that an event will or will not occur (1controls
and 0PD subjects). The classication threshold was set to 0.5.
2.4. Probabilistic neural networks
PNN is a feedforward ANN developed by Specht (1990), in which the response
to an input pattern is processed from one layer to the next, without feedback paths
to previous layers. A typical PNN has four layers: input, pattern, summation and
output. The input units supply the same values to all pattern units. The pattern
units form a dot product between the input pattern vector x and a weight vector wi
(zi = xwi), which is followed by the nonlinear neuron activation function:
 
w  x0 wi  x
gzi exp  i 2
2s2
This Bayesian function takes into account the relative likelihood of events and
uses a priori information to improve the prediction (Specht and Romsdahl, 1994).
The summation units simply sum the inputs from the pattern units, corresponding
to the category from which the training patterns were selected. Repeating this
procedure for each class, the un-normalized density functions gk (x), for k= 1,
2, y, K were estimated. The Bayesian probability that the case was from class k is
as follows:
gi x
Px A k Pk 3
i1 gi x
The output units have a competitive transfer function that picks the maximum
of the probabilities and produces 1 for one class (normals) and 0 (PD patients) for
the other.
2.5. Support vector machine
The SVM estimates a function for classifying data into two classes (Vapnik,
2000). Using a nonlinear transformation ^ that depends on a regularization
parameter C (Begg et al., 2005), the input vectors are placed into a high-
dimensional feature space, where a linear separation is employed. To construct a
nonlinear support vector classier, the inner product (x,y) is replaced by a kernel
function K(x,y)

X
l
f x sgn ai yi Kxi ; x b 4
i1
where f(x) determines the membership of x. In this study, the normal subjects
were labeled as  1 and PD subjects as +1. The SVM has two layers. During the
learning process, the rst layer selects the basis K(xi,x), i= 1,2, y, N from the given
set of kernels, while the second layer constructs a linear function in the ^ space.
This is equivalent to nding the optimal hyperplane in the corresponding feature
space. The SVM algorithm can construct a variety of learning machines using
different kernel functions.
2.6. Variable selection
The broken stick criterion (Jolliffe, 2002) was used for choosing the signicant
PCs of vertical, anteriorposterior and mediallateral GRF components for the
analysis. Moreover, to build a more accurate classier model, it was necessary to
evaluate which scores contributed to improvement in the task (Chang, 1983).
In the LR model, a stepwise approach was used to select the input variables by
the Akaike information criterion (AIC), followed by w2 tests to contrast with a full
model including all PC scores selected by the broken stick criterion or with subsets
of variables close to the nal model (Krzanowski, 1998).
PNN requires the selection of the optimal value for the width (s2) of the radial
basis function. For optimizing s2 and selecting the relevant input variables, PNN
models were trained and evaluated by the bootstrap method, considering each
possible combination of scores and varying the values s2 in the interval [0.1, 1].
Briey, bootstrapping generates training sets drawing samples with replacement
from the original data set.
For the SVM, the appropriate kernel function, the number of PC scores to be
used as input variables and the parameter C were evaluated using the same
bootstrap approach applied to the PNN. The input set was determined considering
each possible combination of scores. All SVM models were trained over the range
C= {0.1, 1, 10, 100, 1000} using linear, polynomial and Gaussian kernels (Lai et al.,
2009).
 ARTICLE IN PRESS
722 A.M.S. Muniz et al. / Journal of Biomechanics 43 (2010) 720726

Fig. 1. Averaged vertical ground reaction force from controls (bold line) and PD (dotted line): (a) mof-sof, (b) mof-son, (c) mon-sof and (d) mon-son conditions.

Fig. 2. Averaged anteriorposterior ground reaction force from controls (bold line) and PD (dotted line): (a) mof-sof, (b) mof-son, (c) mon-sof and (d) mon-son conditions.

2.7. Performance evaluation indices for each classier represented the average and standard error over all 1000
tests with different bootstrap samples. Comparisons among models were
performed using the one-way ANOVA, with a = 0.05 followed by the post-hoc
The models performances were assessed using the AUC and ACC. With small
Bonferroni test. The agreement between each model pair was compared using the
sample sizes, the application of a resampling technique such as the bootstrap
percent agreement and the Cohens kappa coefcients, using the algorithm
approach is recommended to estimate the performance of the classier (Sahiner et
proposed by Cardillo (2007) and interpreted according to the standard criteria
al., 2008). Both ACC and AUC were obtained using the set of samples not included
(Landis and Koch, 1977).
in the bootstrap sample that were obtained using the 0.632+ bootstrap method
(Efron and Tibshirani, 1997).
The predictive power of a classier to ascertain if a PD patient under treatment
has achieved a normal gait prole was evaluated with the negative likelihood ratio
(NLR) between false and true negatives. This ratio gives the odds that a subject 2.8. Effect of treatments
does not have symptoms of the disease, in this case indicating if the treatment has
a positive effect. For quantifying the effect of the treatments, the PC scores from PD subjects in
To perform a comparison among LR, PNN and SVM models, the same bootstrap the mon-sof, mof-son and mon-son conditions were calculated and used as inputs
samples were used for developing and testing the models. The AUC, ACC and NLR in the developed classiers.
 ARTICLE IN PRESS
A.M.S. Muniz et al. / Journal of Biomechanics 43 (2010) 720726 723

Fig. 3. Averaged mediallateral ground reaction force from controls (bold line) and PD (dotted line): (a) mof-sof, (b) mof-son, (c) mon-sof and (d) mon-son conditions.

Fig. 4. (a) Area under ROC (AUC) and accuracy (ACC) of probabilistic neural network classier; and (b) average and 95% condence bands for AUC and ACC for one thousand
bootstrap training and testing samples as a function of the spread constants for the selected inputs.

3. Results normal pattern (Figs. 13), especially when both medication and
stimulation were used (Figs. 1d, 2d, 3d).
Visual inspection of three averaged GRF components from The Broken Stick test indicated that the rst six PCs from
normal and PD subjects showed the PD pattern moving toward a vertical GRF, the rst PC from anteriorposterior GRF and the rst
 ARTICLE IN PRESS
724 A.M.S. Muniz et al. / Journal of Biomechanics 43 (2010) 720726

PC from mediallateral GRF, which respectively explained 91.1%, The best SVM performance was obtained for the linear kernel
69.1% and 62.1% of the total variation, should be considered in the with C = 1 and four inputs (the rst three and the fth vertical GRF
analysis. PC scores), for both AUC and ACC criteria (Table 2).
The stepwise selection of LR variables identied the rst, The resulting values of AUC, ACC and NLR for the 1000
second and fth PC scores from vertical GRF as the signicant bootstrap samples indicate that all methods presented high
inputs. Contrasting with the LR model including all eight scores performance indexes (Fig. 5). The ANOVA showed a signicant
under a w2 distribution, this model yielded a higher P value difference among these values (Table 3), indicating that PNN
(P40.05), indicating a non-signicant difference between mod- presented better AUC and NLR and the SVM a better ACC.
els, which validates the selected PC scores as input variables in the According to the standard criteria for interpreting Cohens kappa
LR classier. coefcients, the classication agreement was substantial in the
The highest PNN performance occurred with the rst three comparison between PNN and SVM, and just moderate between
and the fth vertical PCs, and the rst anteriorposterior PC LR and PNN and between LR and SVM (Table 4).
(Fig. 4a). The nal model including these ve variables was
thus maintained for comparison among model and treatment
assessment. The AUC indicated higher average performance Table 3
values within the whole range of spread constants, reaching its Average and standard error and P value for AUC, ACC and NLR.
maximum with s2 =0.8, while the ACC has the maximum with
s2 = 0.1 (Fig. 4b). To make a compromise between AUC and ACC, a LR PNN SVM P
0.1 spread was chosen. The AUC ( 7Standard Error) and the ACC at
AUC 0.9813 70.0314 0.9867 7 0.0195 0.9463 7 0.0622 o 0.001n, + ,y
this spread constant were 0.98 ( 70.019) and 0.92 ( 70.058), ACC 0.9185 70.0711 0.9273 7 0.0729 0.94607 0.0556 o 0.001n, + ,y
respectively. NLR 0.1216 70.1761 0.0354 7 0.0942 0.0569 7 0.1172 o 0.001n, + ,y

Post-hoc P o 0.05: n LR different from PNN, + LR different from SVM, y PNN

different from SVM.

Table 2
Table 4
The best SVM bootstrap training and test set area under ROC (AUC), accuracy (ACC)
Comparison between models using the Cohens kappa coefcient.
and the selected scores inputs.

Agree (%) Disagree (%) Cohens kappa

Kernel C AUC ACC Variables

LR and PNN 70 30 0.4537 Moderate agreement

Linear 1 0.9463 0.9460 1, 2, 3 and 5a
LR and SVM 67 33 0.4127 Moderate agreement
Gaussian (s = 1) 10 0.8907 0.9115 1, 4 and 7a
PNN and SVM 86.5 13.5 0.8649 Substantial
Polynomial (d =3) 1 0.8979 0.9291 1, 5 and 7a
agreement

a
Variables 15 refer to the rst to fth PC scores of the vertical GRF, while LRlogistic regression, PNNprobabilistic neural network, and SVM support
variable 7 is the rst PC score of the anteriorposterior horizontal GRF. vector machine

Fig. 5. Box plot of performance indexes of LR, PNN and SVM models obtained with 1000 bootstraps: (a) AUC; (b) ACC; and (c) NLR.
 ARTICLE IN PRESS
A.M.S. Muniz et al. / Journal of Biomechanics 43 (2010) 720726
Table 5
Comparison among logistic regression (LR), probabilistic neural network (PNN) and support vector machine (SVM) models.
LR PNN
Normal PD Normal
Mon-sof 4 (28.6%) 10 (71.4%) 4 (28.6%)
Mof-son 10 (83.3%) 2 (16.7%) 4 (33.3%)
Mon-son 10 (90.9%) 1 (9.1%) 7 (63.3%)
When assessing treatments (Table 5), the classiers ranked
treatments in the same order, with DBS-STN presenting better
results than just medication, and further GRF improvements with
combined treatments.
4. Discussion
This is the rst study that used force platforms for assessing the
effects of the DBS-STN on PD gait. Although most DBS-STN
assessments show an improvement in gait speed (Liu et al., 2005;
Ferrarin et al., 2005), none of them quantied its effect by analyzing
the entire waveform. GRF is one important indicator of joint moments
and muscle activities (Zajac et al., 2003), and is partially explained by
subjects velocity or cadence (Winter, 1991). Its improvement with
treatment as measured by force peaks represents only a preliminary
step in gait analysis (Loslever and Barbier, 1998). Indeed, it does not
allow a clear classication of subjects in different conditions.
Conversely, PCA coefcients account for information from the entire
GRF waveforms, rather than discrete parameters as in most past
studies (Li and Hamill, 2002; Hsiang and Chang, 2002), which
disregard the high correlation among those parameters (Chester et al.,
2007). Additionally, PCA provided a substantial dimension reduction
of classier inputs and simplied the use of the bootstrap technique,
which are recommended for small sample sizes (Sahiner et al., 2008).
Not all PCs selected by the broken stick method are necessarily
important for classication, since the best trade-off between
variance and classication should be taken into account (Chang,
1983; Jolliffe, 2002). Dreiseitl and Ohno-Machado (2002) also
recommend determining which variables are relevant to reach a
good model performance. The stepwise and bootstrap approaches
allowed for the selection of the PCs that maximized performance
of the three tested models, using an adequate criteria. The
comparison using AUC and ACC also allowed for the selection of
the best spread value in PNN, as recommended by Hernandez-
Caraballo et al. (2005), as well as the best kernel and C parameter
in SVM.
The bootstrap is recognized as a robust approach for estimat-
ing classier performance under a limited sample size (Sahiner et
al., 2008). According to Sahiner et al. (2008), the 0.632+ bootstrap
approach provides the lowest bias with the greatest accuracy. In
the present study, with 1000 bootstrap samples, the PNN
presented the highest AUC and the lowest NLR values compared
to LR and SVM, while SVM had the highest ACC value (Table 3).
AUC has been recommended as better performance index, since it
presents higher convergence than ACC (Brandley, 1997), and
represents the average sensitivity across all possible specicities
(Glas et al., 2003). On the other hand, ACC is still the most
commonly used index, although dependent on the prevalence of
the disorder in the studied group (Glas et al., 2003). The NLR
describes the discriminatory properties of negative test results.
According to Deeks (2009), the NLR values correspond to strong
diagnostic evidence. Thus, PNN was more rigorous than LR and
SVM in accepting a pattern as normal, which agreed with other
725
SVM Total
PD Normal PD
10 (71.4%) 2 (14.3%) 12 (85.7%) 14 (100%)
8 (66.7%) 4 (33.3%) 8 (66.7%) 12 (100%)
4 (36.7%) 7 (63.3%) 4 (36.7%) 11 (100%)
studies that compared only the LR and ANN (Nguyen et al., 2002;
Song et al., 2005).
Following Landis and Koch (1977) method for describing the
strength of agreement associated with the Cohens kappa statistics
(Table 4), the present results indicate that PNN and LR had a
moderate agreement and PNN and SVM presented a substantial
agreement. These ndings are also in agreement with Dhana-
lakshmi et al. (2009), who demonstrated that SVM and ANN can
be effectively used as classication approaches. Additionally,
these results are in line with other research (Begg et al., 2005;
Lai et al., 2009), indicating that SVM had good performance in
pathological gait pattern recognition.
When applied for PD treatment comparison, the three models
ranked results from testing conditions in the same order
indicating that DBS-STN alone was more effective than medication
alone in moving PD patients to a normal GRF pattern. The best
results occurred with the combination of treatments (Table 2).
Similarly, past studies (Ferrarin et al., 2005; Xie et al., 2001) have
reported enhancements in gait performance with stimulation
alone and further improvement when combined with medication.
Those authors suggested a synergistic effect of DBS-STN and
levodopa for PD symptoms. PNN classied fewer PD subjects as
normal, presenting reduced classication difference between
DBS-STN and medication alone (Table 2), agreeing with past
studies (Ferrarin et al., 2005; Xie et al., 2001). Moreover, PNN had
lower NLR or odds of a subject having a normal gait pattern;
therefore, being more reliable to classify the PD subjects as
normal when the GRF pattern is improved.
5. Conclusion
The LR, PNN and SVM models presented high performance
indexes for classifying GRF pattern of normal subjects and
untreated PD. When using the bootstrap approach, PNN per-
formed better according to AUC and NLR criteria and the SVM
showed the best ACC.
When evaluating the effect of treatments, the three classiers
indicated DBS-STN alone was more effective than medication
alone, with the greatest improvement with combined treatments.
However, PNN was more restrictive for accepting the patients GRF
as normal.
Conict of interest statement
None of the authors have any nancial or personal relationship
that could inappropriately inuence the work submitted for
publication.
Acknowledgements
This work was partially supported by the Brazilian Research
Council (CNPq) and by CAPES and FAPERJ foundation.
 ARTICLE IN PRESS
726 A.M.S. Muniz et al. / Journal of Biomechanics 43 (2010) 720726
References
Brandley, A.P., 1997. The use of the area under the roc curve in the evaluation of
machine learning algorithms. Pattern Recognition 30 (7), 11451159.
Begg, R.K., Palaniswami, M., Owen, B., 2005. Support vector machines for
automated gait classication. IEEE Transactions on Biomedica Engineering 53
(5), 828838.
Cardillo G., 2007. Cohens kappa: compute the Cohens kappa ratio on a 2x2 matrix.
http://www.mathworks.com/matlabcentral/leexchange/15365.
Chang, W.C., 1983. On using principal components before separating a mixture of
two multivariate normal distribuitions. Applied Statistics 32 (3), 267275.
Chester, W.L., Tingley, M., Biden, E.N., 2007. An extended index to quantify
normality of gait in children. Gait and Posture 25, 549554.
Dhanalakshmi, P., Palanivel, S., Ramalingam, V., 2009. Classication of audio
signals using SVM and RBFNN. Expert Systems with Applications 36, 6069
6075.
Deeks, J.J., 2009. Systematic reviews in health care Systematic reviews of
evaluations of diagnostic and screening tests. British Medical Journal 323,
157162.
Dreiseitl, S., Ohno-Machado, L., 2002. Logistic regression and articial neural
network classication models: a methodology review. Journal of Biomedical
Informatics 35, 352359.
Efron, B., Tibshirani, R., 1997. Improvements on cross-validation: The .632 +
bootstrap method. Journal of the American Statistical Association 92 (438),
548560.
Ferrarin, M., Rizzone, M., Bergamasco, B., Lanotte, M., Recalcati, M., Pedotti, A.,
Lopiano, L., 2005. Effects of bilateral subthalamic stimulation on gait
kinematics and kinetics in Parkinsons disease. Experimental Brain Research
160, 517527.
Glas, A.S., Lijmer, J.G., Prins, M.H., Bonsel, G.J., Bossuyt, P.M.M., 2003. The diagnostic
odds ratio: a single indicator of test performance. Journal of Clinical
Epidemiology 56, 11291135.
Hahn, M.E., Farley, A.M., Lin, V., Chou, L., 2005. Neural network estimation of
balance control during locomotion. Journal of Biomechanics 38, 717724.
Hernandez-Caraballo, E.A., Rivas, F., Perez, A.G., Marco-Parra, M., 2005. Evaluation
of chemometric techniques and articial neural networks for cancer screening
using Cu, Fe, Se and Zn concentrations in blood serum. Analytica Chimica Acta
533, 161168.
Hsiang, S, Chang, C., 2002. The effect of gait speed and load carrying on the
reliability of ground reaction forces. Safety Science 40, 639657.
Jolliffe, I.T., 2002. In: Principal Component Analysis second ed. Springer, New York.
Krack, P., Batir, A., Van Blercom, N., Chabardes, S., Fraix, V., Ardouin, A., et al., 2003.
Five-year follow-up of bilateral stimulation of the subthalamic nucleus in
advanced parkinsons disease. The New England Journal of Medicine 340 (20),
19251934.
Krzanowski, W.J., 1998. In: An Introduction to Statistical Modelling. Arnold,
London.
Lafuente, R., Belda, J.M., Sanchez-Lacuesta, J., Soler, C., Prat, J., 1997. Design and test
of neural networks and statistical classiers in computer-aided movement
analysis: a case study on gait analysis. Clinical Biomechanics 13 (3), 216229.
Lai, D.T., Levinger, P., Begg, R.K., Gilleard, W.L., Palaniswami, M., 2009. Automatic
recognition of gait patterns exhibiting patellofemoral pain syndrome using a
support vector machine approach. IEEE Transactions on Information Technol-
ogy in Biomedicine 13 (4), 810817.
Landis, J.R., Koch, G.G., 1977. The measurement of observer agreement for
categorical data. Biometrics 33 (1), 159174.
Li, L, Hamill, J., 2002. Characteristics of the vertical ground reaction force
component prior to gait transition. Research Quarterly for Exercise and Sport
73 (3), 229237.
Liu, W., Mcintire, K., Kim, S.H., Zhang, J., Dascalos, S., Lyons, K.E., Pahwa, R., 2005.
Quantitative assessments of the effect of bilateral subthalamic stimulation on
multiple aspects of sensoriomotor function for patients with Parkinsons
disease. Parkinsonism and Related Disordens 11, 503508.
Loslever, P., Barbier, F., 1998. Multivariate graphical presentation for gait
rehabilitation study. Gait and Posture 7, 3944.
Nguyen, T., Malley, R., Inkelis, S., Kuppermann, N., 2002. Comparison of prediction
models for adverse outcome in pediatric meningococcal disease using articial
neural netweork and logistic regression analyses. Journal of Clinical Epide-
miology 55, 687695.
Ostergaard, K., Sundae, N.A., 2006. Evolution of Parkinsons disease during 4 years
of bilateral deep brain stimulation of the subthalamic nucleus. Movement
Disorders 21 (5), 624631.
Sahiner, B., Chan, H., Hadjiiski, L., 2008. Classier performance prection for
computer-aided diagnosis using a limited dataset. Medical Physics 35 (4),
15591570.
Schumacher, M., Robner, R., Wemer, V., 1996. Neural networks and logistic
regression. Part I Computational Statistics and Data Analysis 21, 661682.
Schutte, L.M., Narayanan, U., Stout, J.L., Selber, P., Gage, J.R., 2000. An index for
quantifying deviations from normal gait. Gait and Posture 11, 2531.
Schwartz, M.H., Rozumalski, A., 2008. The gait deviation index: A new
comprehensive index of gait pathology. Gait and Posture 28, 351357.
Song, J.H., Venkatesh, S.S., Conant, E.A., Arger, P.H., Sehgal, Chandra M., 2005.
Comparative analysis of logistic regression and articial neural network
for computer-aided diagnosis of breast masses. Academic Radiology 12,
487495.
Specht, D.F., 1990. Probabilistic neural networks. Neural Networks 3, 109118.
Specht, D., Romsdahl, H., 1994. Experience with adaptive probabilistic neural
networks and adaptive general regression neural networks. Neural Networks
12031208.
Su, F., Wu, W., 2000. Design and testing of a genetic algorithm neural network in
the assessment of gait patterns. Medical Engineering and Physics 22, 6774.
Vapnik, VN., 2000. In: The Nature of Statistical Learning Theory second ed.
Springer, New York.
Winter, D.A., 1991. In: The Biomechanics and Motor Control of Human Gait:
Normal, Elderly and Pathological second ed. University of Waterloo Press,
Waterloo.
Xie, J., Krack, P., Benabid, A., Pollak, P., 2001. Effect of bilateral subthalamic nucleus
stimulation on parkinsonian gait. Journal of Neurology 248, 10681072.
Zajac, F.E., Sachin, R.R., Kautz, S.A., 2003. Biomechanics and muscle coordination of
human walking part II: lessons from dynamical simulations and clinical
implications. Gait and Posture 17, 117.