Online data supplement

Exhaled nitric oxide as a marker of asthma control in

smoking patients

____________________________________________

Alain Michils1, MD; Renaud Louis2, MD; Rudi Peché3, MD; Sandra
Baldassarre1, MD; Alain Van Muylem1, PhD

1
Chest Department - CUB Erasme, Brussels, Belgium
2
Chest Department - CHU Sart-Tilmant, Liège, Belgium
3
Chest Department- CHU André Vésale, Montigny- le-Tilleul, Belgium
METHODS

Statistical methods

1. Intrinsic characteristics of an index

1.1. Sensitivity and specificity

If N events are rated as positive or negative (e.g. positive = well-controlled asthma; negative

= not-well controlled asthma) by a reference method (Gold Standard: GS), the ability of an

index to discriminate between these complementary states, for a given cut-off value, may be

assessed by using a contingency table. The index is considered as positive or negative

depending on its position relative to the cut-off value. The contingency table appears like

GS positive GS negative
n+ n-

Index positive TP FP

Index negative FN TN

where n+ an n- are the amounts of positive and negative events (N = n+ + n-) according to the

Gold standard, respectively. TP and TN are the amounts of true positive and true negative

events, respectively; i.e. the amounts of events correctly rated by the index. FP and FN are the

amounts of false positive and false negative events, respectively, i.e. the amounts of events

incorrectly rated by the index.

From the contingency table, sensitivity (Se) and specificity (Sp) may be derived by

Se = TP/n+ (1)

Sp = TN/n- (2)
Se and Sp are intrinsic characteristics of the index (for this given cut-off value). Se and Sp are
independent from the prevalence, the latter being defined as the positive events rate in a
considered context (the "prior" probability of a positive event). From the contingency table,
the prevalence is equal n+/N.
1.2. ROC curve

A contingency table may be constructed, and sensitivity and specificity computed, for a range
of cut-off values. The Receiving Operator Characteristics (ROC) curve is the true positive rate
(sensitivity) as a function of the false positive rate (1-specificity) for the considered range of
cut-off values. This curve allows visually assessing the relative "quality" of several indexes.
It also allows "choosing" an optimal cut-off value for a given index according to investigator's
criteria.

The area under the ROC curve (AUC) may be computed and its difference relative to 0.5
statistically assessed. If AUC not different from 0.5 (which is the area under the first bisector),
it means that the considered index is as useful as tossing a coin.

1.3. The Youden's index

The Youden's index (J), is the difference between the true positive rate and the false positive

rate. Maximizing this index allows to find, from the ROC curve, an optimal cut-off point

independently from the prevalence. According to its definition and as illustrated on Fig.1, J is

the vertical distance between the ROC curve and the first bisector (or chance line). If F(x) is

the function describing the ROC curve, with x = 1-specificity, we may write

J(x) = F(x)-x (3)

When J is maximal, J'(x) = 0, where J' is the derivative of J.

From Eq. 1: J'(x)= F'(x) -1, (4)

where F' is the derivative of F.

Hence, when J is maximal, F'(x) = 1, meaning that the tangent to the ROC curve is parallel to

the first bisector (slope = 1). It implies that, around this point, a gain (or a loss) in specificity

results in a loss (or a gain) of the same amplitude in sensitivity.

 Figure 1

100 urv
e
Cc
RO

80 Youden index
(true positive rate)
Sensitivity (%)

60
true positive
rate
40
or
ct
se
is

20 false positive
b
st

rate
fir

0
0 20 40 60 80 100

100-specificity (%)
(false positive rate)

2. Predictive values

Positive (PPV) and negative (NPV) predictive values are the probability to rate correctly an

event using the index; i.e. the probability that the event is actually positive (negative) if the

index is positive (negative). They may be derived from the contingency table by

PPV = TP/(TP+FP) (5)

NPV = TN/(TN+FN) (6)

Contrary to Se and Sp, PPV and NPV are dependent on the context through the prevalence. It

implies that PPV and NPV, derived from the contingency table, may be used only if n+/N is

close to the actual prevalence of the considered event in the considered "real-life" context.

The accuracy (Acc) defined as (TP+TN)/N is also context dependent.

It is to be noted that PPV and NPV may be computed from Se and Sp (intrinsic

characteristics) and any given prevalence P (linked to a specific context) by the Baye's

formula:
P  Se
PPV  (7)
P  Se  (1  P)  (1  Sp)

NPV 
1  P   Sp
(8).
P  Se  1  P   1  Sp 
RESULTS

Operating characteristics of FeNO at various cut-off values

Table 1: Cross-sectional assessment of asthma control

Non-smokers (N=411, n+=197)

p<0.001
Cut-off 30 40 50* 60 70
(ppb)
TP 112 134 142 171 179
TN 150 131 120 86 58
FP 64 83 94 128 156
FN 85 63 55 26 18
Se (%) 57 68 72 87 91
Sp (%) 70 61 56 40 27
PPV (%) 64 62 61 57 53
NPV (%) 64 68 68 77 76
Acc (%) 64 64 64 63 58
Smokers (N=59, n+=15)
p=0.39
Cut-off 15 20 25* 35 45
(ppb)
TP 7 10 10 11 12
TN 30 26 21 14 11
FP 14 18 23 30 33
FN 8 5 5 4 3
Se (%) 48 64 66 76 80
Sp (%) 68 59 48 32 24
PPV (%) 33 36 30 27 27
NPV (%) 79 84 81 78 79
Acc (%) 63 61 53 42 39
*: cut-off value corresponding to maximal Youden's index. N and n+ are the total amount of cases
and the amount of positive cases, respectively. A positive case is a (partially or totally) controlled
asthma. TP, TN, FP, FN are the amount of true positive, true negative, false positive, and false
negatives cases, respectively. Se, Sp, PPV, NPV, and Acc are the sensitivity, specificity, positive
predictive value, negative predictive value, and accuracy, respectively; p is the statistical significance
of rejecting the hypothesis AUC=0.5. A true positive event is defined as FeNO<cut-off associated with
a (partially or totally) controlled asthma.
Table 2: Assessment of a change from an uncontrolled (ACQ score  1.5) to a controlled
(ACQ < 1.5) asthma

Non-smokers (N=283, n+=133)

p<0.001
Cut-off -20 -25 -30* -35 -40
(%change)
TP 101 96 90 88 82
TN 93 98 107 113 120
FP 57 52 43 37 30
FN 32 37 43 45 51
Se (%) 76 72 68 66 62
Sp (%) 62 65 71 75 80
PPV (%) 64 65 68 70 73
NPV (%) 74 73 72 72 70
Acc (%) 69 69 70 71 71
Smokers (N=52, n+=17)
p=0.016
Cut-off 15 20 25* 35 45
(%change)
TP 12 12 12 11 9
TN 22 23 23 23 24
FP 13 12 12 12 11
FN 5 5 5 6 8
Se (%) 71 71 71 65 53
Sp (%) 64 65 66 66 69
PPV (%) 48 50 50 48 45
NPV (%) 81 82 82 79 75
Acc (%) 65 67 67 65 63
*: cut-off value corresponding to maximal Youden's index. N and n+ are the total amount of cases
and the amount of positive cases, respectively. A positive case is an uncontrolled asthma becoming
controlled. TP, TN, FP, FN are the amount of true positive, true negative, false positive, and false
negatives cases, respectively. Se, Sp, PPV, NPV, and Acc are the sensitivity, specificity, positive
predictive value, negative predictive value, and accuracy, respectively; p is the statistical significance
of rejecting the hypothesis AUC=0.5. A true positive event is defined as FeNO change <cut-off (e.g.
-40% vs -30%) associated with a positive case.
Table 3: Assessment of a change from a controlled (ACQ score < 1.5) to an uncontrolled
(ACQ 1.5) asthma

Non-smokers (N=360, n+=65)

p=0.001
Cut-off 40 45 50* 55 60
(%change)
TP 29 29 27 27 26
TN 212 215 221 224 227
FP 83 80 74 71 68
FN 36 36 38 38 39
Se (%) 44 44 42 41 40
Sp (%) 72 73 75 76 77
PPV (%) 26 27 26 28 28
NPV (%) 85 86 86 85 85
Acc (%) 67 68 69 70 70
Smokers (N=40, n+=10)
p=0.017
Cut-off 40 45 50* 55 60
(%change)
TP 7 7 7 6 6
TN 24 25 26 26 26
FP 6 5 4 4 4
FN 3 3 3 4 4
Se (%) 70 70 68 62 58
Sp (%) 80 84 87 87 88
PPV (%) 54 58 63 60 60
NPV (%) 89 89 89 87 87
Acc (%) 78 80 83 80 80
*: cut-off value corresponding to maximal Youden's index. N and n+ are the total amount of cases
and the amount of positive cases, respectively. A positive case is a controlled asthma becoming
uncontrolled. TP, TN, FP, FN are the amount of true positive, true negative, false positive, and false
negatives cases, respectively. Se, Sp, PPV, NPV, and Acc are the sensitivity, specificity, positive
predictive value, negative predictive value, and accuracy, respectively; p is the statistical significance
of rejecting the hypothesis AUC=0.5. A true positive event is defined as FeNO change >cut-off (e.g.
60% vs 50%) associated with a positive case.
Table 4: Improvement (ACQ < -0.5) assessment of asthma control

Non-smokers
Total ICS dose 500$ ICS dose 500$
(N=643, n+=257) (N=306, n+=116 (N=337, n+=97)
p<0.001 p<0.001 p<0.001
Cut-off -10 -15 -20* -25 -30 -10 -15 -20* -25 -30 -10 -15 -20* -25 -30
(%change)
TP 146 177 164 159 149 85 82 86 73 68 100 96 78 82 79
TN 274 255 274 282 297 127 135 127 143 150 108 122 145 133 141
FP 112 131 112 104 89 63 55 63 47 40 88 74 51 63 55
FN 111 80 93 98 108 31 34 30 43 48 41 45 63 59 62
Se (%) 57 69 64 62 58 73 71 74 63 59 71 68 55 58 56
Sp (%) 71 66 71 73 77 67 71 67 75 79 55 62 74 68 72
PPV (%) 57 57 61 62 63 57 60 58 61 63 53 56 60 57 59
NPV (%) 71 76 74 74 73 80 80 80 77 76 72 73 70 69 69
Acc (%) 65 67 68 69 69 69 71 70 71 71 62 65 66 64 65
Smokers
Total ICS dose 500$ ICS dose >500$
(N=92, n+=40) (N=35, n+=14) (N=57, n+=26)
p<0.001 p<0.001 p=0.070
Cut-off -10 -15 -20* -25 -30 -10 -15 -20* -25 -30 -10 -15 -20* -25 -30
(%change)
TP 25 24 23 20 20 12 12 9 8 8 14 13 13 12 11
TN 37 37 38 40 42 16 16 18 18 18 21 22 22 22 23
FP 15 15 14 12 10 5 5 3 3 3 10 9 9 9 8
FN 15 16 17 20 20 2 2 5 6 6 12 13 13 14 15
Se (%) 62 60 57 51 49 88 86 62 54 54 54 50 50 46 42
Sp (%) 72 72 74 76 80 74 74 84 84 84 68 71 71 71 74
PPV (%) 63 62 62 63 67 71 71 75 73 73 58 59 59 57 58
NPV (%) 71 70 70 67 68 89 89 78 75 75 64 63 63 61 61
Acc (%) 67 66 66 65 67 80 80 77 74 74 61 61 61 60 60
*: cut-off value corresponding to maximal Youden's index. $ : ICS dose in g equ BDP.day-1 . N and
n+ are the total amount of cases and the amount of positive cases, respectively. A positive case is a
decrease amplitude of ACQ score > 0.5. TP, TN, FP, FN are the amount of true positive, true negative,
false positive, and false negatives cases, respectively. Se, Sp, PPV, NPV, and Acc are the sensitivity,
specificity, positive predictive value, negative predictive value, and accuracy, respectively; p is the
statistical significance of rejecting the hypothesis AUC=0.5. A true positive event is defined as FeNO
change <cut-off (e.g. -30% vs -20%) associated with a positive case.
Table 5: Worsening (ACQ > 0.5) assessment of asthma control

Non-smokers
Total ICS dose 500$ ICS dose 500$
(N=643, n+=161) (N=306, n+=64) (N=337, n+=97)
p<0.001 p<0.001 p<0.001
Cut-off *
(%change) 20 25 30 35 40 20 25 30* 35 40 20 25 30* 35 40
TP 90 85 82 76 72 43 43 43 39 38 46 43 41 38 36
TN 342 357 366 376 381 169 179 184 189 191 182 182 187 190 194
FP 140 125 116 106 101 73 63 58 53 51 58 58 53 50 46
FN 71 76 79 85 89 21 21 21 25 26 51 54 56 59 61
Se (%) 56 53 51 47 45 67 67 67 61 59 47 44 42 39 37
Sp (%) 71 74 76 78 79 70 74 76 78 79 76 76 78 79 81
PPV (%) 37 40 37 42 42 37 41 43 42 43 44 43 44 43 44
NPV (%) 84 82 84 82 81 89 90 90 88 88 78 77 77 76 76
Acc (%) 67 69 70 70 70 69 73 74 75 75 68 67 68 68 68
Smokers
Total ICS dose 500$ ICS dose >500$
(N=92, n+=26) (N=35, n+=11) (N=57, n+=15)
p<0.001 p<0.001 p=0.037
Cut-off *
(%change) 20 25 30 35 40 20 25 30* 35 40 20 25 30* 35 40
TP 17 17 17 17 16 8 8 8 8 8 10 10 10 10 9
TN 46 50 51 51 52 20 21 22 22 23 28 30 30 30 30
FP 20 16 15 15 14 4 3 2 2 1 14 12 12 12 12
FN 9 9 9 9 10 3 3 3 3 3 5 5 5 5 6
Se (%) 67 67 67 66 63 70 70 70 70 70 64 64 64 64 57
Sp (%) 70 76 77 78 79 82 86 91 93 96 66 71 71 71 71
PPV (%) 46 52 52 53 53 67 73 78 80 89 42 43 43 43 43
NPV (%) 84 85 86 85 84 87 88 87 88 88 85 85 85 85 83
Acc (%) 68 73 74 74 74 80 83 86 86 89 67 70 70 70 68
*: cut-off value corresponding to maximal Youden's index. $ : ICS dose in g equ BDP.day-1 . N and
n+ are the total amount of cases and the amount of positive cases, respectively. A positive case is a
increase amplitude of ACQ score > 0.5. TP, TN, FP, FN are the amount of true positive, true negative,
false positive, and false negatives cases, respectively. Se, Sp, PPV, NPV, and Acc are the sensitivity,
specificity, positive predictive value, negative predictive value, and accuracy, respectively; p is the
statistical significance of rejecting the hypothesis AUC=0.5. A true positive event is defined as FeNO
change >cut-off (e.g. 40% vs 30%) associated with a positive case.