Professional Documents
Culture Documents
7/16/2009
Summary
The Experimental Design section of STATGRAPHICS contains a set of procedures that support
the design and analysis of many different types of experiments. These procedures enable the
analyst to construct a set of experimental runs that will yield the maximum amount of
information about a process in the smallest number of trials. In contrast to haphazard
experimentation, designed experiments are characterized by systematic manipulation of a
process in order to determine the effects attributable to different factors.
In the early stages of an investigation, the analyst is often faced with a long list of factors that
could affect the process. For example, in a typical chemical process, there could easily be
dozens of factors which have an impact on the yield of the process, such as the temperature at
which it is run, the amount of catalyst added, the mixing speed, and so on. Since it is difficult to
study many factors in detail simultaneously, screening designs have been developed to quickly
determine which factors have the greatest impact on a process.
This document describes the construction and analysis of designs that are intended to identify the
most important factors. After the critical factors are determined, a more complicated
experimental design involving a larger set of factor levels may be constructed to find the optimal
settings for those factors.
Example
As an example, a typical screening experiment will be constructed involving 5 factors and 1
response. The example, which involves a chemical reaction, is discussed in Chapter 12 of the
well-known book by Box, Hunter and Hunter (1976). The factors that will be varied are:
Y: percent reacted
The first step of the design creation process displays a dialog box used to specify the response
variables. For the current example, there is a single response variable:
Impact: The relative importance of each response (not relevant if only one response).
Sensitivity: The importance of being close to the best desired value (in this case, the
Maximum). Setting Sensitivity to Medium implies that the desirability attributed to the
response rises linearly between the Minimum and Maximum values indicated.
The second step displays a dialog box on which to specify the factors that will be varied. In the
chemical reaction example, there are 5 factors:
Type – Set the type of each component to Continuous, since they can be set at any value
within a continuous interval.
Since all of the factors are controllable process factors, only one Options button is enabled.
Pressing that button displays a second dialog box:
1. Screening - designs intended to select the most important factors affecting a response.
Most of the designs involve only 2 levels of each factor. The factors may be quantitative
or categorical.
2. Response Surface - designs intended to select the optimal settings of a set of experimental
factors. The designs involve at least 3 levels of the experimental factors, which must be
quantitative.
If Screening Designs is selected, a third dialog box will be displayed listing all of the screening
designs available for 5 experimental factors:
Name - the design name, including an abbreviation such as 2^5 if relevant. For screening
designs, the following types may appear in the list, depending upon the number of
experimental factors:
1. Factorial - includes runs at all combinations of the low and high levels of each factor,
for a total of 2k runs. Such designs are capable of estimating the main effects of all
factors and all interactions amongst the factors.
2. Factorial in m blocks - includes the same runs as a full factorial design. However, the
runs are divided into blocks, which are groups of runs to be done together (on the
same day, or by the same operator, or from the same batch of raw material) to
eliminate the effect of one or more nuisance factors. As the number of blocks
increases, the ability to estimate certain interactions is lost. The Alias Structure table
displayed after the design is initially created shows which interactions are confounded
with block effects.
3. Half fraction (or quarter, eighth, …) - a subset of the runs in a full factorial design,
either one-half of the full 2k runs, one-fourth of the runs, one-eighth of the runs, or
some other regular fraction. The number of runs in the design equals 2k-p, where p =
1 for a half-fraction, p = 2 for a quarter fraction, p = 3 for an eighth fraction, etc. For
such designs, the Resolution field indicates important information about what order of
4. Irregular fraction - fractional factorial designs in which the number of runs is not a
power of 2. Certain irregular fractions, although not completely orthogonal, have
attractive confounding patterns. The designs included here are those described by
Haaland (1989).
5. Mixed level fraction - in contrast to all of the other screening designs, these designs
allow one factor (factor A) to be run at 3 levels rather than 2. This allows a quadratic
effect to be estimated for that factor, which must be quantitative. For the other
factors, the runs form a standard fractional factorial design. The designs included are
those described by Haaland (1989).
Runs - the number of runs in the base design, before adding any additional replicates or
centerpoints.
Resolution - an indication of the confounding pattern of the design. Designs are classified as
having one of the following resolutions:
Resolution III: designs which confound the estimates of the main effects with two-factor
interactions. Such designs can be safely interpreted only if all two-factor interactions are
small or non- existent.
Resolution IV: designs which are capable of obtaining clear estimates of all main effects.
However, some or all of the two-factor interactions are confounded with other two-factor
interactions or block effects. The Alias Structure table described below indicates where
the confounding occurs.
Resolution V: designs which are capable of obtaining clear estimates of all main effects
and all two-factor interactions. Higher order interactions, however, are confounded with
these effects. In most cases, this is not a problem since third-order and higher effects are
Resolution V+: the design has resolution greater than 5, allowing for the estimation of 3-
factor or higher order interactions if desired.
For blocked designs, an asterisk is shown next to the design resolution to indicate that the
stated resolution assumes that blocking factors do not interact with experimental factors, the
standard assumption when the analysis is performed.
Error d.f. - the number of degrees of freedom from which the experimental error may be
estimated remaining after estimating all main effects, second-order interactions, and
quadratic effects (if relevant). This is prior to any replication or addition of centerpoints. In
general, at least 3 d.f. must be available if the statistical tests to be performed during the
analysis are to have reasonable statistical power.
Block Size - for a design that is run in more than 1 block, the number of runs in the largest
block.
For the current example, a 16 run half-fraction will be selected. This design is resolution V,
which means it is capable of estimating all main effects and two-factor interactions. However,
there are 0 degrees of freedom remaining to estimate the experimental error. In order to do
formal statistical testing, additional runs will need to be added to the base design.
The final dialog box allows the analyst to add additional runs to the design and to specify the
order in which the runs will be performed:
Centerpoints (number) - the number of centerpoints to be added to the base design, which
are additional experimental runs located at a point midway between the low and high level of
all the factors. Each additional centerpoint adds one degree of freedom from which to
estimate experimental error. If the design involves a single categorical factor, the
centerpoints will be placed at a middle level of the quantitative factors and divided equally
between the two levels of the categorical factor.
2009 by StatPoint Technologies, Inc. DOE Wizard – Screening Designs - 7
STATGRAPHICS – Rev. 7/16/2009
Centerpoints (placement) - positioning of the centerpoints with respect to the runs in the
base design. They may be randomly scattered throughout the other experimental runs,
spaced evenly throughout the other runs, or placed at the beginning or end of the experiment.
The first two options are usually preferable.
Replicate design - if a number other than 0 is entered, the entire design will be repeated the
indicated number of times.
Randomize - check this box to randomly order the runs in the experiment. Randomization is
generally a good idea, since it can reduce the effect of lurking variables such as trends over
time. However, when replicating the examples in this documentation, do not randomize the
designs.
Generate Button - this button displays a dialog box that allows experienced analysts to
change the design generators for fractional factorial designs:
In order to generate a 2k-p fractional factorial design, a full factorial design for k - p factors is
first generated. Columns for the additional p factors are then created by multiplying together
various combinations of columns of the initial factorial. In the current example, the column
for factor E is created by multiplying column A by column B by column C by column D.
This is abbreviated as
E = ABCD (1)
Alternatively, column E could have been created by multiplying together the same four
columns and then changing the sign, i.e.,
2009 by StatPoint Technologies, Inc. DOE Wizard – Screening Designs - 8
STATGRAPHICS – Rev. 7/16/2009
E = -ABCD (2)
which would result in a different set of 16 runs. For details of these procedures, see Box,
Hunter and Hunter (1976).
Note that 3 centerpoints have been added to the base design in the current example, resulting in a
total of 19 runs and providing 3 degrees of freedom from which to estimate the experimental
error. Since the Spaced option has been selected, the three centerpoints will be positioned at the
beginning, middle, and end of the experiment. The tentatively selected design is displayed in the
Select Design dialog box:
If the design is acceptable, press OK to save it to the STATGRAPHICS DataBook and return to
the DOE Wizard’s main window, which should now contain a summary of the design:
Before evaluating the properties of the design, a tentative model must be specified. Pressing the
fourth button on the DOE Wizard’s toolbar displays a dialog box to make that choice:
Since we intend to run all of the runs in the base design, this step can be omitted.
Design Properties
Several of the selections presented when pressing button #6 are helpful in evaluating the selected
design:
Design Worksheet
The design worksheet shows the 19 runs that have been created, in the order they are to be run:
Note that 3 centerpoints have been added to the 16 runs in the base design, one at the beginning
of the experiment, one halfway through, and one at the end.
ANOVA Table
The ANOVA table shows the breakdown of the degrees of freedom in the design:
ANOVA Table
Source D.F.
Model 15
Total Error 3
Lack-of-fit 1
Pure error 2
Total (corr.) 18
15 of the 18 total degrees of freedom are used to estimate the main effects and two-factor
interactions.
Model Coefficients
Since the design is perfectly orthogonal, all of the variance inflation factors (VIF) are equal to
their ideal value of 1.0. The rightmost column shows that there is a 75.5% chance of detecting
any effects with a magnitude equal to 2 times the standard deviation of the experimental error.
Correlation Matrix
The correlation matrix has 0’s in all the off-diagonal locations, showing that the estimates of the
main effects and two-factor interactions will all be uncorrelated.
Correlation Matrix
A B C D E AB AC AD AE BC BD BE CD CE DE
A 1.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000
B 0.000 1.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000
C 0.000 0.000 1.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000
D 0.000 0.000 0.000 1.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000
E 0.000 0.000 0.000 0.000 1.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000
AB 0.000 0.000 0.000 0.000 0.000 1.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000
AC 0.000 0.000 0.000 0.000 0.000 0.000 1.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000
AD 0.000 0.000 0.000 0.000 0.000 0.000 0.000 1.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000
AE 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 1.000 0.000 0.000 0.000 0.000 0.000 0.000
BC 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 1.000 0.000 0.000 0.000 0.000 0.000
BD 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 1.000 0.000 0.000 0.000 0.000
BE 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 1.000 0.000 0.000 0.000
CD 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 1.000 0.000 0.000
CE 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 1.000 0.000
DE 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 1.000
The graph of the design points shows that each pair of factors is run at all combinations of the
high and low levels:
15
feed rate
10
2
catalyst
1
120
agitation
100
180
temperature
140
6
concentration
3
The prediction variance plot shows that the variance of the predicted response will be fairly
constant over most of the experimental region;
1
0.8
Stnd. error
0.6
0.4
0.2
2
0 1.8
1.6
10 1.4
11 12 13 1.2
14 15 1 catalyst
feed rate
The only location where the variance is relatively high is close to the vertices.
Once the experiment has been created and any additional runs entered, it must be saved on disk.
Press the button labeled Step 7 and select a name for the experiment file:
Design files are extended data files and have the extension .sgx. They include the data together
with other information that was entered on the input dialog boxes.
To reopen an experiment file, select Open Data File from the File menu. The data will be loaded
into the datasheet, and the Experimental Design Wizard window will be displayed.
Important Notes:
1. If more than one sample was taken at each set of experimental conditions, the data values
should be entered into data tables B through Z. The summary statistics in data table A
will then be automatically calculated from the other tables. Do not treat the samples as
replicates unless you actually reset the process between each sample.
2. If any experiments were not performed, leave the corresponding cell blank. The program
will recognize the imbalance in the design and handle it.
3. If any experimental runs were done at conditions different than originally planned,
change the entries in the experimental factor columns to correspond to the values that
were actually used.
4. If additional runs were performed, you may add them to the bottom of the datasheet.
They will be included in the fit.
Statistical Model
The statistical model upon which the analysis of screening designs is based expresses the
response variable Y as a linear function of the experimental factors, interactions between the
factors, and an error term. There are two types of models that are generally fit, illustrated below
for 5 experimental factors:
The experimental error is assumed to be randomly drawn from a normal distribution with a
mean of 0 and a standard deviation equal to . In rare cases, interactions amongst 3 or more
factors may be included by adding terms consisting of cross-products of more than 2 factors. For
the occasional screening design that has 3 levels of a factor, a term such as 11 X 12 would also be
included in the second-order model.
For quantitative variables, STATGRAPHICS represents the experimental factor Xj using its
original values as entered into the datasheet. For categorical factors, indicator variables are used
of the form
where the “low” and “high” levels are those defined when the design was constructed.
Effects
In order to simplify the interpretation of screening designs, it is common to reexpress the above
model in terms of “effects”. The main effect of factor j is defined as the change in the response
variable Y when Xj is changed from its low level to its high level, with all other factors being
held constant midway between their lows and their highs. In a balanced two-level factorial
design, the estimated effect of factor j equals the difference between the average response at the
high level of the factor and the average response at the low level of the factor:
j y y (6)
2009 by StatPoint Technologies, Inc. DOE Wizard – Screening Designs - 17
STATGRAPHICS – Rev. 7/16/2009
where
In an unbalanced design, the effect is a more complicated function of the coefficients, but the
basic interpretation remains the same.
Two-factor interactions may also be defined. In general, a two-factor interaction may be thought
of as the additional effect of one factor over and above the main effect when the second factor is
held at its high level. In a balanced two-level factorial design, this interaction effect equals
ˆ jk y y ˆ j ˆ k (7)
where
y i
y i 1
(8)
n
An additional important characteristic of the effects is that they are all expressed in units of the
response variable, so that effects of factors can be compared directly, regardless of the units in
which the factors are expressed.
Once the data have been entered, press the button labeled Step #8 on the Experiment Design
Wizard toolbar. This will display a dialog box listing each of the response variables:
If more than one response has been measured, you should repeat this step once for each response.
Analysis Summary
The analysis of a screening design involves estimating the average or main effect of each
experimental factor and interactions between the factors. The Analysis Summary displays
information about the estimated effects:
Average - the estimated response at the center of the design region. For complete data from
most orthogonal designs, this equals the grand average of all the data values y .
Estimated main effects - the difference between the response at the high level of a factor
and the response at the low level of a factor, when all other factors are held at their central
values.
Estimated 2-factor interactions - the additional effect of one factor when a second is held at
its high level. Interactions occur when the effect of one factor is different at different levels
of another factor.
Other effects - defined as twice the coefficient associated with the corresponding term in the
regression model when all variables are standardized according to:
X j low j
X j (9)
high j low j
Standard errors - Each effect is shown by default with its estimated standard error. The
standard errors are measures of the estimation error associated with each effect. The display
can be changed to show confidence intervals for each effect using the Analysis Options
dialog box.
Standard errors are based on … - an indication of how the experimental error has been
estimated, as determined by the Analysis Options dialog box.
Pareto Chart
The Pareto Chart shows a graphical depiction of each of the effects in the above table. There are
two forms of the Pareto chart: a standardized form and an unstandardized form. The
unstandardized chart displays the absolute value of the effects in decreasing order:
B:catalyst
D:temperature +
BD -
DE
E:concentration
CE
A:feed rate
BC
AB
BE
AE
AD
AC
CD
C:agitation
0 4 8 12 16 20 24
Effect
The color of the bars shows whether an effect is positive or negative. From the above plot, it is
easy to see that the three most important factors in the example are catalyst, temperature, and
concentration.
To create a standardized Pareto chart, each effect is converted to a t-statistic by dividing it by its
standard error. These standardized effects are then plotted in decreasing order of absolute
magnitude:
B:catalyst
D:temperature +
BD -
DE
E:concentration
CE
A:feed rate
BC
AB
BE
AE
AD
AC
CD
C:agitation
0 5 10 15 20 25
Standardized effect
In addition, a line is drawn on the chart beyond which an effect is statistically significant at a
specified significance level, usually 5%. In the above chart, the main effects of factors B, D, and
E are significant, as are the BD and DE interactions. Noticeably absent from the list are any
effects involving factors A and C.
Pane Options
Standardize: check to plot the standardized effects rather than the absolute effects.
Alpha: the significance level corresponding to the vertical line on the chart. Bars extending
beyond the line are statistically significant at the significance level.
ANOVA Table
To determine the level of significance for each effect, the ANOVA Table may be used:
The ANOVA partitions the variance of the response into several components: one for each main
effect, one for each interaction, and one for the experimental error. The ANOVA table shows:
Sum of Squares - the Type III sums of squares attributable to each term in the model. This
measures the increase in the variance of the experimental error that would occur if each term
was separately removed from the model. The sum of squares for total error is also included,
where
n n
S error ei2 ( yi y i ) 2 (10)
i 1 i 1
ei is the i-th residual, measuring the difference between the observed response for run i and
the value predicted by the fitted model.
Mean Square - the mean square associated with each term, obtained by dividing the
associated sum of squares by its degrees of freedom. The mean squared error (MSE)
estimates the variance of the experimental error:
Serror
ˆ 2 MSE (11)
df error
F-Ratio - an F ratio which divides the mean square of an effect by the mean squared error:
2009 by StatPoint Technologies, Inc. DOE Wizard – Screening Designs - 23
STATGRAPHICS – Rev. 7/16/2009
MSeffect
F (12)
MSE
The F-ratios may be used to determine the statistical significance of each effect.
P-Value - the P-Value associated with testing the null hypothesis that the coefficient for a
selected effect equals 0, implying that the effect is not present. P-Values below a critical
level (such as 0.05 if operating at the 5% significance level) indicate that the corresponding
effect is statistically significant at that significance level.
R-squared - the percentage of the variability in the response variable that has been
accounted for by the fitted model, calculated from
S error
R 2 100 1 % (13)
S total
R-squared ranges from 0% to 100% and measures how well the model fits the observed
response data.
R-squared (adjusted for d.f.) - the adjusted R-squared, which accounts for the number of
degrees of freedom in the fitted model. In situations such as the current one where the
number of coefficients in the fitted model is large relative to the total number of runs, the
ordinary R-squared statistic may overstate the ability of the fitted model to predict the
response. The adjusted R-squared compensates for this effect by
n 1 S error
2
Radj 100 1 % (14)
n p S total
Standard error of est. - the estimated standard deviation of the experimental error, given by
MSE (15)
This value is used when constructing prediction intervals for the response.
Mean absolute error - the average of the absolute values of the residuals, given by
n
|e |
i 1
i
MAE (16)
n
This value indicates the average error in predicting the observed response using the fitted
model.
(e
i 1
i 1 ei ) 2
DW n (17)
e
i 1
2
i
The Durbin-Watson statistic measures serial correlation in the residuals to determine whether
there is any dependence between successive observations. In this case, it could detect drifts
over the course of the experiment. A small P-value would indicate that the analyst should
take a close look at the residuals to look for any trends, which may be done using the
Diagnostic Plots graph option.
There are five effects in the ANOVA table with P-values below 0.05. These are the same five
effects identified as significant on the standardized Pareto chart (the two methods are
equivalent). As a whole, the model accounts for at least 98% of the observed variability in the
response. It is unnecessarily complicated, however, since many effects are not statistically
significant. A later section illustrates how to remove selected effects using Analysis Options.
Pane Options
Include Lack-of-Fit Test: If checked, a line will be added to the ANOVA table to determine
whether the current model adequately represents the observed data. Note: this option has no
effect unless there are replicate experimental runs at identical settings of the experimental
factors. The resulting ANOVA table is shown below:
1. Pure error: an estimate calculated by pooling the variance within sets of observations at
identical levels of X. It is “pure” in the sense that it estimates the experimental error
whether or not the proper model has been selected.
2. Lack-of-fit: an estimate calculated from the deviation between the average response for
each group of replicate values and the values predicted by the fitted model. If the model
is not correct, this estimates plus a positive quantity that measures the lack-of-fit of the
selected model.
The P-Value in the lack-of-fit line may be used to test the hypothesis that the current model is
adequate. A small P-Value would indicate an inadequate model. In the current example, the P-
Value is well above 0.05, so the selected model appears to be adequate.
80
50
20
5
1
0.1
-12 -2 8 18 28
Standardized effects
In this plot, the standardized effects are ordered from smallest to largest and plotted versus
quantiles of a normal distribution. Any estimates which are just noise will fall approximately
along a straight line. Any estimates which correspond to real signals will lie off the line to the
left or right.
Two types of normal probability plots are available, a full normal and a half-normal, which may
be chosen using Pane Options.
Plot Type: Select Normal to plot each effect while retaining it positive or negative sign.
Select Half-Normal to plot the absolute values of the effects.
Direction: Select Horizontal to plot percentages on the horizontal axis or Vertical to plot
them on the vertical axis.
Fitted Line: If checked, a reference line is added to the plot by fitting a least squares
regression line to the smallest 50% of the effects.
Label Effects: If checked, the names of the effects are added to the plot.
2.4
B:catalyst
2
Standard deviations
1.6 D:temperature
BD
1.2 DE
E:concentration
CE
0.8 A:feed rate
BC
AB
0.4 AE
BE
AD
AC
CD
0 C:agitation
0 5 10 15 20 25
Standardized effects
This plot has the advantage that all signals fall to the right of the noise line. The above plot
confirms the conclusion that 5 significant effects are present.
Analysis Options
The mathematical model currently being used to fit the data contains 5 main effects and 10 two-
factor interactions. As seen above, many of these terms are not statistically significant. When
building empirical models based solely on observed data, it is important to keep the models as
simple as possible, since simple models tend to be easier to interpret and have a better chance of
extrapolating to other combinations of the experimental factors.
2. Remove any insignificant main effects that are not involved in significant interactions.
Note that main effects corresponding to factors that are involved in significant interactions
should not normally be removed, since doing so would place artificial constraints on the
underlying polynomial models.
Maximum Order Effect - the maximum order effect to be included in the model. Set to 2
by default to request fitting of both main effects and 2-factor interactions. If set to 1, only
main effects will be estimated.
Ignore Block Numbers - for designs that contain more than 1 block, indicates whether block
effects should be estimated or ignored. Note that column 1 of the datasheet for any
experiment file contains block numbers corresponding to each row.
Estimate Sigma From - whether the standard deviation of the experimental error is to be
estimated from the experimental error, or whether the analyst will provide a known value. If
a known value is provided, the statistical tests and confidence intervals will use that value.
Display - affects the output displayed for each effect after the on the Analysis Summary.
Confounding Pattern - specifies how the procedure determines which effects to estimate
when fitting the model. The choices are:
1. From Original Design - examines the confounding pattern of the original design to
determine which effects can be estimated. For example, in a resolution IV design, the
program will estimate specific combinations of the two-factor interactions. This is the
default choice and is appropriate in all but very special circumstances.
2. From Data - examines the X matrix of all runs performed to determine which effects can
be estimated. This may be desirable when the analyst has added additional runs to the
base design to clear certain interactions that would otherwise be confounded. Since the
program attempts to estimate all effects when this option is chosen, the Exclude dialog
box may have to be used to tell the program exactly which effects are to be estimated.
Effects can be excluded from the model by double-clicking on them one at a time. Double-
clicking on an effect in either of the two columns moves it to the other column.
In the current example, all effects other than the 5 that appeared to be statistically significant
were removed. The standardized Pareto chart for the new model shows the remaining effects:
B:catalyst +
-
D:temperature
BD
DE
E:concentration
0 4 8 12 16
Standardized effect
78
74
70
reacted
66
62
58
54
1.0 2.0 140.0 180.0 3.0 6.0
It shows how the predicted response Y varies when each of the factors in the model is changed
from its low level to its high level, with all other factors held at the center of the experimental
2009 by StatPoint Technologies, Inc. DOE Wizard – Screening Designs - 30
STATGRAPHICS – Rev. 7/16/2009
region (halfway between the low level and the high level). When all of the factors are plotted
together as in the above plot, it is easy to judge which factors have the greatest impact. When
plotted individually, the predicted response at the extremes of a selected factor is shown:
78
75.4605
74
70
reacted
66
62
58
54.9605
54
1.0 2.0
catalyst
Note: In some cases, the values shown at the endpoints of the line in the above plot will be equal
to the average response at the low and high level of the plotted factor. That is not the case in
general, however. It is important to note that STATGRAPHICS plots the predicted response
from the current model, not the observed data. This allows the plot to be used with many types
of designs other than a two-level factorial.
Pane Options
+
84
-
reacted
74
64 - +
+
-
+-
54
1 2 140 180
BD DE
A pair of lines will be plotted for each interaction, corresponding to the predicted response when
one factor is varied from its low value to its high value, at each level of the other factor. All
factors not involved in the interaction are held at their central value.
The plot is usually easier to understand if Pane Options if used to plot each interaction
separately:
94
temperature=180.0
84
reacted
74
64 temperature=140.0
temperature=180.0
temperature=140.0
54
1.0 2.0
catalyst
The predicted response for each combination of the low and high levels of two factors is
displayed at the end of each line segment. If two factors do not interact, the effect of one factor
will not depend upon the level of the other and the two lines in the interaction plot will be
Pane Options
Factors: two or more factors to include on the plot. All interactions for which both factors
have been checked will be included.
Reverse Factors: If checked, the first factor rather than the second will be used to define the
lines on the plot.
Regression Coefficients
The underlying regression model may be displayed by selecting the Regression Coefficients
pane:
Interpretation
This pane displays the regression equation which has been fitted to the data. The equation of the fitted model is
where the values of the variables are specified in their original units. To have STATGRAPHICS evaluate this function, select
Predictions from the list of Tabular Options. To plot the function, select Response Plots from the list of Graphical Options.
Note: In the regression equation, all factors that were defined as continuous when the experiment
was initially created are expressed in their original units (e.g., temperature is expressed in
degrees C). Factors that were not defined as continuous use the coding of -1 for the low level
and +1 for the high level.
Correlation Matrix
The correlation matrix displays the estimated correlation between the coefficients in the fitted
regression model:
The correlations are estimated from the variance-covariance matrix of the coefficients, given by
s 2 (b) MSE X X
1
(18)
A diagonal matrix such as that shown above indicates that the estimates of each of the effects are
uncorrelated with the other estimates, which stems from the orthogonality of the original design.
If any data values were missing (indicated by leaving the corresponding cells of the response
column in the spreadsheet empty), or additional runs have been added by the user, there may
well be non-zero values for the off-diagonal terms. Large correlations are likely to lead to poorly
defined effects that are difficult to interpret.
Response Plots
The list of graphs available in the Analyze Data procedure contains two selections labeled
Response Plots that allow the predicted values of the response to be plotted in various ways. By
default, the first selection creates a response surface plot and the second a contour plot.
However, each is controlled by the same Pane Options dialog box, which allows the analyst to
display any of four types of plots:
1. a surface plot.
2. a contour plot.
3. a square plot.
4. a cube plot.
The surface plot displays a plot of the predicted response as a function of any two of the
experimental factors, with the other factors held at selected values. For example, the plot below
shows reacted as a function of catalyst and temperature:
2009 by StatPoint Technologies, Inc. DOE Wizard – Screening Designs - 34
STATGRAPHICS – Rev. 7/16/2009
94
84
reacted
74
64
180
170
54 160
1 1.2 150
1.4 1.6 1.8 140 temperature
2
catalyst
The height of the surface represents the predicted value Y , which is plotted over the range of the
experimental factors.
Contour plots draw lines or colored regions based on values of the predicted response. For
example, the plot below displays the range of the predicted values for reacted using colors
extending from blue at 50 to red at 85:
65.0
160 70.0
75.0
80.0
150 85.0
140
1 1.2 1.4 1.6 1.8 2
catalyst
The color ramp used is controlled by the Palette tab on the Graphics Options dialog box.
The Square Plot shows the predicted response at combinations of the low and high levels for any
2 factors:
54.2105 63.9605
140.0
1.0 2.0
catalyst
The Cube Plot shows the predicted response at combinations of the low and high levels for any 3
factors:
47.8355
79.0855
6.0 55.8355
65.5855
concentration
63.5855
94.8355180.0
3.0 52.5855
1.0 62.3355
2.0140.0 temperature
catalyst
In the example, the highest predicted value for reacted is obtained at catalyst = 2, temperature =
180, and concentration = 3. The strong interactions amongst the factors result in a predicted
value of nearly 95 at that combination of the factors.
From: location at which the first contour line is drawn, or the start of the first region.
To: location at which the last contour line is drawn, or the end of the last region.
Lines: if selected, a sequence of contour lines is drawn at selected levels of the predicted
response, as on a topographical map.
Painted Regions: if selected, a set of regions is drawn covering various ranges of the
predicted response.
Resolution: defines the resolution m of an m-by-m grid of predicted values which is used to
draw the surface and contour lines. Increasing the resolution may improve the smoothness
and definition of the plots, at the expense of computer time and memory.
Horizontal Divisions: the number of divisions along the first experimental axis. This
determines how many vertical lines will be drawn on the surface plot.
Vertical Divisions: the number of divisions along the second experimental axis. This
determines how many horizontal lines will be drawn on the surface plot.
Show Points: requests that the observed data values Yi be added to the plot, with vertical
lines drawn from each point to the surface.
Wire Frame: requests that the surface be drawn using cross-hatched lines as shown in the
figure above. This is the most effective choice for black-and-white presentation.
Contoured: requests that the surface be drawn showing contour levels of the response.
Factors: specifies the factors to be plotted on each axis and the levels at which the other
factors will be held:
If creating a surface, contour, or square plot, two factors must be checked. If creating a cube plot,
three factors must be checked.
The current example plots predicted values versus catalyst and temperature, when feed rate =
12.5, agitation = 110, and concentration = 4.5.
Predictions
The Predictions pane may be used to generate predictions from the fitted model:
The table may include all rows in the datasheet, or only rows for which the value of the response
variable Y has not been entered. The latter feature allows the analyst to make predictions at
combinations of X that were not included in the experiment. For example, the above table shows
the result of adding a 20th row with feed rate = 12.5, catalyst = 1.2, agitation = 105, temperature
= 165, and concentration = 3.5. The predicted value of reacted is 62.66. The 95% confidence
interval for the mean value of reacted at that same combination of the factors ranges from 60.9 to
64.6.
The table also displays the average of the experimental runs performed at the center of the
experimental region, together with the predicted response. If the assumed model is correct, the
two values should be close. If not, there may be unmodeled curvature with respect to one or
more of the experimental factors. Determining the nature of that curvature would require
performing additional runs at different levels of the factors. To add additional runs to a screening
experiment, you can use the Augment Design selection on the DOE menu.
One other noticeable entry in the above table is the Studentized residual for row #11. The
Studentized residual measures the difference between the observed response and the predicted
response, in units of its standard error, when the observation in question is not used to fit the
model. The Studentized residual for observation #11 equals 4.1. Values in excess of 3.0 are
unusual and would typically require further scrutiny. If the point in question gave a desirable
result (which it does not), a rerun of that set of experimental conditions might be necessary.
2. Fitted Y - the predicted values Yi calculated from the fitted model.
d i Yi Y( i ) (19)
di
ei* (20)
s( d i )
where
s 2 (d i ) MSE (i ) 1 X i ( X (i ) X ( i ) ) 1 X i (21)
5. Standard Errors for Forecasts - the standard error for new observations at a selected
combination of the experimental factors Xh, given by
MSE 1 X h ( X X ) 1 X h (22)
6. Confidence Limits for Individual Forecasts - confidence limits for new observations at a
selected combination of the experimental factors Xh, given by
Yh t n p MSE 1 X h ( X X ) 1 X h (23)
7. Confidence Limits for Forecast Means - confidence limits for the mean response at a
selected combination of the experimental factors Xh, given by
Yh t n p MSE X h ( X X ) 1 X h (24)
Predict - whether forecasts are displayed for all of the runs in the experiment data file, or
only for runs that have a missing value in the response column.
Diagnostic Plots
Several plots are also provided under Diagnostic Plots to examine the residuals from the fitted
model. The Pane Options dialog box displays the various choices, which include the following:
Plot of reacted
95
85
observed
75
65
55
45
45 55 65 75 85 95
predicted
0
-2
-4
-6
45 55 65 75 85 95
predicted
The residuals should vary randomly around the line. Changes in the magnitude of the residuals
from left to right may signal that the variance of the experimental error varies with the mean
level of the response. Such heteroscedasticity may frequently be eliminated by a variance-
stabilizing transformation such as a logarithm or a square root.
0
-2
-4
-6
0 4 8 12 16 20
run number
Any non-random pattern may indicate a time trend or other effect. In such cases, addition of a
factor to account for the change may improve the fit of the model. The above plot does suggest
an increase in variability during the second half of the experiment, which would be worthy of
further investigation.
0
-2
-4
-6
1 1.2 1.4 1.6 1.8 2
catalyst
Any curvature around the line may suggest the need for a model with quadratic effects. The
above plot suggests that the variability amongst the replicated values at the centerpoint may be
somewhat less than that of the residuals at the low and high levels of catalyst.
80
50
20
5
1
0.1
-3.6 -1.6 0.4 2.4 4.4 6.4
residuals
If the experimental error follows a normal distribution, the points should lie along a straight line.
The above plot suggests that the largest residual (row #11) is somewhat higher than expected,
since it lies off the line defined by the others. This could indicate the presence of an outlier or
significant curvature.
0.8
0.6
0.4
0.2
0
-8 -4 0 4 8
true effect
This plot is similar to the power curve explained in detail earlier when the design was being
constructed, except that the horizontal axis is displayed in units of the response instead of a
signal-to-noise ratio. The above plot shows that the current experiment has an excellent chance
of detecting any effects for which the change in reacted is 5 or more.
Pane Options
Plot versus: selects the experimental factor to be shown in the plot, for those plots where a
factor is needed.
Fitted Line: specifies whether a line should be fit to the data on the normal probability plot.
Once a statistical model has been developed for each response, the analyst may now determine
what combination of factors will yield the best results. Pressing the button labeled Step #9 on the
Experimental Design Wizard toolbar first displays the dialog box shown below:
Since optimization requires searching for the best conditions throughout the experimental region,
it is a good idea to begin that search at many different points in order to avoid finding only a
local optimum.
When the optimization is complete, a message similar to that shown below will be displayed:
The dialog box indicates the “Desirability” of the final result, based on a metric designed to
balance competing requirements of multiple responses (see the document titled DOE Wizard for
full details). The value displayed in this case indicates that the predicted reactivity at the
optimum factor settings is 74.18% of the distance between 80 and 100, which was the desired
range specified when the design was created.
If you press OK, additional information will be added to the main DOE Wizard window:
The table shows the estimated response at the optimal settings of the experimental factors. For
the chemical reaction data, it is estimated that the mean percent reacted will equal 94.84% when
the factors are set at feed rate = 10, catalyst = 2, agitation = 119.9, temperature = 180, and
concentration = 3. The 95% confidence interval for the mean ranges between 91.63% and
98.04%.
NOTE: Since feed rate and agitation have been completely eliminated from the statistical model,
the solution displayed is only one of many. Any setting of feed rate and agitation would give the
same predicted response.
If you push the Tables and Graphs button on the analysis toolbar, you can display the estimated
desirability throughout the experimental region. An interesting type of display is the 3-D Contour
plot shown below (use Pane Options and the Factors button to select the factors to plot on each
axis):
Desirability Plot
feed rate=10.0,agitation=120.0,concentration=6.0
Desirability
0.0
0.1
6 0.2
5.5 0.3
concentration
5 0.4
0.5
4.5
0.6
4 0.7
3.5 0.8
180
3 170 0.9
160
1 1.2 1.4 150 1.0
1.6 1.8 2 140 temperature
catalyst
It is clear that the best place to operate is in the lower right back corner.
The button labeled Step 10 allows you to save the results in a StatFolio:
Actually, the StatFolio can be saved at any point and reloaded at a later date.
IMPORTANT: When using the Experimental Design Wizard, two files are created:
1. An experiment file with the extension .sgd which stores information about the
experimental data.
2. A StatFolio with the extension .sgp that stores the results of the analysis.
If you move the experiment to another computer, be sure to transfer both files.
Since the conclusions from the design are fairly clear, there is no need to augment the design.
Extrapolation
The maximum predicted reactivity within the design space is 94.84%. To use the statistical
model to predict settings of the factors outside the experimental region that might produce even
better results, press the button labeled Step 12. The following dialog box will be displayed:
Change: the factors you wish to consider changing. Since feed rate and agitation have been
completely eliminated from the model, they have been unchecked.
Display steps of: The program will begin at the starting location and follow the path of
steepest ascent in an attempt to increase the desirability of the predicted response. Specify the
increment of increased desirability at which the results should be displayed.
In this case, we have asked to program to search from the derived optimal conditions and display
improvement of 1% in desirability.
The results of the search are shown in the following table, which will be added to the main DOE
Wizard window:
The program suggests that the best course of action would be to increase catalyst and
temperature while decreasing concentration.