CAE-Based Robust Design:

Basic Concepts
Analysis of Variance (ANOVA)

06/15/15

Robust Synthesis & Analysis

Basic Concepts - ANOVA - 1

ANOVA
What is ANOVA?
ANOVA stands for ANalysis Of VAriance
It is a way to decompose a complicated function into a sum of simpler
functions in order to facilitate understanding the complicated function

What can you learn from ANOVA?

Find most important variables

Determine if you can desensitize design to noise
Visualize nature of input-output relationships
Learn how coupled the simulation is, that is, if the effect of one
variable depends on the settings of others

06/15/15

Robust Synthesis &

Basic Concepts

Familiar functional decompositions

Taylor series
Gives easy to
manipulate
approximations for use
in other calculations
(e.g., determine step
direction in
optimization)

di

f ( x1 , x 2 ,..., x n ) f x

L
f di
x O
P
dx x i
M
x P
M
N Q

L
f di
x O
P
x x id
x x i
d
M
M
Nx x P
Q

i 1

i 1 j 1

Fourier series
Expresses an arbitrary
periodic function as sum of
sines and cosines. System
response to a complicated
input can be found by
adding response to simple
sine and cosine inputs.

f ( x ) a0 a n cos(nx ) bn sin( nx )

06/15/15

Robust Synthesis &

1
2

n 1

n 1

Basic Concepts

ANOVA functional decomposition

n

i 1

f ( x1 , x 2 ,..., x n ) g i ( xi ) hij ( xi , x j )
i 1

i 2 j 1

A function of n variables is decomposed into the sum of

a constant
functions of one variable
functions of two variables
functions of three variables

(overall mean)
(main effects)
(two-way interactions)
(three-way interactions)

Interpreting the results

Main effects provide way to visualize effect of individual variables on output
Can assess importance of a main effect or two-way interaction by computing how
much the squared error in the approximation goes down when add that term
If no interactions are important, can optimize one variable at a time and essentially
read the best design from the main effect plots
If a two-way interaction exists between a control and noise variable, you may be
able to adjust that control to desensitize the response to the noise factor.

06/15/15

Robust Synthesis &

Basic Concepts

A specific example
Suppose we have a simulation with two inputs x1 and x2 that vary over [0,1].
We run the simulation on a 5x5 grid, and obtain these results

X1

Function Evaluated Over a 5x5 Grid

X2
0.00
0.25
0.50
0.75
0.00
0.00
-1.41
0.00
1.41
0.25
2.81
1.53
3.06
4.59
0.50
3.75
2.59
4.25
5.91
0.75
2.81
1.78
3.56
5.34
1.00
0.00
-0.91
1.00
2.91

1.00
0.00
3.31
4.75
4.31
2.00

In addition, we fit a response surface to this data that gives us a function

f(x1,x2) that predicts the value of the simulation at any point (x1,x2).

06/15/15

Robust Synthesis &

Basic Concepts

Quantifying the total variation in f

X1

Function Evaluated Over a 5x5 Grid

X2
0.00
0.25
0.50
0.75
0.00
0.00
-1.41
0.00
1.41
0.25
2.81
1.53
3.06
4.59
0.50
3.75
2.59
4.25
5.91
0.75
2.81
1.78
3.56
5.34
1.00
0.00
-0.91
1.00
2.91

1.00
0.00
3.31 Average:
4.75 2.38
4.31
2.00

Y - Mean

X1

0.00
0.25
0.50
0.75
1.00

0.00
-2.38
0.44
1.38
0.44
-2.38

0.25
-3.78
-0.84
0.22
-0.59
-3.28

X2
0.50
-2.38
0.69
1.88
1.19
-1.38

0.75
-0.97
2.22
3.53
2.97
0.53

1.00
-2.38
0.94
2.38 Variation
1.94 from
-0.38 Mean

X2
0.50
5.64
0.47
3.52
1.41
1.89

0.75
0.94
4.92
12.47
8.81
0.28

1.00
5.64
0.88
5.64
3.75
0.14

(Y - Mean) ^ 2

X1

0.00
0.25
0.50
0.75
1.00

06/15/15

0.00
5.64
0.19
1.89
0.19
5.64

0.25
14.30
0.71
0.05
0.35
10.77

The constant in
the ANOVA
decomposition is
just the average
of all the values,
here 2.38.
Using only the
constant (mean)
results in a
squared
approximation
error of 96.14

Total
Squared
Deviations
From Mean
96.14

Robust Synthesis &

Basic Concepts

The main effect of x1

Y - Mean

0.00
0.25
0.50
0.75
1.00

X1

0.00
-2.38
0.44
1.38
0.44
-2.38

0.25
-3.78
-0.84
0.22
-0.59
-3.28

X2
0.50
-2.38
0.69
1.88
1.19
-1.38

0.75
-0.97
2.22
3.53
2.97
0.53

1.00
-2.38
0.94
2.38
1.94
-0.38

Main Effect of x1

2.5
2
1.5
1
0.5
0
-0.5 0
-1

0.25

0.5

0.75

-1.5
-2
-2.5
-3

06/15/15

x1

Robust Synthesis &

Average
(Main
Effect)
-2.38
0.69
1.88
1.19
-1.38

The main effect

of x1 is the
function that
best explains
(Y-Mean)
using only x1.
We try to explain
(Y-Mean) and not
just Y because
we higher-order
terms are only
used to explain
what is left over
from lower-order
terms.

Basic Concepts

Variation explained by x1

X1

X1

X1

Function Evaluated Over a 5x5 Grid

X2
0.00
0.25
0.50
0.75
0.00
0.00
-1.41
0.00
1.41
0.25
2.81
1.53
3.06
4.59
0.50
3.75
2.59
4.25
5.91
0.75
2.81
1.78
3.56
5.34
1.00
0.00
-0.91
1.00
2.91
Y - [ Mean + MainEffect(x1) ]
X2
0.00
0.25
0.50
0.00
-1.41
0.00
-0.25
-1.53
0.00
-0.50
-1.66
0.00
-0.75
-1.78
0.00
-1.00
-1.91
0.00

0.75
1.41
1.53
1.66
1.78
1.91

1.00
0.00
0.25
0.50
0.75
1.00

{ Y - [ Mean + MainEffect(x1) ] } ^ 2
X2
0.00
0.25
0.50
0.75
0.00
1.98
0.00
1.98
0.06
2.34
0.00
2.34
0.25
2.74
0.00
2.74
0.56
3.17
0.00
3.17
1.00
3.63
0.00
3.63

1.00
0.00
0.06
0.25
0.56
1.00

0.00
0.25
0.50
0.75
1.00

0.00
0.25
0.50
0.75
1.00

1.00
0.00
3.31 Average:
4.75 2.38
4.31
2.00

06/15/15

Variation
from
Mean
Plus
MainEffect(x1)

Total
Squared
Deviations
31.49

Robust Synthesis &

With only the

constant, the total
squared error was
96.14.
But when we take
into account the
main effect of x1,
the variation falls to
31.49.
The difference,
64.65, is due to x1.
The percent
contribution of x1 is
therefore
64.65 / 96.14 = 67%

Basic Concepts

The main effect of x2

Y - Mean

0.00
0.25
X1
0.50
0.75
1.00
Average = Main Effect

0.00
-2.38
0.44
1.38
0.44
-2.38
-0.50

0.25
-3.78
-0.84
0.22
-0.59
-3.28
-1.66

X2
0.50
-2.38
0.69
1.88
1.19
-1.38
0.00

0.75
-0.97
2.22
3.53
2.97
0.53
1.66

1.00
-2.38
0.94
2.38
1.94
-0.38
0.50

The main effect

of x2 is the
function that
best explains
(Y-Mean)
using only x2.

Main Effect of x2

1.5
1
0.5
0
-0.5

0.25

0.5

0.75

-1
-1.5
-2

06/15/15

x2

Robust Synthesis &

Basic Concepts

Variation due to x2

X1

X1

X1

Function Evaluated Over a 5x5 Grid

X2
0.00
0.25
0.50
0.75
0.00
0.00
-1.41
0.00
1.41
0.25
2.81
1.53
3.06
4.59
0.50
3.75
2.59
4.25
5.91
0.75
2.81
1.78
3.56
5.34
1.00
0.00
-0.91
1.00
2.91

0.00
0.25
0.50
0.75
1.00

0.00
0.25
0.50
0.75
1.00

Y - [ Mean + MainEffect(x2) ]
X2
0.00
0.25
0.50
-1.88
-2.13
-2.38
0.94
0.81
0.69
1.88
1.88
1.88
0.94
1.06
1.19
-1.88
-1.63
-1.38

1.00
0.00
3.31 Average:
4.75 2.38
4.31
2.00

0.75
-2.63
0.56
1.88
1.31
-1.13

1.00
-2.88
0.44
1.88
1.44
-0.88

{ Y - [ Mean + MainEffect(x2) ] } ^ 2
X2
0.00
0.25
0.50
0.75
3.52
4.52
5.64
6.89
0.88
0.66
0.47
0.32
3.52
3.52
3.52
3.52
0.88
1.13
1.41
1.72
3.52
2.64
1.89
1.27

1.00
8.27
0.19
3.52
2.07
0.77

06/15/15

Variation
from
Mean
Plus
MainEffect(x2)

Total
Squared
Deviations
66.21

Robust Synthesis &

With only the

constant, the total
squared error was
96.14.
But when we take
into account the
main effect of x2,
the variation falls to
66.21.
The difference,
29.93, is due to x2.
The percent
contribution of x2 is
therefore
29.93 / 96.14 = 31%

Basic Concepts

Interaction effects
The function doesnt
have to be completely
explained by the main
effects.
In this case, there is a
residual after the mean
and main effects are
considered. This is due
to effects related to
specific combinations of
(x1,x2), the so-called
interaction effect.
The percent contribution
of the interaction is
1.56 / 96.14 = 2 %

06/15/15

Robust Synthesis &

Basic Concepts

Is the formula helpful?

f ( x1 , x2 ) = 15x1 (1 - x1 ) + 30 x2 ( x2 - 0.5)(1 - x2 ) + 2 x1 x2
Here is the formula for the function used in the example.

If I had showed you this formula, would you have been able
to tell me the relative importance of the variables?

How would your answer to the last question change if I

showed you a function of 50 variables?

06/15/15

Robust Synthesis &

Basic Concepts

Grid-based versus functional ANOVA

If we want to visualize the input-output relationship over a continuous
interval (not just at the end points), we need a fine grid
But using a fine grid is not practical with many (> 10) variables, since
requires too many simulations
Surprisingly,using an infinite grid is practical if we do the calculations on the
response surface as opposed to the simulation. Why?
With infinite grid, all the averages in the calculations change to integrals
Polynomial and kriging response surfaces have a form which makes it easy to
compute these integrals

Doing ANOVA with an infinite grid is called functional ANOVA.

As long as the response surface fits well, the functional ANOVA of the
surface will be as informative, if not more so, than a grid-based ANOVA on
a coarse grid.

06/15/15

Robust Synthesis &

Basic Concepts

ANOVA with different grid sizes

Grid-Based ANOVA
Grid
fineness
2
3
4
5
6
7
8
9
10
20
30
40
50
60
70
80
90
100

Number
points
4
9
16
25
36
49
64
81
100
400
900
1,600
2,500
3,600
4,900
6,400
8,100
10,000

Mean
0.50
1.75
2.17
2.38
2.50
2.58
2.64
2.69
2.72
2.87
2.91
2.94
2.95
2.96
2.96
2.97
2.97
2.97

Total Variance
0.75
3.57
3.94
3.85
3.73
3.63
3.55
3.49
3.44
3.22
3.15
3.12
3.10
3.08
3.07
3.06
3.06
3.06

Percent Due Percent Due

to x1
to x2
33.33%
33.33%
92.22%
4.67%
74.12%
23.92%
67.24%
31.13%
63.22%
35.32%
60.47%
38.17%
58.43%
40.27%
56.86%
41.90%
55.60%
43.19%
49.93%
49.02%
48.03%
50.97%
47.07%
51.94%
46.50%
52.53%
46.12%
52.92%
45.85%
53.20%
45.64%
53.40%
45.48%
53.57%
45.36%
53.70%

The percent contributions

change with the grid size.
As the grid size approaches
infinity, we get the
functional ANOVA. For our
simple example, these
integrals can actually be
computed on the real
simulation function.
Doing the functional ANOVA
on a fitted kriging surface
gives essentially the same
results using only 41 points.

Functional ANOVA
Infinite !

All points !

3.00

3.02

44.20%

54.90%

Kriging-Based Functional ANOVA using GASP

Latin HC

41

06/15/15

3.01

not reported

42.26%

55.64%

Robust Synthesis &

Basic Concepts

Main effects in functional ANOVA

Main Effect of x1

ANOVA with 5 x 5 grid

2.5
2
1.5
1
0.5
0
-0.5 0
-1
-1.5
-2
-2.5
-3

0.25

0.5

0.75

Functional ANOVA

Main Effect of x2

x1

2.00000E+00
1.50000E+00
1.00000E+00
5.00000E-01
0.00000E+00
-5.00000E-01
-1.00000E+00
-1.50000E+00
-2.00000E+00
-2.50000E+00
-3.00000E+00
-3.50000E+00
0.00

2.00000E+00

1.5

1.50000E+00

1.00000E+00

0.5

5.00000E-01

0
-0.5 0

0.20

0.40

0.60

0.20

0.40

0.60

0.80

1.00

0.00000E+00

0.25

0.5

0.75

-5.00000E-01

-1

-1.00000E+00

-1.5

-1.50000E+00

-2
x2

-2.00000E+00
0.00

0.80

1.00

Recalling form of function (see right),

f ( x1 , x2 ) = 15x1 (1 - x1 ) + 30 x2 ( x2 - 0.5)(1 - x2 ) + 2 x1 x2
do these main effects make sense?

06/15/15

Robust Synthesis &

Basic Concepts

Graphical interpretation of % contribs

5

Here we plot Y-Mean versus

X1 for all the 25 points in
the example.

4
3

The total variation is the

sum of squared distances
from the blue dots to the
dotted line (Y-mean = 0).

2
1

Y - Mean 0
-1

The variation explained by

x1 is the sum of squared
distances from the main
effect to the dotted line.

-2
-3
-4
-5

0.25

0.5

0.75

X1

06/15/15

Robust Synthesis &

The variation not explained

by x1 is the sum of squared
distances of the blue dots to
the main effect (red line).

Basic Concepts

Effect of variable ranges

5
4

Consider what would

happen if we reduce the
range of X1 to (say)
[0.25,0.5]

3
2
1

Y - Mean 0
-1
-2
-3

-5

The effect of x1 will usually

seem smaller relative to that
of the other variables, and
its percent contribution will
go down.

-4
0

0.25

0.5

X1

0.75

2
1

In short, the importance of a

variable is a function both of
the sensitivity (local slope)
and the range over which it
varies.

Y - Mean 0
-1
-2
-3

06/15/15

0.25

X1

0.50

Robust Synthesis &

Basic Concepts

Available ANOVA software

ANOVA on grids or fractional factorial designs
DEXPERT
Provides fractional-factorial or full-factorial experimental designs
Runs in Windows with GUI, available from Knowledge Center

Minitab
Runs in Windows with GUI, available from Knowledge Center

Functional ANOVA
Kriging Wizard

Provides space-filling experimental designs

Fits kriging surfaces
Performs exact functional ANOVA on surface. Creates plots.
MS Excel Add-In with GUI, available from RSA website

Robust Design Wizard

Fits up to third-order polynomial surfaces
Performs exact functional ANOVA on polynomial surface. Creates plots.
UNIX executable with GUI, available from RSA website

06/15/15

Robust Synthesis &

Basic Concepts

Test your understanding

What does ANOVA stand for?
What questions does ANOVA help answer?
How does one quantify the total variation of a function?
How does one quantify the variation due to a variable?
What is the percent contribution of a variable?
What is a main effect plot?
What is an interaction effect?
How does functional ANOVA differ from grid-based ANOVA?
How do percent contributions depend on variable bounds?

06/15/15

Robust Synthesis &

Basic Concepts