Webinar September 19, 2007 p

20

Stat-Ease, Inc.

09

Shari Kraber Sh i K b

St at -E

Factorial D i F t i l Design Planning Process g

Pat Whit P t Whitcomb b
Stat-Ease, Inc. pat@statease.com
1

shari@statease.com

DOE Process

as

e,

In c.

Factorial Design Planning Process g g

Our t lk has three parts: O talk h th t

2. Illustrate key points via an example 3. Summary

DOE Process

20

09

St at -E

1. Broad brush description of th DOE 1 B db hd i ti f the planning process

as

e,

In c.
2

Design of Experiments g p
DOE (Design of Experiments) is:

Controllable Factors x

Process

St at -E

Noise Factors z

DOE Process

20

09

as

Responses y
so that you may identify causes for significant changes p p in the output responses.

e,

A systematic series of tests, A in which purposeful changes are made to input factors,

In c.

Iterative Experimentation p
Conjecture

Analysis

St at -E
Experiment

as
Design
DOE Process

Expend no more than 25% of budget on the 1st cycle cycle.

4

20

09

e,

In c.

Factorial Design Planning Process (1 of 2)

1. Identify opportunity and define objective.

2. State objective in terms of measurable responses.

b. b Estimate experimental error () for each response.

DOE Process

3. 3 Select the input factors to study (Remember that study. the factor levels chosen determine the size of y.)

20

09

c. c Use the signal to noise ratio (y/) to estimate power.

St at -E

a. a Define the change (y) that is important to detect for each response.

as

e,

In c.
5

Factorial Design Planning Process (2 of 2)

4. Select a design and:

Evaluate power (desire power > 80% for effects of interest); generally use main effects (ME) model ( o robust des g use o y 1 ME). ode (for obus design only )

DOE Process

20

Examine the design layout to ensure all the factor combinations are safe to run and are likely to result in meaningful information (no disasters).
6

09

St at -E

Evaluate aliases (fractional factorials and/or blocked designs); generally use two-factor two factor interaction (2FI) model.

as

e,

In c.

Factorial Design Planning Process

Tools: Brainstorming (fishbone) Consensus Outputs Voting Form Outputs, y, %Contribution O t t , %C t ib ti

Identify Opportunity Define Objective

Factors Voting Form DOE inputs, levels, operating range Other inputs Select S l t an appropriate factorial d i i t f t i l design Evaluate aliases (fractional factorials and/or blocked designs) Evaluate power Examine the design layout

St at -E

as

e,

20

09

DOE Process

In c.

State Objective in Terms of Measurable Responses Select Input S l tI t Factors

Select Design Evaluate Aliases & Power Examine E i Layout

Factorial Design Planning Process g g

Our t lk has three parts: O talk h th t

2. Illustrate key points via an example 3. Summary

DOE Process

20

09

St at -E

1. Broad brush description of th DOE 1 B db hd i ti f the planning process

as

e,

In c.
8

Stent Delivery System

A stent is a wire mesh tube used to prop open an artery p p p y that's recently been cleared using angioplasty. The stent is collapsed to a small diameter over a balloon catheter. It's then It' th moved i t the area of the blockage. d into th f th bl k

DOE Process

20

When the balloon is inflated the stent expands locks in inflated, expands, place and forms a scaffold. This holds the artery open. The stent stays in the artery permanently, holding it open y yp y, g p to improve blood flow to the heart muscle.

09

St at -E

as

e,

In c.

Stent Delivery System

1. Identify opportunity and define objective. Relate stent deliverability and safety to process factors.

DOE Process

Scheduling: Two meetings no more than 2 days apart. First day spend approximately 1 to 2 hours discussing Goal Objective Outputs and Inputs Goal, Objective, Inputs. Second day spend approximately 2 to 4 hours developing DOE.

20

09

St at -E

Team Make Up: Experts p p p Semi-Experts or Peripheral Experts Technicians or Operators Customers

as

Guidelines for Brainstorming a Designed Experiment g g p

e,

In c.

10

Stent Delivery System

2. State objective in terms of measurable responses. j p Deliverability is quantified by Trackability and Pushability; safety is quantified by Burst pressure. Want to estimate 2FI model, this requires a resolution V (or higher) design model design.

b. Estimate experimental error () for each response.

DOE Process

20

09

c. Use the signal to noise ratio (y/) to estimate power.

St at -E

a. Define the change (y) that is important to detect for each response response.

as

e,

In c.

11

Outputs Voting Form

Evaluation of Outputs Voting Form p g

Response Output (y) Type of Response (V) Variable (D) Destructive (A) Attribute Team Member's Ranking of Importance of the Responses

St at -E
M1 M2 M3 M4 M5

as
M6 M7 M8 Average Standard Deviation

1: low rank; 5: high rank

DOE Process

20

09

e,
12

In c.

Objective as Measurable Responses

Stent Delivery System

2. 2 Typical of responses in actual DOE:
Response Y1: Burst Y2: Push Y3: Track

g/cm

St at -E

psig

Maximize Minimize

as

Unit of Specification Practical Measure or Target Difference 6 15 10 Maximize

e,

In c.
Std Dev 8 30 6 %MC* 27% 75% 19%
13

g*cm

* % measurement contribution

DOE Process

20

09

Stent Delivery System

3. Select the input factors to study. (Remember that the factor levels chosen determine the size of y.) Lengths and diameters of various components, e.g. tip, balloon, catheter, etc. Materials used for the components. Assembly parameters e g weld locations how the parameters, e.g. locations, balloon is folded, etc.

DOE Process

20

Stent geometry, wall thickness, how it is crimped on the balloon, etc.

09

St at -E

as

yp Typical factors include:

e,

In c.

14

Factors Voting Form

Evaluation of Factors Voting Form
Factor Name Input (x) Type of Factor (C) Control (N) Noise Team Member's Ranking of Importance of the Factors

1: low rank; 5: high rank

St at -E
M1 M2 M3 M4 M5

as
M6 M7 M8 Average Standard Deviation

DOE Process

20

09

e,
15

In c.

Select the input factors

Factors
Factor Unit of Measure F M V B Target

DOE Process

F: Fixed factors that do not change for the duration of the experiment, e.g. One batch of raw material, one operator, etc. M: Monitor factors allowed t vary d i th course of the experiment and will be M M it f t ll d to during the f th i t d ill b monitored, e.g. temperature, humidity, etc. V: Vary factors allowed to vary, but will not be monitored, e.g. temperature, humidity, etc. p , g y, , B: Block factors used to block experimental runs, e.g. time of day, machine, etc.

20

09

St at -E

as

e,
16

In c.
Range

Other Inputs
Comments

Stent Delivery System

3. Select the input factors to study.
Factor A B C D E Type numeric numeric numeric numeric Low Level () 1 1 1 1

St at -E
1 1 1 1 1 L1

numeric 1 +1 There were 11 factors: 10 numeric and 1 categoric; F numeric 1 +1 the actual factor details are proprietary. numeric i numeric numeric +1 1 +1 +1 +1 L2
17

G H J K L

DOE Process

20

categoric

09

numeric

as
+1 +1 +1

e,
+1

High Level (+)

In c.
Operating Range

Stent Delivery System

4. Select a design (want resolution V):

DOE Process

20

Examine the design layout to ensure all the factor E i th d i l tt ll th f t combinations are safe to run and are likely to result in meaningful information (no disasters)

09

St at -E

Evaluate power (desire power > 80% for effects of interest) Evaluation; Order: Main effects

as

Evaluate aliases (fractional factorials and/or blocked designs)

e,

In c.
18

Stent Delivery System

Candidate resolution V designs:

An i A irregular algebraic f ti requires 96 runs. l l b i fraction i A MR5* fraction requires 68 runs. Add center points to check for curvature.

DOE Process

* Small, Efficient, Equireplicated Resolution V Fractions of 2k designs and their Application to Central Composite Designs, Gary Oehlert and Pat Whitcomb, 46th Annual Fall Technical Conference, Friday, October 18, 2002. A PDF copy of this paper is available at: www statease com/pubs/small5 pdf www.statease.com/pubs/small5.pdf 19

20

09

Used the MR5 design with 4 center points at each level of the categoric factor for a total of 76 runs.

St at -E

as

e,

A regular fraction requires a 211-4 or 128 runs.

In c.

Select a Design

Stent Delivery System

1. Build an 11 factor MR5 design ( g (the first ten factors are numeric and the last factor is categoric). Use the default factor names (letters A L) and levels (1 for numeric factors; L1 and L2 for the categoric factor) factor).

3. There are three responses:

DOE Process

20

09

St at -E

2. Add 4 center points. (Because factor L is categoric the number of center doubles to eight ) eight.)

as

e,

In c.

Build the MR5 Design

20

MR5 Design
All the main effects (MEs) and two-factor interactions (2FIs) are partially aliased with scores three-factor (and higher) interactions. But MEs and 2FIs are not aliased g ) with one another. If the three factor (and higher) interactions are ignored th th f t ( d hi h ) i t ti i d during design evaluation:

DOE Process

20

09

No aliases found for 2FI Model

St at -E

as

e,

In c.

Evaluate Aliases

21

MR5 Design
Power is reported at a 5.0% alpha level to detect the specified signal/noise ratio. Recommended power is at least 80%. R d d i tl t 80% Burst psig g ( ) ( g ) g ( g ) Signal (delta) = 6.00 Noise (sigma) = 8.00 Signal/Noise (delta/sigma) = 0.75 A B C D E F G H J K L 85.6% 85.6% 85.7% 85.6% 85.5% 85.3% 85.9% 86.0% 85.9% 85.8% 88.9% Push g/cm Signal (delta) = 15.00Noise (sigma) = 30.00 Signal/Noise (delta/sigma) = 0.50 A B C D E F G H J K L 52.2% 52.3% 52.4% 52.3% 52.1% 51.9% 52.5% 52.7% 52.5% 52.4% 56.4% 52 2% 52 3% 52 4% 52 3% 52 1% 51 9% 52 5% 52 7% 52 5% 52 4% 56 4% Track g*cm Signal (delta) = 10.00Noise (sigma) = 6.00 Signal/Noise (delta/sigma) = 1.67 A B C D E F G H J K L 99.9% 99.9% 99.9% 99.9% 99.9% 99.9% 99.9% 99.9% 99.9% 99.9% 99.9%

DOE Process

20

09

St at -E

as

e,

In c.

Evaluate Power

22

MR5 Design
Power is low (~52%) for Push; to increase power: ( 52%)
1. Increase design size: replicating the design 76x2=152 runs gives adequate power (~82%) ( 82%). No there are too many runs to be practical. 2. 2 Increase P h = 15 g/cm. No we are interested in g/cm Push a difference of 15 g/cm. 3. Decrease Push= 30 g/cm: By partitioning the variance, we determine that the push measurement contributes most (75%) of the variation. Yes Y repeating th push t t ( t replicating th ti the h test (not li ti the DOE runs) to reduce is the answer.

20

09

DOE Process

St at -E

See S next slide. t lid

23

as

e,

In c.

Evaluate Power

MR5 Design
2 2 2 Push = 30 & Push = Pr ocess + Measurement

2 2 Measurement Then by the CLT Average = n

St at -E

Make three independent push measurements f each run. M k th i d d t h t for h Enter the average of the measurements as the response:
:

20

09

2 Push

675 = 225 + = 450 3 Push = 450 21

DOE Process

as

e,

900 = 225 + 675 %Contribution = 675 900 = 75%

%Contribution = 50% 15 = = 0.71 21

24

In c.

Evaluate Power

MR5 Design
Power at 5 % alpha level for effect of p 0.71 Std Dev 0.75 Std Dev 1.67 Std Dev 81.6 % 85.6 % 99.9 % 8 81.7 % 85 6 85.6 % 99 9 99.9 % 81.8 % 85.7 % 99.9 % 81.7 % 85.6 % 99.9 % 81.5 % 85.5 % 99.9 % 81.3 % 85.3 % 99.9 % 81.9 % 85.9 % 99.9 % 82.1 82 1 % 86.0 86 0 % 99.9 99 9 % 81.9 % 85.9 % 99.9 % 81.8 % 85.8 % 99.9 % 85.4 85 4 % 88.9 88 9 % 99.9 99 9 %

DOE Process

20

Term StdErr** VIF Ri-Squared A 0.12 1.02 0.0148 B 0 0.12 1.01 0 00 0 0.0140 C 0.12 1.01 0.0109 D 0.12 1.01 0.0139 E 0.12 1.02 0.0183 F 0.12 1.02 0.0237 G 0.12 1.01 0.0070 H 0.12 1.00 0 12 1 00 0.0035 0 0035 J 0.12 1.01 0.0070 K 0.12 1.01 0.0101 L 0.12 1.02 0 12 1 02 0.0230 0 0230 **Basis Std. Dev. = 1.0

09

St at -E

as

e,

In c.

Evaluate Power

25

Evaluate Power
Summary:

DOE Process

20

The magnitude of each of these errors and the relative cost of replicating runs versus repeating measurements dictates which will give the most bang bang for your buck.

09

St at -E

Repeating the measurement reduces only the measurement error error.

as

Replicating runs will reduce the system error; from both process and measurement. measurement

e,

In c.
26

MR5 Design

DOE Process

20

Each run in the DOE builds a stent delivery system by combining different components and using a variety of assembly techniques. A knowledgeable engineer must look at each DOE run to ascertain that the unit can be made and will result in an operable stent delivery system. t

09

St at -E

as

Examine the design layout to ensure all the factor combinations are safe to run and are likely to result in meaningful information (no disasters)

e,

In c.

Examine Design Layout

27

Stent Delivery System

Design-Expert Software Burst Error from replicates Shapiro-Wilk test W-value = 0.987 p-value = 0.771 A: A B: B C: C D: D E: E F: F G: G H: H J: J K: K L: L Positive Effects Negative Effects
99.9 99.0

Half-Normal Plot

e, as
0.00 4.45 8.90

In c.
B BC
13.34

Results Burst

A C

Half-Norma % Probabili al i

95.0 90.0 80.0 70.0 50.0

20

09

St at -E
30.0 20.0 10.0 0.0

17.79

DOE Process

|Standardized Effect|

28

Stent Delivery System

ANOVA for selected factorial model Analysis of variance table [Partial sum of squares - Type III] Sum of Mean F p-value Source Squares df Square Value Prob > F Model 13264.56 13264 56 4 3316.14 3316 14 65.14 65 14 < 0 0001 0.0001 A-A 5324.51 1 5324.51 104.60 < 0.0001 B-B 1820.10 1 1820.10 35.75 < 0.0001 C-C 4172.29 4172 29 1 4172.29 4172 29 81.96 81 96 < 0 0001 0.0001 BC 2482.47 1 2482.47 48.77 < 0.0001 Curvature 43.69 1 43.69 0.86 0.3574 Residual 3563.38 3563 38 70 50.91 50 91 Lack of Fit 3071.38 64 47.99 0.59 0.8667 Pure Error 492.00 6 82.00 Cor Total 16871.63 75

DOE Process

20

09

St at -E

as

e,

In c.

Results Burst

29

Stent Delivery System

Design-Expert Software Push Error from replicates Shapiro-Wilk test W-value = 0.993 p-value = 0.978 A: A B: B C: C D: D E: E F: F G: G H: H J: J K: K L: L Positive Effects Negative Effects
99.9 99.0

Half-Normal Plot

e, as
0.00 9.88

In c.
K D
19.76 29.63

Results Push

F G FG

Half-Norma % Probabil al li

95.0 90.0 80.0 80 0

20

09

St at -E
70.0 50.0 30.0 30 0 20.0 10.0 0.0

39.51

DOE Process

|Standardized Effect|

30

Stent Delivery System

ANOVA for selected factorial model Analysis of variance table [Partial sum of squares - Type III] Sum of Mean F p-value Source Squares q df Square q Value Prob > F Model 85628.80 5 17125.76 48.14 < 0.0001 D-D 14300.39 1 14300.39 40.20 < 0.0001 F-F 26366.49 1 26366.49 74.12 < 0.0001 G-G 21637.78 1 21637.78 60.83 < 0.0001 K-K 5796.68 1 5796.68 16.30 0.0001 FG 17673.16 17673 16 1 17673 16 17673.16 49.68 49 68 < 0 0001 0.0001 Curvature 116.64 1 116.64 0.33 0.5688 Residual 24545.29 69 355.73 Lack of Fit 22153.54 22153 54 63 351.64 351 64 0.88 0 88 0.6466 0 6466 Pure Error 2391.75 6 398.63 Cor Total 1.103E+005 75

DOE Process

20

09

St at -E

as

e,

In c.

Results Push

31

Stent Delivery System

Design-Expert Software Track Error from replicates Shapiro-Wilk test W-value = 0.972 p-value = 0.181 A: A B: B C: C D: D E: E F: F G: G H: H J: J K: K L: L Positive Effects Negative Effects
99.9 99.0

Half-Normal Plot

e, as
E
0.00 3.95

In c.
BD D L B
7.91 11.86

Results Track

DE

Half-Norma % Probabil al li

95.0 90.0 80.0 80 0

20

09

St at -E
70.0 50.0 30.0 30 0 20.0 10.0 0.0

15.81

DOE Process

|Standardized Effect|

32

Stent Delivery System

ANOVA for selected factorial model Analysis f A l i of variance table [Partial sum of squares - T i t bl [P ti l f Type III] Sum of Mean F p-value Source Squares df Square Value Prob > F Model M d l 11175.64 11175 64 6 1862.61 1862 61 52.19 52 19 < 0 0001 0.0001 B-B 1388.59 1 1388.59 38.91 < 0.0001 D-D 982.81 1 982.81 27.54 < 0.0001 E-E 477.87 1 477.87 13.39 0.0005 L-L 1223.48 1 1223.48 34.28 < 0.0001 BD 2355.11 1 2355.11 65.99 < 0.0001 DE 4457.51 1 4457.51 124.90 < 0.0001 Curvature 178.00 2 89.00 2.49 0.0902 Residual 2391.15 67 35.69 Lack of Fit 1962.40 61 32.17 0.45 0.9471 Pure Error 428.75 6 71.46 Cor Total 13744.79 13744 79 75

DOE Process

20

09

St at -E

as

e,

In c.

Results Track

33

Stent Delivery System

Find optimal factor settings

DOE Process

A PDF copy of this paper (with permission) is available at: www.statease.com/pubs/derringer.pdf 34

20

* Derringer, G. C., A Balancing Act: Optimizing a Product's Properties, Quality Progress. 2002 American Society for Quality

09

St at -E

Track, Minimize (LL=170, UL=200) importance +++

as

Push, Push Maximize (LL=175 UL=275) importance +++ (LL 175, UL 275)

e,

Burst, Maximize (LL=40, UL=65) importance ++++

In c.

Results Optimize

Stent Delivery System

DOE Process

20

09

St at -E

as

e,
35

In c.

Optimal Factor Settings

Factorial Design Planning Process g g

Our t lk has three parts: O talk h th t

2. Illustrate key points via an example 3. Summary

DOE Process

20

09

St at -E

1. Broad brush description of th DOE 1 B db hd i ti f the planning process

as

e,
36

In c.

Factorial Design Planning Process (1 of 2)

1. Identify opportunity and define objective.

2. State objective in terms of measurable responses.

b. b Estimate experimental error () for each response.

DOE Process

3. 3 Select the input factors to study (Remember that study. the factor levels chosen determine the size of y.)

20

09

c. c Use the signal to noise ratio (y/) to estimate power.

St at -E

a. a Define the change (y) that is important to detect for each response.

as

e,

In c.
37

Factorial Design Planning Process (2 of 2)

4. Select a design and:

DOE Process

20

Examine the design layout to ensure all the factor E i th d i l tt ll th f t combinations are safe to run and are likely to result in meaningful information (no disasters) disasters).

09

St at -E

Evaluate power (desire power > 80% for effects of interest); generally use ME model (for robust design use only 1 ME).

as

Evaluate aliases (fractional factorials and/or blocked designs); generally use 2FI model model.

e,

In c.
38

Factorial Design Planning Process

Tools: Brainstorming (fishbone) Consensus Outputs Voting Form Outputs, y, %Contribution O t t , %C t ib ti

Identify Opportunity Define Objective

Factors Voting Form DOE inputs, levels, operating range Other inputs Select S l t an appropriate factorial d i i t f t i l design Evaluate aliases (fractional factorials and/or blocked designs) Evaluate power Examine the design layout

St at -E
39

as

e,

20

09

DOE Process

In c.

State Objective in Terms of Measurable Responses Select Input S l tI t Factors

Select Design Evaluate Aliases & Power Examine E i Layout

Caveat
The factorial design planning process emphasis on power should not be applied to response surface or mixture design.

Sizing for precision will covered in future webinars. webinars

DOE Process

20

09

St at -E

Precision should be used to size response surface and mixture designs designs.

as

e,

In c.

40

Factorial Design Planning Process g g

If you have questions please email them to:

and reference the September 19th webinar. f S

DOE Process

20

09

stathelp@statease.com

St at -E

as

Thank You for Attending g

e,

In c.
41