ESD.

36J System & Project Management

+
-
Copyright 2003
James M. Lyneis
Dynamics of Project Performance
System Dynamics and Project Management
Class Three (9/30/03)
- ESD.36J SPM 9/30/03 2
+
-
Copyright 2003
James M. Lyneis
Topics
Review Practice for 9/30
The dynamics of project performance --
knock-on effects
Modeling the feedback effects
Model Calibration
Practice for 10/2
- ESD.36J SPM 9/30/03 3
+
-
Copyright 2003
James M. Lyneis
Topics
Review Practice for 9/30
The dynamics of project performance --
knock-on effects
Modeling the feedback effects
Model Calibration
Practice for 10/2
- ESD.36J SPM 9/30/03 5
+
-
Copyright 2003
James M. Lyneis
What did you conclude re. the
effect of productivity, quality,
and rework discovery time on
time required to finish the
project?
- ESD.36J SPM 9/30/03 6
+
-
Copyright 2003
James M. Lyneis
Sensitivity of Completion Date to
+/-33% Variation in Parameters
Quality - Completion Date
0
20
40
60
80
Low Q Base High Q
Productivity - Completion Date
0
20
40
60
80
Low Pdy Base High Pdy
Rework Discovery Delay -
Completion Date
0
20
40
60
80
Low Rwdd Base High Rwdd
- ESD.36J SPM 9/30/03 7
+
-
Copyright 2003
James M. Lyneis
Sensitivity of Total Work Done to
+/-33% Variation in Parameters
Quality - Total Work Done
0
50
100
150
200
Low Q Base High Q
Productivity - Total Work Done
0
50
100
150
200
Low Pdy Base High Pdy
Rework Discovery Delay - Total
Work Done
0
50
100
150
200
Low Rwdd Base High Rwdd
- ESD.36J SPM 9/30/03 8
+
-
Copyright 2003
James M. Lyneis
Sensitivity of Completion Date
Variations in Quality have biggest
impact --
Nonlinear effect
Determines total amount of work
Rework discovery time has the least
impact (but the simple model does not
have a quality on quality feedback).
- ESD.36J SPM 9/30/03 9
+
-
Copyright 2003
James M. Lyneis
Graph for Undiscovered Rework
8
6
4
2
0
0 5 10 15 20 25 30 35 40 45 50
Time (Month)
Undiscovered Rework : Class2 Step1 Task
Undiscovered Rework : Class2 Step1 Q100 Task
Undiscovered Rework : Class2 Step1 Q50 Task
Quality = .5
Quality = .75
Quality = 1.0
- ESD.36J SPM 9/30/03 10
+
-
Copyright 2003
James M. Lyneis
Graph for Work Done
100
75
50
25
0
0 5 10 15 20 25 30 35 40 45 50
Time (Month)
Work Done : Class2 Step1 Task
Work Done : Class2 Step1 Q100 Task
Work Done : Class2 Step1 Q50 Task
Quality = 1.0
Quality = .75
Quality = .5
- ESD.36J SPM 9/30/03 11
+
-
Copyright 2003
James M. Lyneis
Rework Shows Up As Revised
Work Products
Virtually all complex projects have revision cycles.
REV 0
REV 1
REV 3...
REV 2
Accomplishment of Revisions
to Design Products (e.g., Drawings)
TIME
- ESD.36J SPM 9/30/03 12
+
-
Copyright 2003
James M. Lyneis
Managing the rework cycle
Do common project metrics handle
(undiscovered) rework?
Do management incentives encourage
the discovery of rework?
- ESD.36J SPM 9/30/03 13
+
-
Copyright 2003
James M. Lyneis
Additional Metrics (Used in
Model)
Fraction Perceived to be Complete =
(Work Done + Undiscovered Rework) /
Initial Work to Do
Fraction of Work Really Complete =
(Work Done) / Initial Work to Do
- ESD.36J SPM 9/30/03 14
+
-
Copyright 2003
James M. Lyneis
Perceived vs. Really Complete
in Simple Rework Cycle Model
Progress
1
0.75
0.5
0.25
0
0 6 12 18 24 30 36 42 48 54 60
Time (Months)
Fraction Perceived to be Complete : Step2 Fraction
Fraction Really Complete : Step2 Fraction
- ESD.36J SPM 9/30/03 15
+
-
Copyright 2003
James M. Lyneis
On an actual project
Perceived vs. Actual Progress: SRS Development
Perceived Actual
100.
75.
50.
25.
0.
TIME
1/93 1/94 1/95 1/96 1/97 1/98 1/99 1/00 1/01 1/02
c:\mydocu~1\jml\hughes\hisjml.dyn 8/19/97 9:29
Disguised results
from actual
aerospace project
Year 1 Year2 Year 3 Year 4 Year 5 Year 6 Year 7 Year 8 Year 10 Year 9
- ESD.36J SPM 9/30/03 16
+
-
Copyright 2003
James M. Lyneis
Topics
Review Practice for 9/30
The dynamics of project performance --
knock-on effects
Modeling the feedback effects
Model Calibration
Practice for 10/2
- ESD.36J SPM 9/30/03 17
+
-
Copyright 2003
James M. Lyneis
Examples of Project Behavior Modes
Project
Staffing
Time
Typical
Plan
- ESD.36J SPM 9/30/03 18
+
-
Copyright 2003
James M. Lyneis
Examples (continued)
Productivity
(Normalised)
Time
1
2
Typical
Plan
- ESD.36J SPM 9/30/03 19
+
-
Copyright 2003
James M. Lyneis
Dynamics of Project Performance
The rework cycle
Quality
Undiscovered rework
Feedback effects on productivity and work
quality
Positive, re-enforcing, often vicious
circles
Negative, controlling
Knock-on effects between work
phases
- ESD.36J SPM 9/30/03 20
+
-
Copyright 2003
James M. Lyneis
A system dynamics model usually represents several
phases of work, but is more aggregate than a CPM
model
System
Engineering
Software Code
and Test
Hardware
Design
Hardware Build
and Test
Integrate
and Test
Software
Design
- ESD.36J SPM 9/30/03 21
+
-
Copyright 2003
James M. Lyneis
Software
System
Engineering
Software Development
Peace Shield Air Defence System Model
Architecture
SRS
Development
HW
Installation
& ICO Test
Op. &
Main/
ILS
Mgt. &
Admin.
Top Level
& Detailed
Design
Code
& Unit
Test
Integration
& Type II
Test
Type I
Test
KOSA
SW & Testing
Program Management Office
Software Support
CONUS
Customer Hardware Logistics Subcontractors
Other HASI
Programs
Downstream
Progress,
Availability,
& Quality Effects
Upstream Rework
Discovery Effects
Support Effects
- ESD.36J SPM 9/30/03 22
+
-
Copyright 2003
James M. Lyneis
Inter-phase Effects
Downstream &/or Parallel --
Availability of work products
Quality of work products
Upstream --
Rework discovery
Above apply internally and to suppliers
General --
Sharing of staff
- ESD.36J SPM 9/30/03 23
+
-
Copyright 2003
James M. Lyneis
LABOR
APPLIED
REWORK
DISCOVERY
WORK
BEING DONE
WORK
TO BE
DONE
WORK
REALLY
DONE
UNDISCOVERED
REWORK
KNOWN
REWORK
Productivity Quality
Inter-phase Connections:
Upstream-Downstream
LABOR
APPLIED
REWORK
DISCOVERY
WORK
BEING DONE
WORK
TO BE
DONE
WORK
REALLY
DONE
UNDISCOVERED
REWORK
KNOWN
REWORK
Productivity Quality
Progress; Quality
Progress
Design
Construction
- ESD.36J SPM 9/30/03 24
+
-
Copyright 2003
James M. Lyneis
LABOR
APPLIED
REWORK
DISCOVERY
WORK
BEING DONE
WORK
TO BE
DONE
WORK
REALLY
DONE
UNDISCOVERED
REWORK
KNOWN
REWORK
Productivity Quality
Inter-phase Connections: Parallel
LABOR
APPLIED
REWORK
DISCOVERY
WORK
BEING DONE
WORK
TO BE
DONE
WORK
REALLY
DONE
UNDISCOVERED
REWORK
KNOWN
REWORK
Productivity Quality
Progress;
Quality
Progress:
Quality
Software
Design
Hardware
Design
- ESD.36J SPM 9/30/03 25
+
-
Copyright 2003
James M. Lyneis
Dynamics of Project Performance
The rework cycle
Quality
Undiscovered rework
Feedback effects on productivity and work
quality
Positive, re-enforcing, often vicious circles
Negative, controlling
Knock-on effects between work phases
Availability and quality of work products
Progress to discover upstream rework
- ESD.36J SPM 9/30/03 26
+
-
Copyright 2003
James M. Lyneis
How might we represent suppliers?
platforms? ...
- ESD.36J SPM 9/30/03 27
+
-
Copyright 2003
James M. Lyneis
Topics
Review Practice for 9/30
The dynamics of project performance --
knock-on effects
Modeling the feedback effects
Model Calibration
Practice for 10/2
- ESD.36J SPM 9/30/03 28
+
-
Copyright 2003
James M. Lyneis
If the world consists only of stocks and flows,
what are those other variables indicated on the
diagrams??
Auxillaries
Constants (e.g., factors which may be
stocks or flows, but which do not
change over the time span of the
simulation)
External inputs
- ESD.36J SPM 9/30/03 29
+
-
Copyright 2003
James M. Lyneis
Auxillaries or Converters Serve
Several Purposes ...
Break up rates into meaningful
components
Provide alternative measures for stocks
or flows
Reduce diagram clutter
- ESD.36J SPM 9/30/03 30
+
-
Copyright 2003
James M. Lyneis
Potential Work Rate
Potential Work Rate = Staff * Productivity
Tasks/Month People * Tasks/Month/Person
- ESD.36J SPM 9/30/03 31
+
-
Copyright 2003
James M. Lyneis
Productivity & Quality
Productivity = ??
Quality = ??
- ESD.36J SPM 9/30/03 32
+
-
Copyright 2003
James M. Lyneis
Productivity --
PRODUCTIVITY = NORMAL
PRODUCTIVITY * EFFECT OF STAFF
EXPERIENCE * EFFECT OF QUALITY OF
PRIOR WORK * ...
Dimensions:
Productivity -- Tasks/Month/Person
Normal Productivity -- Tasks/Month/Person
Effects -- Dimensionless
- ESD.36J SPM 9/30/03 33
+
-
Copyright 2003
James M. Lyneis
Quality --
QUALITY = NORMAL QUALITY * EFFECT
OF STAFF EXPERIENCE * EFFECT OF
QUALITY OF PRIOR WORK * ...
Dimensions:
Quality -- Fraction
Normal Productivity -- Fraction
Effects -- Dimensionless
- ESD.36J SPM 9/30/03 34
+
-
Copyright 2003
James M. Lyneis
Effect of Staff Experience
Effect of Experience = Function ( Years
on Project)
- ESD.36J SPM 9/30/03 35
+
-
Copyright 2003
James M. Lyneis
Effect of Staff Experience
Effect of Experience = Function (Years on Project)
?
0
.2
.4
.6
.8
1.0
2
4 6 8 10
Years on Project
- ESD.36J SPM 9/30/03 36
+
-
Copyright 2003
James M. Lyneis
How do we determine these effects?
The effects are first estimated based on
common sense, later to be verified
during model calibration.
It helps to specify likely values at
extreme points, and draw a smooth
curve in between.
- ESD.36J SPM 9/30/03 37
+
-
Copyright 2003
James M. Lyneis
Effect of Staff Experience
Extreme points --
?
0
.2
.4
.6
.8
1.0
2
4 6 8 10
X
X
Years on Project
- ESD.36J SPM 9/30/03 38
+
-
Copyright 2003
James M. Lyneis
Connecting the extreme points ...
0
.2
.4
.6
.8
1.0
2
4 6 8 10
X
X
Years on Project
- ESD.36J SPM 9/30/03 39
+
-
Copyright 2003
James M. Lyneis
Some other connections are plausible ...
0
.2
.4
.6
.8
1.0
2
4 6 8 10
Years on Project
- ESD.36J SPM 9/30/03 40
+
-
Copyright 2003
James M. Lyneis
Others are not ...
0
.2
.4
.6
.8
1.0
2
4 6 8 10
Years on Project
- ESD.36J SPM 9/30/03 41
+
-
Copyright 2003
James M. Lyneis
Relationships Similar to the Experience Effect Would
Be Specified For Other Effects on Productivity and
Quality
Experience
Adequacy of supervision
Availability of resources
Morale
Schedule pressure
Fatigue
Quality of prior work
Skills available vs. needs
Tools
Processes
Availability of supplier
information &/or materials
...
- ESD.36J SPM 9/30/03 42
+
-
Copyright 2003
James M. Lyneis
Note difference with statistical
estimation --
Productivity = a + b*experience+c*prior
quality
- ESD.36J SPM 9/30/03 43
+
-
Copyright 2003
James M. Lyneis
Rules of Thumb and Checks
1. Variables and constants should correspond to real system
variables
2. Remember to represent delays -- in flow channels, in
perceiving information, and in making or implementing
decisions
3. Distinguish desires from realizations
4. Make sure that equations balance dimensionally
5. Check the validity of equations at extreme points, e.g.,
zero or very high values of the inputs
6. Avoid the use of imbedded functions -- they make the
equations harder to understand and can lead to errors
- ESD.36J SPM 9/30/03 44
+
-
Copyright 2003
James M. Lyneis
Connecting Morale & Overtime ...
from a students email:
On a couple of occasions in lecture yesterday you discussed
"Overtime" as it relates to "Morale". In doing causal loops, I find
that the relationship between the 2 can be different and
dependent on time. For example, initially the morale of
employees may increase or support the increase of overtime
which provides a positive relation between the 2. However, as
the overtime continues, I believe the morale will decrease as
employees will start to desire a life outside of work which
supports a negative relation between the 2.
I guess that in the actual SD modeling the developer would create
a curve/equation for morale over time, but what about for causal
loops? How do you handle the dynamics with causal loops?
Maybe the loops are just an initial starting point for the
development?
- ESD.36J SPM 9/30/03 45
+
-
Copyright 2003
James M. Lyneis
Representing morale
- ESD.36J SPM 9/30/03 46
+
-
Copyright 2003
James M. Lyneis
Topics
Review Practice for 9/30
The dynamics of project performance --
knock-on effects
Modeling the feedback effects
Model Calibration
Practice for 10/2
- ESD.36J SPM 9/30/03 47
+
-
Copyright 2003
James M. Lyneis
Calibration (Parameter Estimation)
- ESD.36J SPM 9/30/03 48
+
-
Copyright 2003
James M. Lyneis
Parameter Estimation in System
Dynamics
A priori estimates based on
direct observation
educated guesses by managers
similar parameters from other
applications
Refine initial estimates in the process
of calibration
- ESD.36J SPM 9/30/03 49
+
-
Copyright 2003
James M. Lyneis
Calibration is part of the scientific
method
Calibration is not curve-fitting via
exogenous variables, but systematic
comparison of model output to all
data and resolution of any
discrepancies
changes in structure
changes in parameters
decision to ignore (or sometimes use exogenous
effect)
- ESD.36J SPM 9/30/03 50
+
-
Copyright 2003
James M. Lyneis
Calibration Should Be a Part of the
Standard Method
Data is a real source of information
Calibration often uncovers
errors/omissions in insight-based
models
Cost is small relative to the stakes in
most large organizations
- ESD.36J SPM 9/30/03 51
+
-
Copyright 2003
James M. Lyneis
Why do we calibrate a model?
Assure model contains all of the structure
necessary to create problem
Accurately determine leverage points
Accurately price cost-benefits of
alternatives
Sell results to those not on project team
Generate a benchmarking data base for
future use and learning
- ESD.36J SPM 9/30/03 52
+
-
Copyright 2003
James M. Lyneis
Sources of information for calibration
Hard, numerical time series (e.g., for
staffing, work accomplished, overtime
levels, etc.)
Soft, management observations of
what happened (e.g., how morale,
experience, and other factors might
have caused productivity and quality to
change)
such observations must be tied to what
was happening on the project
- ESD.36J SPM 9/30/03 53
+
-
Copyright 2003
James M. Lyneis
Work Quality
to Date
Scheduled
Completion
Time
Expected
Completion
Time
Availability
of Prerequisites
Perceived
Progress
Schedule
Pressure
Out-of-Sequence
Work
Morale
Expected
Hours at
Completion
Hours
Expended
to Date
Skill &
Experience
Hiring
Equivalent
Staff on
Project
Staffing
Requested
Progress
Rework
Discovery
Turnover
Organisational
Size
Changes
Staff
Productivity Quality
Added
Work
Obsoleted
Work
Overtime
Time
Remaining
Work To
Be Done
Undiscovered
Rework
Known
Rework
Work
Really Done
Project Dynamics
Data likely to
be available
- ESD.36J SPM 9/30/03 54
+
-
Copyright 2003
James M. Lyneis
Once calibration is completed ...
We have a model that is useful for:
Learning
what really happened on prior projects
what could have been done better
Dispute resolution
- ESD.36J SPM 9/30/03 55
+
-
Copyright 2003
James M. Lyneis
What happens if you cant calibrate?
Use logical parameter estimates
Simulation model still better than mental
models
- ESD.36J SPM 9/30/03 56
+
-
Copyright 2003
James M. Lyneis
Topics
Review Practice for 9/30
The dynamics of project performance --
knock-on effects
Modeling the feedback effects
Model Calibration
Practice for 10/2
- ESD.36J SPM 9/30/03 57
+
-
Copyright 2003
James M. Lyneis
Lessons
Rework (Errors) Can Linger for Long Periods of
Time as Undiscovered Rework
Reducing Rework, and Detecting it Earlier, is the
Best Way to Cut Cycle Times, Finish Earlier, and
increase Delivered Quality
Common metrics & reward systems do not
encourage high quality and rework discovery
Interactions between phases of work, different
projects, suppliers, create opportunities for knock-
on effects
Calibration will improve the accuracy of our
parameter estimates, and ultimately policy decisions
- ESD.36J SPM 9/30/03 58
+
-
Copyright 2003
James M. Lyneis
Practice for 10/2
1. Download the Class2 Step2 model. This is
the rework cycle model with a variable time to
discover rework. Time to discovery rework,
as illustrated in the following figure, depends
on a maximum time and a minimum time, and
an effect of work progress. Time to discover
rework falls from the maximum to the
minimum time as fraction complete increases.
- ESD.36J SPM 9/30/03 59
+
-
Copyright 2003
James M. Lyneis
Rework cycle
with variable
time to discover
rework
Work to Do Work Done
Undiscovered
Rework
Work Accomplishment
Rework Generation
Rework Discovery
Quality
Staff
Productivity
Potential Work
Rate
Project Finished
Switch
Cumulative
Work Done
Rate of Doing Work
<Rework Generation>
<Work Accomplishment>
Feasible Work
Rate
Maximum Work
Rate
Minimum Time to
Perform a Task
Time to Discover Rework
<Work Done>
Work Believed to
Be Done
Fraction Really
Complete
Fraction Perceived
to be Complete
<Initial Work to Do>
Effect of Work Progress
Maximum Time to
Discover Rework
Table for Effect of
Work Progress
Minimum Time to
Discover Rework
- ESD.36J SPM 9/30/03 60
+
-
Copyright 2003
James M. Lyneis
Practice for 10/2
2. Add the effect of prior work quality on quality
as discussed in class and illustrated in the
following figure.
3. Simulate the model and compare to the
original Class2 Step 2 results. How does the
addition of quality on quality feedback affect
project performance?
4. If we essentially eliminate undiscovered
rework by setting the maximum time to
discover rework to 0.25 months, what
happens to project performance?
- ESD.36J SPM 9/30/03 61
+
-
Copyright 2003
James M. Lyneis
Work to Do Work Done
Undiscovered
Rework
Work
Accomplishment
Rework Generation
Rework Discovery
Quality
Staff Level
Productivity
Potential Work Rate
Time to Discover Rework
Project Finished
Switch
Cumulative
Work Done
Rate of Doing Work
<Rework Generation>
<Work Accomplishment>
Effect of Prior Work
Quality on Quality
Normal Quality
Table for Effect of
Prior Work Quality on
Quality
<Work Done>
Work Believed to
Be Done
Fraction Really
Complete
Fraction Perceived
to be Complete
<Initial Work to Do>
Cumulative
Effort
Expended
Effort Expended
<Staff Level>
<Project Finished Switch>
Effect of Work Progress
Maximum Time to
Discover Rework
Table for Effect of
Work Progress
Average Work
Quality
Feasible Work Rate
Maximum Work
Rate
Minimum Time to
Perform a Task
<Initial Work to
Do>
Minimum Time to
Discover Rework
- ESD.36J SPM 9/30/03 62
+
-
Copyright 2003
James M. Lyneis
Hints -- 1
Effect of Prior Work Quality on Quality:
kup - Table for Effect of Prior Work Qua
1
0
0 1
Note: The effect of prior quality on current quality is assumed to be proportional -- and
error in past work creates an error in current work. Given that in this simple model
quality represents several effects of work errors, this strong relationship may be
reasonable.
- ESD.36J SPM 9/30/03 63
+
-
Copyright 2003
James M. Lyneis
Hints -- 2
Average Work Quality:
Max(1e-006,Work Done)/Max(1e-
006,Work Believed to Be Done)
[Note: the max in the denominator
prevents division by 0, and in the
numerator allows the equation to return
a quality of 1.0 when no work is yet
completed.]
- ESD.36J SPM 9/30/03 64
+
-
Copyright 2003
James M. Lyneis
Hints -- 3
Normal Quality:
Set this to 0.85 (vs. 0.75 in Class2 Step2
model) to reflect that we are now
explicitly representing other factors
affecting project quality.
- ESD.36J SPM 9/30/03 65
+
-
Copyright 2003
James M. Lyneis
Graphical Functions
in Vensim
- ESD.36J SPM 9/30/03 66
+
-
Copyright 2003
James M. Lyneis
Step 1 - Set Up Model As Shown Below
Quality
Normal
Quality
Effect of Prior
Work Quality
Average Work
Quality
Table for Effect of
Prior Work Quality
- ESD.36J SPM 9/30/03 67
+
-
Copyright 2003
James M. Lyneis
Step 2 -- Equations ...
Quality = Normal Quality*Effect of Prior
Work Quality
Effect of Prior Work Quality = Table for
Effect of Prior Work Quality(Average
Work Quality)
- ESD.36J SPM 9/30/03 68
+
-
Copyright 2003
James M. Lyneis
Step 3 -- Define the Table
Click to define equation for Table for
Effect of Prior Work Quality
Select the Type as Lookup
Click on the As Graph button to see the
function
You can then type in values or create a
line on the blank graph
You can edit the values directly if desired
- ESD.36J SPM 9/30/03 69
+
-
Copyright 2003
James M. Lyneis
Voila!
kup - Table for Effect of Prior Work Qua
1
0
0 1
- ESD.36J SPM 9/30/03 70
+
-
Copyright 2003
James M. Lyneis
In the short-term, overtime increases
morale and productivity
Productivity
Morale
Overtime
Income
+
+
+
- ESD.36J SPM 9/30/03 71
+
-
Copyright 2003
James M. Lyneis
But in the longer-term, sustained overtime
reduces morale and productivity ...
Productivity
Morale
Overtime
Income
+
+
+
Average
Overtime
+
-
- ESD.36J SPM 9/30/03 72
+
-
Copyright 2003
James M. Lyneis
Connecting the feedbacks through
progress ...
Productivity
Morale
Overtime
Income
+
+
+
Average
Overtime
+
-
Progress
Staff Needed
+
-
+
+