Evaluating and Institutionalising Organisational Development Interventions

Presented by-

Yusra Iqbal Prateek Singh Molla Asmare Taufeeq Ahmad

Evaluation is concerned with providing

feedback to practitioners and organization members about the progress and impact of OD interventions interventions a permanent part of organizations normal functioning.

Institutionalization involves making OD

 Evaluating

Organisational Development Interventions

Implementation and Evaluation Feedback Measurement Research Design

Institutionalising

Interventions

Institutionalisation Framework
Organisational characteristics
Intervention Characteristics Institutionalisation Process Indicators of Institutionalisation

Evaluating Organisational Development Interventions

Necessity of Evaluating OD Interventions

Judge whether intervention has been

implemented as intended or not and whether it is having desired results money=> Rigorous assessment necessary Decisions about the measurement of relevant variables and design of evaluation process is to be made early in the OD cycle so as to integrate evaluation with intervention decisions

Implementation of OD intervention cost

Implementation and Evaluation Feedback

Most OD intervention require significant

changes in peoples behaviour and attitude yet intervention typically offer broad prescriptions only for how such changes are to occur.
variety and meaningful feedback to the jobs. Implementing this can be a challenge as managers learn and experiment to translate the prescriptions into specific behaviours.

Eg. Job enrichment calls for addition of

So evaluation needs to include during implementation assessment to see if interventions are actually being implemented and after implementation assessment evaluation of whether interventions are producing expected results

Implementation Feedback consists of two

types of information
itself

Data about different features of intervention Data about immediate effect of interventions

Data is collected repeatedly at intervals This information is used to gain clearer

understanding of intervention and lan for next implementation step

This cycle may proceed for several rounds

Once implementation feedback shows that

intervention was sufficiently in place evaluation feedback begins. job satisfaction, absenteeism and turnover.

It includes outcome measures like performance, Negative results indicate that either initial

diagnosis was flawed or choice of intervention was wrong. implementation of intervention and should prompt search for ways to institutionalize the change.

Positive results indicate successful

Implementation Feedback Diagnosis Implementation of Intervention Choice of intervention Clarification of Intervention Development of plan for next implementati on steps Measures of features of implementation and immediate effect

Evaluation feedback Measureme nt of long term effects

Alternative Interventions

MEASUREMENT
Select the right variables to measure Design good measurements
Operational Reliable Valid

Research Design

Sources of Reliability
Rigorous Operational Definition
Provide precise guidelines for measurement:
How high does a team have to score on a five-point scale to say that it is effective?

Multiple Measures
Multiple items on a survey
Multiple measures of the same variable
(survey, observation, unobtrusive measure)

Standardized Instruments

Types of Validity
Face Validity: Does the measure
appear to reflect the variable of interest?

Ask colleagues and clients if a proposed

measure actually represents a particular variable.

Types of Validity
Content Validity: Do experts agree that
the measure appears valid?

If experts and clients agree that the

measure reflects the variable of interest then there is increased confidence in the measures validity

Types of Validity
Criterion or Convergent Validity: Do
measures of similar variables correlate?

Use multiple measures of the same variable,

to make preliminary assessments of the measures criterion or convergent validity.

If several different measures of the same

variable correlate highly with each other, especially if one or more of the other measures have been validated in prior research, then there is increased confidence in the measures validity.

Types of Validity
Discriminant Validity: Do measures of
non-similar variables show no association?

This exists when the proposed measure

does not correlated with measures that is not supposed to correlate with. daily measures of productivity to correlate with daily air temperature.

Example: there is not good reason for

Types of Validity
Predictive Validity: Are present variables
indicative of future or other variables?

This is demonstrated when the variable

of interest accurately forecasts another variable over time. can be said to be valid if it accurately predicts improvements in team performance in the future.

Example: A measure of team cohesion

RESEARCH DESIGN
Longitudinal Measurement Change is measured over time Ideally, the data collection should start
before the change program is implemented and continue for a period considered reasonable for producing expected results.

CONT..
Comparison Units

Appropriate use of control groups

It is always desirable to compare
results in the intervention situation with those in another situation where no such change has taken place.

CONT..
Statistical Analysis Alternative sources of variation have
been controlled

Whenever possible, statistical methods

or chance.

should be used to rule out the possibility that the results are caused by random error

Evaluating Different Types of Change

Alpha Change
Refers to movement along a measure that
reflects stable dimensions of reality.

For example, comparative measures of

perceived employee discretion might show an increase after a job enrichment program. If this increase represents alpha change, it can be assumed that the job enrichment program actually increased employee perceptions of discretion.

CONT
Beta Change Involves the recalibration of the
intervals along some constant measure of reality. For example, before-and-after measures of perceived employee discretion can decrease after a job enrichment program. If beta change is involved, it can explain this apparent failure of the intervention to increase discretion.

CONT
The first measure of discretion may accurately reflect the individuals belief about the ability to move around and talk to fellow workers in the immediate work area. During implementation of the job enrichment intervention, however, the employee may learn that the ability to move around is not limited to the immediate work area. At a second measurement of discretion, the employee using this new and recalibrated understanding, may rate the current level of discretion as lower than before.

CONT
Gamma Change
Involves fundamentally redefining the
measure as a result of an OD intervention. In essence, the framework within which a phenomenon is viewed changes.

For example, the presence of gamma

change would make it difficult to compare measures of employee discretion taken before and after a job enrichment program.

CONT..
The measure taken after the intervention
might use the same words, but they represent an entirely different concept. After the intervention, discretion might be defined in terms of the ability to make decisions about work rules, work schedules, and productivity levels. In sum, the job enrichment intervention changed the way discretion is perceived and how it is evaluated.

Institutionalizing Organizational Development Interventions

Institutionalising Interventions
The model of Institutionalisation
framework shows that both organisation as well as intervention characteristics affect different institutionalization processes operating in organisations. intervention processes

Organisation characteristics also affect

Eg. Organisation with powerful unions
find it difficult to get support for OD interventions.

Institutionalization Framework

Organizational Characteristics
Congruence Stability of environment Unionization

Institutionalization Framework

Intervention Characteristics
Goal specificity Programmability Level of Change Target Sponsor

Institutionalisation Framework
Institutional process
Socialization Commitment Reward allocation

Diffusion
Sensing and recalibration

Institutionalisation Framework
Indicators of Institutionalization
Knowledge Performance Preferences Normative consensus Value consensus

Organizational Characteristics Congruence Stability of environment Unionization

Institutionalization Process Socialization Commitment Reward allocation Diffusion Sensing and calibration Indicators of Institutionalization Knowledge performance Preference Normative Consensus Value Consensus

Intervention Characteristics

Goal specificity Programmability Level of Change Target Sponsor

Conclusion
Evaluating Evaluation

and Institutionalising interventions are the final two stages of them are planned change. of intervention also involves decisions about its measurement and research design. focuses on quantifying dimensions and magnitude of change. design focuses on setting up conditions for making valid assessment of intervention effect. is concerned with making the change permanent.

Measurement Research

Institutionalising