Professional Documents
Culture Documents
model
Charles Tennant
Warwick Manufacturing Group, University of Warwick, Coventry, UK
Mahithorn Boonkrong
Royal Thai Army, Thailand
Paul A.B. Roberts
Warwick Manufacturing Group, University of Warwick, Coventry, UK
[ 230 ]
Charles Tennant, design the main elements of the survey is aimed at developing organisational and
Mahithorn Boonkrong questionnaire. This research was individual competencies to enable delivery of
and Paul A.B. Roberts subsequently critically analysed to design both long- and short-term business goals, this
The design of a training
programme measurement the proposed training programme aspect is often absent in Western
model measurement model. organisations (Tennant and Roberts, 2000). A
Journal of European Industrial number of current approaches that can be
Training
26/5 [2002] 230±240 applied to achieve this include the following:
Literature review 1 Seven wastes. The seven wastes in
manufacturing were identified by Taiicho
The compelling need for effective training Ohno (creator of the Toyota Production
The literature review revealed that it has System) as:
long been well established that Japanese . overproduction;
companies have emphasised training more . waiting;
than US and European manufacturing . transporting;
companies. This is particularly evident in the . inappropriate processing;
automotive sector with 380.3 training hours . unnecessary inventory;
per year for Japanese workers, compared . unnecessary motions; and
with 46.4 for US and 173.3 for European . defects (Bicheno, 1998).
workers, which has led to dramatic They can provide an effective framework
performance improvements in terms of cost, for developing a strategy based on kaizen
quality and delivery in Japanese companies (continuous improvement) principles, by
(Womack et al., 1990). developing an appropriate training
The potential business benefits to be programme of data gathering, problem
accrued from the effective training of people identification, cause-and-effect analysis,
has been recognised in the UK, as research by and solution development (Imai, 1986).
the DTI has concluded that: The subsequent training programme can
Recent evidence suggests a 76 per cent profit
be based around the seven basic tools of
gain per employee in businesses, which have
quality, which are ideally suited for
achieved ``Investor in People'' status.
Furthermore, in times of great change, application by production operators
learning is the key skill (DTI et al., 2000). (Ishikawa, 1972).
2 Hoshin kanri. Hoshin kanri (also known
Also, Goldstein and Associates (1989) have as policy deployment) is an organising
previously indicated that employees, framework for strategic management,
managers and organisations frequently rely which is concerned with the following
on training as a solution to work issues. four primary tasks (Tennant and Roberts,
Yet, although the benefits are recognised, 2001):
other research has concluded that 85 per cent . the setting of a few strategic priorities;
of UK-based employers do not attempt to . alignment of the strategic priorities
assess the benefits from training with local plans and programmes;
programmes, and therefore are unable to . integration of the strategic priorities
quantify their results (HMSO, 1989). In view with daily management; and
of this, it is believed that developing more . a structured review of progress.
effective training programmes for production Planning and deployment are critical
operators, that include appropriate levels of elements of hoshin kanri, which imply
objective setting and evaluation, is an that the process of determining targets,
ongoing problematic issue in the UK the development of means to achieve the
manufacturing sector. targets, and the implementation of both
are crucial for success. The hoshin kanri
A strategic approach to training plan can form the basis for a
It has previously been emphasised that a company-wide training programme,
strategic approach should be adopted, to which is aimed at aligning the strategic
ensure that training programmes are goals with daily management and
consistent with the overall objectives of the working processes. Japanese companies
organisation (Beer et al., 1985). This view is have been using this approach for over 30
supported by Kenny and Reid (1986), who years, as a more effective technique for
argue that evaluation: directing and motivating the workforce
Can be used to demonstrate a clear and than the western concept of management
reliable linkage between training by objectives (MBO) (Tennant and
effectiveness and the fulfilment of strategic
Roberts, 2000).
organisational objectives.
3 The balanced score-card. The balanced
Yet, although Japanese companies tend to score-card is a performance measurement
take a strategic approach to training, which technique, which was created by Kaplan
[ 231 ]
Charles Tennant, and Norton (1996). It can be described as a NVQ system, in terms of measurement, are
Mahithorn Boonkrong derivation of hoshin kanri, in that it is that they provide organisations with a
and Paul A.B. Roberts aimed at deploying strategic goals into mechanism for evaluating employees' skills
The design of a training
programme measurement individual score-card plans under four after the training has been completed. The
model perspectives: authors discussed these approaches with a
Journal of European Industrial . financial; leading UK-based NVQ provider, who stated
Training customer;
26/5 [2002] 230±240
. that they recommend a training cycle to their
. internal business processes; and customer organisations incorporating:
. innovation and learning. identification of training needs, plan and
The innovation and learning perspective design training, deliver training, and
represents an ideal platform from which evaluate training outcomes (Weatherly and
to develop a company-wide training Du Toit, 2000).
programme.
Preferred learning styles
Different methods of training Knowles (1990) describes two models for
Training methods can generally be learning, where pedagogy is defined as the
categorised as either on-the-job or off-the-job. process of teaching children, and andragogy
On-the-job training is the most frequent is the process for teaching adults. The main
method adopted, where the learner develops distinction is the differing responsibilities of
skills in the real-work environment, by teacher and student between the two models,
actually using the same machinery and as shown in Table I.
materials during the training as they would As adults, production operators in
when actually carrying out their work manufacturing industry are andragogical
post-training. Coles (2000) concludes that it is learners and have specific assumptions about
an effective method, because the learners their learning expectations. These are based
apply their training in real-time rather than around learning, which is relevant to
sitting in a classroom environment, and themselves as individuals, whilst allowing
forgetting what they have learned when they them to apply the learning in an industrial
return to their work. However, off-the-job context to further their personal
training provides opportunities to widen the development. The assumptions are described
boundaries of the teaching and can often be a against five criteria, as shown in Table II
useful initial step ahead of on-the-job (Gibbs, 1995).
training. Further considerations regarding the
The training delivery options for either needs of the learners are that, although as
method can be sourced from either in-house mature adults they may be highly motivated,
or external sources, or a combination of both. they can also be anxious, particularly if they
In-house training is more suited to on-the-job have not been in a training environment for
training and should be set in the context of some time. A key issue, which inhibits the
``focused learning'', where the learners begin effectiveness of training, is that people often
to practise new knowledge and skills for their return to the same inadequate workplace
own work during the actual training course environment. This leads to trainees feeling
(Tennant, 1995). External training sources frustrated that they have not been able to
can be either single or multi-company apply their learning, and the company
schemes. The latter provides a useful believing that the training programme was
opportunity for the learners to mix with not effective.
people from other companies and share There are a number of models and
experiences. concepts, which need to be considered when
Many organisations in the UK are using designing teaching methods for andragogical
National Vocational Qualifications (NVQs) learners. Kolb (1984) describes a concept
as a means for developing their people, by known as experiential learning, where it is
relating the training programmes directly to believed that experience plays a central role
their normal work. In effect they are in the learning process. A four-stage cycle for
assessments, which focus on the attainment learning is proposed, as shown in Figure 1.
of competencies in the workplace, rather The model relies on here-and-now concrete
than merely providing examinations. They experience to validate concepts, where
are a relevant method for enabling immediate personal experience is the focal
businesses to achieve business objectives, point for the learning process.
through more effective people development
(Holyfield and Moloney, 1996). Organisations The evaluation of training
can participate in NVQs either by becoming Customer satisfaction with training
an NVQ centre, or by using local NVQ programmes (whether internally or
centres. The benefits of participation in the externally delivered) is an important topic
[ 232 ]
Charles Tennant, Table I
Mahithorn Boonkrong Comparison between pedagogy and andragogy models
and Paul A.B. Roberts
The design of a training Assumptions
programme measurement
model About . . . Pedagogy Andragogy
Journal of European Industrial The learner Dependent personality Increasingly self-reliant
Training Role of learner's experience To be built on Resource for learning
26/5 [2002] 230±240
Readiness to learn Determined by age Developed from life tasks
Orientation Subject-oriented Task or problem-centred
Motivation By external rewards Internal incentives/curiosity
Source: Knowles (1990)
Table II
Assumptions regarding andragogical learners
Assumptions
The learner Increasingly self-reliant
Role of learner's experience Resource for learning
Readiness to learn Developed from life tasks
Orientation Task or problem-centred
Motivation Internal incentives/curiosity
Source: Gibbs (1995)
Table III
Considerations for designing assessment tasks
Considerations for assessment
Validity Does it measure what you want it to measure?
Reliability Does it measure in a predictable way?
Practicality Is it useful in real life?
Fairness Does it advantage or disadvantage any students?
Source: Cotton (1995)
[ 234 ]
Charles Tennant, Table IV
Mahithorn Boonkrong Comparison between the Kirkpatrick and CIRO training models
and Paul A.B. Roberts
The design of a training Criteria Kirkpatrick CIRO
programme measurement
model Focused areas Reaction Context
Journal of European Industrial Learning Input
Training Behaviour Reaction
26/5 [2002] 230±240
Result Outcome
Strengths Identify behavioural change with an Measures pre- and post-training to establish
emphasis on the change of ability and whether the objectives of the training were
apply to jobs achieved
Weaknesses No focus on training objectives, only Requires more training resources and has no
post-training measures behavioural focus
Table V
Typical training evaluation measures
Industry sector Evaluation measures
Food and beverage Feedback from trainees, trainers and managers
Department performance
Defects per period
Accident rate
Output quality
Machine utilisation and productivity
Individual performance
Pharmaceutical NVQs awarded
Job improvement
Individual attitude
Management views
Cost savings
Improved asset performance
Electronics Output and speed of production
Product quality
Financial turnover
Accidents
Customer complaints
Automotive Impact on cost, quality and delivery
Impact on customer perception
Employability of individuals
NVQs awarded
Accidents
Absenteeism levels
Construction materials Customer complaints
Lead time
Accidents
Steel Safety
Rework
Quality
Shoes Revenue
Productivity
Quality
[ 236 ]
Charles Tennant, strategic objectives is in fact the most the effectiveness of management training, its
Mahithorn Boonkrong influential barrier, as this often leads to the main elements could be applied in measuring
and Paul A.B. Roberts
The design of a training manifestation of the other causes. The training programmes for production
programme measurement research has led the authors to believe that operators. This is because it considers
model training programmes for production important aspects, such as context, inputs
Journal of European Industrial operators in the UK tend to be reactive and and reactions, which should be measured
Training
26/5 [2002] 230±240 ad hoc, suggesting an attitude of ``if it aint before the training is delivered. However, the
broke, don't fix it!'' Kirkpatrick model does not focus on
pre-training measures, rather it considers
measuring the behavioural change of the
Recommendation of a new training learners post-training.
measurement model Therefore, positive attributes of the two
models have been combined with new
This research has led to the development of a
elements, which consider measuring the
training programme measurement model,
change during the actual training. This
which can be used as a framework to enable
aspect was considered to be as important as
organisations to measure the effectiveness of
measuring before and after training, because
their training programmes for production
it gives the training provider an opportunity
operators. The model was created by
to improve the quality of the training in
combining elements of both the Kirkpatrick real-time.
and the CIRO models, and incorporating the The training measurement model can be
authors' personal perspectives. Although the seen in Figure 2.
CIRO model is generally used for measuring The model shows that the measures of
effectiveness have to be considered
concurrently with the training programme.
Figure 2 Before delivering the training programme,
Training programme measurement model
the organisation and training provider
(whether internal or external) have to
consider measuring the initial factors, which
will influence the ability of the training
programme to provide the learners with the
required new skills. Measurement should
also be carried out during the actual training
in order to allow the training provider to
respond to the individual needs of particular
learners in real-time. After completion of the
training programme, three additional
measurement steps are recommended within
the model as follows
Immediate test
An immediate test is required for testing the
new skills that the learners have gained from
the training programme. It should be carried
out immediately after the training
programme has been completed.
Intermediate test
An intermediate test should be carried out
when the learners return to their jobs, to
identify whether they can apply what they
have learned in their workplace
environment.
Ultimate test
An ultimate test should be used after an
appropriate time has elapsed in order to
measure the improvement of the skills, and
behavioural changes.
Table VI shows how the various elements
of the model can be implemented.
[ 237 ]
Charles Tennant, Table VI
Mahithorn Boonkrong Training measurement framework
and Paul A.B. Roberts
The design of a training What? Who? When? Where? Why? How?
programme measurement
model Objectives Company and Before training At the company To ensure that Define company
Journal of European Industrial training provider the requirements objectives and
Training of the company new skill levels
26/5 [2002] 230±240
will be met required
Inputs Company and Before training At the company To ensure that Test the training
training provider the inputs will inputs against
satisfy the the objectives
training needs
Reactions Company and Before training At the company To identify Issue a
trainee attitudes prior questionnaire to
to the training the trainees
Skills Trainer and Before training At the company To identify initial Monitor and
trainee skills prior to record trainee's
the training current job
performance
Measurement Trainer and During training At the training To identify Practical tests
during training trainee venue progress and and
develop the examinations
training material
Immediate test Trainer and After training At the training To identify new Practical tests,
trainee venue skills and questionnaires
attitudes after and module
the training reviews
Intermediate Company and Upon return to At the company To identify any Monitor new
test trainee work changes in work work
performance performance and
environment
Ultimate test Company and Ongoing At the company To identify skills Monitor new
trainee process retention and work
impact on performance and
company environment
performance
The model has not yet been tested in measures should be implemented
industry. However, the authors believe that concurrently with the training
this research has led to the development of a programme, in order to achieve
model that addresses the barriers previously continuous improvement of training
identified. Further research needs to be courses, and help organisations identify
carried out to test the model, and it is the true value of training rather than
intended to invite training practitioners to merely the cost.
respond to this. . A number of frameworks and tools exist,
which organisations should consider,
when developing an appropriate set of
Conclusions training objectives and output measures.
. The research has led to the conclusion However, organisations have to be careful
that UK-based manufacturing to select the most compatible tools for
organisations continue to struggle with implementation, which must be
the task of developing and implementing championed and led by top management.
effective training programmes for . The authors have created a training
production operators, which are linked to programme measurement model, by
their business objectives of improving combining elements of the Kirpatrick and
quality, whilst reducing costs and CIRO models, and including additional
lead-times. A number of causal factors measures. An important aspect of the
exist, of which insufficient training model is the incorporation of tests
evaluation is significant. The evaluation (immediate, intermediate and ultimate) to
[ 238 ]
Charles Tennant, evaluate new knowledge and skills gained DTI, DFEE and FEDA (2000), Quality Learning
Mahithorn Boonkrong as a direct result of the training Skills, and Campaign for Learning, The
and Paul A.B. Roberts Future of Corporate Learning, London.
The design of a training programme.
programme measurement . The implications of this research are that Education + Training (1998), ``Medium-size firms:
model UK manufacturing organisations need to waste money on training'', Education +
Journal of European Industrial radically change their policies for the Training, Vol. 40 No. 4, pp. 163-5.
Training Gibbs, G. (1995), ``Improving the quality of student
26/5 [2002] 230±240 training of production operators to ensure
that the training is aimed at delivering learning through course design'', in
key strategic goals, rather than relying on Goldstein, I.L. and Associates (Eds) (1998),
standard suites of courses. This is Training and Development in Organisations,
problematic, as it has been established Jossey-Bass Publishers, San Francisco, CA.
Hesseling, P.G.M. (1966), Strategy of Evaluation
that the involvement of employees in the
Research in the Field of Supervisory and
strategic management process is not well
Management Training, Van Gorcum,
practised in the West, let alone the UK.
Amsterdam.
Therefore, further research needs to be
HMSO (1989), Training in Britain: A Study of
carried out to develop adaptations of
Funding, Activity and Attitudes, Her Majesty's
established strategy execution methods
Stationery Office, London.
(such as the Japanese technique of hoshin Holyfield, J. and Moloney, K. (1996), Using
kanri), which can then be applied in National Standards to Improve Performance,
practice at UK manufacturing Kogan Page, London.
organisations. This may enable Imai, M. (1986), Kaizen ± The Key to Japan's
companies to apply training programme Competitive Success, McGraw-Hill Publishing,
measurement models (such as suggested New York, NY.
in this paper) to integrate their training Ishikawa, K. (1972), Guide to Quality Control,
programmes successfully with strategic Asian Productivity Organisation, Tokyo.
objectives, and deliver bottom-line value. Kaplan, R.S. and Norton, D.P. (1996), The
The authors (Tennant and Roberts) are Balanced Score-card, Harvard Business
currently developing such adaptations as School Press, Boston, MA.
part of an ongoing research programme. Kenny, J. and Reid, M. (1986), Training
Interventions, IPM, London.
References Kirkpatrick, D.L. (1994), Evaluating Training
ASTD (1998), ``National HRD executive survey: Programs: The Four Levels, Berrett-Koehler,
measurement and evaluation (1997)'', Fourth San Francisco, CA.
Quarter Survey Report, available at: Knowles, M. (1990), The Adult Learner: A
www.astd.org/virtual_community/research/ Neglected Species, Gulf, Houston, TX.
nhrd_executive_survey_97me.htm (Web site Kolb, D.A. (1984), Experiential Learning:
accessed 12 May 2000). Experience as the Source of Learning and
Beer, M., Spector, B., Lawrence, P.R., Quinn Development, Prentice-Hall, Englewood Cliffs,
Mills, D. and Walton, R. (1985), Human NJ.
Resource Management: A General Manager's Lewis, P. and Thornhill, A. (1994), ``The
Perspective, The Free Press, New York, NY. evaluation of training: an organisational
Bicheno, J. (1998), The Lean Toolbox, Picsie culture approach'', Journal of European
Books, Buckingham. Industrial Training, Vol. 18 No. 8, pp. 25-32.
Boonkrong, M. (2000), ``Measuring the benefits of McCarthy, A.M. and Garavan, T.N. (2001), ``360
training'', MSc project dissertation, Warwick degree feedback process: performance,
Manufacturing Group, University of improvement and employee career
Warwick, Coventry. development'', Journal of European
Campbell, C. (1998), ``Training course/programme Industrial Training, Vol. 25 No. 1, pp. 5-32.
evaluation: principles and practices'', Journal Mann, S. and Robertson, I.T. (1996), ``What should
of European Industrial Training, Vol. 22 No. 9, training evaluations evaluate?'', Journal of
pp. 323-44. European Industrial Training, Vol. 20 No. 9,
Coles, M. (2000), ``Virtual universities are just the pp. 14-20.
job'', The Sunday Times, 21 May. Mulder, M. (2001), ``Customer satisfaction with
Cooper, M. (1994), ``Evaluating professional training programmes'', Journal of European
training'', Training and Development, Industrial Training, Vol. 25 No. 6, pp. 321-31.
October, pp. 26-31. Phillips, J.J. and Chagalis, G.P. (1990),
Cotton, J. (1995), The Theory of Learning ``Evaluation of HRD programmes:
Strategies: An Introduction, Kogan Page, quantitative and qualitative'', in Nadler, L.
London. and Nadler, Z. (Eds), The Handbook of Human
Deming, W.E. (1986), Out of the Crisis, Cambridge Resource Development, John Wiley & Sons,
University Press, Melbourne. New York, NY.
[ 239 ]
Charles Tennant, Reid, M.A. and Barrington, H. (1997), Training Tennant, C. and Roberts, P. (2001), ``Hoshin kanri:
Mahithorn Boonkrong Intervention: Managing Employee implementing the catch-ball process'', Long
and Paul A.B. Roberts Development, IPD, London. Range Planning, Vol. 34, pp. 287-308.
The design of a training
programme measurement Tennant, C. (1995), ``Quality and Torrington, D. and Hall, L. (1995), Personnel
model reliability through common business Management: HRM in Action, P.J. Press,
Journal of European Industrial environment'', TQM Magazine, Vol. 7 Padstow.
Training No. 5, pp. 7-15. Weatherly, L. and Du Toit, P. (2000), ``Interview
26/5 [2002] 230±240 Tennant, C. and Roberts, P. (2000), ``Using carried out on 7 July 2000'', Midland Group
hoshin kanri for strategy deployment'', Training Services, Coventry.
International Journal of Manufacturing Womack, J.P., Jones, D.T. and Roos, D. (1990), The
Technology and Management, Vol. 2 No. 1-7, Machine that Changed the World, Macmillan
pp. 517-31. Publishing, New York, NY.
[ 240 ]