How well integrated is the evaluation process in your organisation's L&D strategy? Kenneth Fee and Dr Alasdair Rutherford give us a few pointers.
Evaluation is one of the essential components of any successful learning and development strategy, and yet it's not widely implemented – or implemented well.
The classic learning and development cycle includes four phases:
- identifying and analysing learning needs
- planning learning interventions
- delivering learning interventions
- evaluating learning and development
Each of these is equally important - not least evaluation, which informs all the other phases of the cycle.
Action required
Organisations need to pay more attention to learning evaluation. Too many do little or nothing – a recent European Union survey indicates that less than one third of organisations even claim to evaluate business impact, or results, of learning and development, and the number who actually carry out this sort of evaluation must be even lower.
"Evaluation should be an integral part of the plan for any learning intervention; if it's not, the intervention will never be complete or successful, or if it is, nobody will know."If you don't evaluate learning and development, you can't tell whether it's worked or not. You can't tell whether it was worth the investment. And you can't tell whether you should do it again. Evaluation should be an integral part of the plan for any learning intervention; if it's not, the intervention will never be complete or successful, or if it is, nobody will know.
A useful rule of thumb is that you should spend 10-15% of your learning and development budget on evaluation. If you don't do this, you're probably not paying enough attention to evaluation.
Attention to detail
Many organisations that claim to conduct evaluation don't do so robustly, systematically, or effectively. They don't stop to think about what they're aiming to find out, and the best way to do that. They don't identify what they need to measure, or how they can measure it.
It's commonplace to use a reaction sheet at the end of a training course, and to do so as a matter of routine, and yet insufficient thought goes into how the information in the sheets will be used. Even then, this is the most basic information that may be collected about learning and development, and probably the least useful. Much more significant are measures of how much people have learned (new knowledge, skills and competences), how much they transfer, or apply, to their work, in terms of changed attitudes and behaviour, and improved performance, and what results, impact or outcomes the business ultimately achieves.
Organisations need to decide what outcomes they want to accomplish, how learning and development contributes to these, and what outputs they want to measure. They need to ensure that evaluation is better understood and applied.
Choosing the right tools
Too much discussion of evaluation is needlessly dogmatic. There are fierce arguments between proponents and opponents of the Kirkpatrick model; there are those who dismiss Kirkpatrick, preferring Six Sigma or ROI; and there are some who dismiss ROI in favour of ROE. The truth is that all models and tools have their place – none is completely wrong, and none is the perfect panacea – and organisations need to be able to select the right one for the right situation. There are a great many evaluation models and tools, and just as many petty disagreements about what works best. Organisations need to be able to rise above these petty disagreements, rather than take sides, and consider with an open mind what will work best for them.
"Evaluation needs to be objective and impartial. Vendors of learning and development services cannot be anything but partial about the services they provide, and therefore should not be given a brief to evaluate their own work – or at least not on their own."
Independence
Evaluation needs to be objective and impartial. Vendors of learning and development services cannot be anything but partial about the services they provide, and therefore should not be given a brief to evaluate their own work – or at least not on their own. Internal learning and development staff may also be biased in favour of their own projects or responsibilities, or may be too close to a problem to see the solution; and internal staff often don't have the range of expert skills and knowledge to apply evaluation methodologies. Organisations should value an independent perspective, and source objective and impartial external support for evaluation.
Business focus
Learning and development should be inextricably linked with the organisation's business outcomes.
Evaluation solutions ought to help organisations make the most of their learning and development investments by:
- identifying what should be measured in learning and development
- establishing clear links between learning and development and business outcomes
- understanding better and demonstrating clearly both the costs involved in learning and development and all the value it generates
- making better business decisions about where (and where not) to invest in learning and development in future.
These are the basics, and yet many organisations are not addressing them. This needs to change: if learning and development is to make a significant contribution to accomplishing business outcomes, it's vital to get learning evaluation right.
This is article first appeared on the Airthrey website
Kenneth Fee and Dr Alasdair Rutherford are the founding directors of learning evaluation firm Airthrey Ltd. Ken is a career learning and development professional, whose latest book, 101 Learning & Development Tools, deals with evaluation among other topics. Alasdair is an evaluation and econometrics specialist, and a Research Fellow at the University of Stirling.
