I was pleased to be notified recently by Training Journal magazine that they were republishing an article I wrote that was first published some 3 years ago. Colleagues & contacts have been complimentary in their feedback to me, and I found myself being surprised by this.
Why am I surprised? Well, I found myself thinking "why this response now? Hasn't anything changed?"
And I don't feel it has - so please, put me right if I'm wrong!
People still seem to talk about evaluation, especially Kirkpatrick. I still see requests for evaluation forms to gauge learner reactions - so called happy sheets.
I had assumed that with the economic downturn there would be a huge interest in doing Level 4 evaluations in particular - measuring the impact of training (yes, including that old chestnut, ROI!) Yet I don't have any sense that this push for L4 evaluations has really happened yet.
So, what's going on? Are the big institutes and major training providers still pushing the 'old stuff' - traditional mindsets and tools? Are they still perpetuating what I see as myths, for example that Level 4 evaluations are expensive, disruptive and only to be conducted on very important or major programmes, or that Level 4 evaluations are too difficult to do?
I can't believe I'm the only person to believe that Level 4 evaluations are actually easy and cheap to do, or that they should be done for 100% of programmes, if not 100% of learners.
So, what's going on, what am I missing, and what do you think?
Martin Schmalenbach
5 Responses
slightly tongue in cheek?
Martin
Perhaps, particularly in an economic downturn, there is a reluctance to carry out a level 4 evaluation (or 3 or 2 or even 1!) for fear of discovering that the ROI isn’t actually laudible….the ramifications of this could be pretty disasterous for a senior manager (and team)who might then become vulnerable to the loss of his or her secure income. (Or an outsourced provider who might lose a lucrative contract).
(On a slightly more serious note there are examples I know of where the training intervention produced a successful business benefit outcome but the company still went belly up due to the bank’s foreclosure.)
Rus
PS “Not everything that counts can be counted and not everything that can be counted counts”
Albert Einstein
Rus
http://www.coach-and-courses.com
Aah – The Fear Factor!
Thanks for that take Rus!
It seems that some practitioners are gripped by fear – as you point out Rus – and understandable.
If this has been you or somebody you know, turning the situation on its head – removing the fear by undertaking a decent evaluation and taking steps to maximise value where gaps are thrown up by the evaluation could be a more useful way forward…
Still, fear is a powerful emotion – been there before!
Cheers
Martin
Seeking opportunity from the paralasys of others
Hi both
Interesting thought that the recession might be causing fear of being discovered and as a result causing paralysis.
Does this not offer a golden opportunity for those that can prove a benefit (by whatever methodology) of a course, programme or intervention?
Garry Platt our esteemed colleague on this forum spoke at our conference a couple of years ago on the very opportunity. A good/proper evaluation resulting in a return provides a USP and puts a trainer (or manager for that matter) at the front of queue for budgets.
I see that there may be some that will be paralised by the blinding light of the recession, but there will be others that do something and survive.
Peter
PS; sorry about the rabbit metaphore I had a nightmare recently that we were overrun with them:)
A nice idea…
I totally agree that being able to demonstrate a ROI in this climate could be (spectacularly) useful. However, perhaps other people have the same problem I do. I am the lone trainer / training manager / training admin / you name it, in an organisation of over 600 staff. We’re a manufacturing site and do a lot of training, ranging from health & safety, management development, technical (industry specific), NVQs, apprenticeships etc. I have developed a strategic approach to evaluate key training interventions to Kirkpatrick’s Level 3 by questioning a percentage of delegates, but even approach takes a HUGE amount of time. I’d love to work with managers to take it to the next level, but I simply don’t have the resources to do this. If you have any suggestions for practical tips, I’m all ears.
Still a lot of ignorance surrounding evaluation
Everyone is still quoting Kirkpatrick’s 4 levels – all of these do take time, are laborious and very bureaucratic – and then still don’t answer the $million question of ‘so did it add any value then?’.
The simple answer is you don’t need to do any post training evaluation if you get the Baseline sorted out before you start. Without this all of Kirkpatrick’s levels are a complete waste of time.
Baseline evaluation is not a ‘nice idea’ Jane – you can’t call yourself a professional without it.