I have been tasked with looking at implementing a long-term training evaluation "process"/"system". We had one previously, but the HR administrators spent more time chasing non-returned evaluation forms than it was worth for the amount of information the forms provided.
However, the desire to get a clear indication of the effectiveness and value of training post the event is still (rightly, I believe) there.
Does anybody have any ideas on, or actual processes/systems for evaluating training received in the long-term, that they are prepared to share with me?
Thanks
Andy
Andy Tattersall
13 Responses
More work
I think the issue with your current system is not that it couldn’t work but that it doesn’t at the moment.
Whenever you rely solely on people returning information to you for this purpose you will be disappointed.
I think if you want to conduct long-term evaluation then it needs to be someone’s responsibility to visit (or call) the trainee and/or their manager and run through an evaluation with them.
I’ve previously used this method when delivering large scale training projects to external clients (incorporating the cost into the project cost) and it has worked really well.
You could take it to the next level (if you have the budget) by actually visiting your trainees and observing them to see if skills are actually being put into practice or whether it’s just lip service being paid to them.
In terms of evaluation I’m assuming here that you trying to evaluate the effectiveness of the learning rather than ROI.
Gauging ROI as has been discussed at length on training zone is often only really worth while on very large or very expensive or high risk programmes.
Evaluation of Training
Thank you for that Nik.
It is more to do with evaluating the effectiveness of the training than the ROI (the “Holy Grail”!!), and I am trying hard to get managers to understand that it is principally their responsibility to confirm that the training that they “stumped up” for, has done for the individual what they hoped it would. The trick as I see it is having a simple but effective “system” to enable this to happen (another “Holy Grail”??!!).
But thanks again for your comments – all grist to the mill as they say.
Andy
In that case
I’d use a very effective system for this one, train the managers on the importance of this (so they can’t say that they don’t know why), then after each training inform them that there will be a deadline for returning evaluation forms for each member who attends from their departments.
And that if they don’t make this deadline then there will be a £100 (or any number you like) charge for failing to do so against their departmental budget.
That should spur them on (again I’ve seen this method used effectively in large organisations).
a few thoughts
Hi there
one of my key roles is to evaluate training effectiveness and I have a few thoughts to share.
In the past, I’ve tried (unsuccessfully), to a implement a ‘fine’ system for unreturned questionnaires but the other departments all rallied against it and as a result, the HR dept wouldn’t support it, (the training and HR functions were separate depts). I also tried having a prize draw that you could only enter if you returned your form – this was marginally more successful but not in a massive way.
I then spent sometime re-designing the form to be more user friendly but nothing I tried worked. I couldn’t understnd why that was because the learning outcomes were really clear, (and externally accredited) and evaluation was clearly made against those outcomes. The best I could ever hope for was a 10% return.
The only thing I did, that worked, was to send out the questionnaires and get someone to follow up with a telephone call and ‘support’ learners to complete the form over the phone. This could be a temp, office junior or whoever has good telephone communication skills.
I feel that the reason that the returns weren’t forthcoming was that other depts didn’t really understand the value of evaluation and didn’t particularly buy into it. We are commited to completing full evaluations and with the method suggested above, we never get less than a 98% return. Once the system runs like clockwork, it takes no more than 2 days every 3 months to complete, (this will depend on the size of your org).
Hope that helps.
thanks
Link training to the business needs
I think that evaluation begins when you first decide that training is the solution to a problem. It is at this point that you can identify what you are trying to acheive. If at this point you can be really specific and clear about what you want to achieve, you can then use these measures later to ascertain whether the training has worked. I have also used a management behaviour survey before and after management training to measure how effective the training has been.
I hope that helps!
JILL PENNINGTON
Accountability
This seems quite simple to me (I’m a simple person!)
Managers are not being held accountable for the way they deploy their limited resources (people, money, time) and the performance results they deliver.
If a manager is going to release somebody from their designated day job to undertake some form of training, it must be for very clear reasons, to achieve very specific aims. It matters not if these aims are bottom-line related or ‘pink & fluffy’. There has to be a sense of ‘it was worth it’ after the event, ideally guaged in advance so that the decision to attend the training is made on the basis of good information with a better than evens chance of delivering value instead of (as is so often the case) destoying value.
It’s not about forms and chasing people – it’s about managers discharging their duties. Is it their fault? Probably not – in my experience so many are just not clear about their roles or how they should discharge their responsibilities – many are in these jobs for all the wrong reasons – and their line managers are just as bad and so it continues – right up to the top.
No, for me the root cause is plain and simple ineffective management.
It’s amazing how behaviours change when somebody ‘important’ starts asking awkward questions such as ‘what return did you get for sending Bloggs on that course – was it worth it – why – show me’
Last comment from my soap box – all the above implies that you don’t do ROI type evaluations for just the big, expensive training – you do it for ALL training – even quickly and in your head can be enough to make you more certain you are doing the right things.
Good luck
Martin Schmalenbach
get pre-training evaluation sorted first
Andy, the secret to sorting out your problem is not post-training but pre-training. First be clear of your measures in advance and second get the trainees to agree to collect these measures before they embark on the training. If you cannot get them to do that there is no point trying to measure the long-term. Also, if they cannot be bothered it probably means they are not taking the training seriously. Evaluation means building in success, not failure.
Incentives to return evaluation forms
Like many of us I experienced the difficulty of getting course participants to return their evaluation form. But as I want to find out what people truly want or need to know about a subject and get ideas on how to redefine my course contents, I find feedback very important. I’ve tried various things. Some more successful than others but what seriously improved the number of returned forms was my pledge to donate three pounds to “Children of Nepal”, a small British charitable organisation that has the aim of extending and improving educational opportunities for children living in Nepal. It gives the learner an incentive to return the form – or ask for 10 more, as happened recently.
Follow-up Activity
There isn’t a simple solution unless the delegates/managers want to or know that they will be measured against return rate. I think that Clive Boorman’s idea of a telephone follow-up is worthwhile, and we have used it successfully in the past.
Paul Kearns also makes a valuable point regarding pre-training. Another successful strategy we have used is to ask delegates to complete a short booklet with their manager. The first part is completed before attending the course and outlines what they want to get out of it. Immediately after the course, they then complete an action plan regarding what they will do as a result. The final section outlines what they have achieved.
Obviously, some managers do this without prompt, but some do need chasing. If time allows, inviting delegates and their managers to a lunchtime debriefing to discuss the outcomes/complete any further evaluations is a good method. A free lunch can be tempting to lots of people!
Long Term Evaluation Strategy
Hi Andy, I’ve got an article that gives a flavour of how to build an evaluation strategy that is right for your business. Feel free to drop me a line if it would be useful.
Best regards,
Richard
Long term evaluation
I can do no better than agree with the comments made above by Martin – if the learners and, more particularly the line managers, are not interested in whether the training has value (subjective or objective), forget it, because obviously nobody, including senior management appear to care.
One other point I noted about HR people chasing up the return of forms – again forget it! Simply ensure that there is time at the end of the event for any validation forms to be completed there and then – ergo, no need to chase anybody.
Leslie Rae
Thank you for responding
Many thanks to all of you who took the time to respond to my question. I’m sorry it has taken until now to acknowledge your contributions, but I have found mysylf busy with other issues since posting the question and only now able to find the time to return to it.
I take onboard all that has been said, and can see that there are a number of areas I need to tackle if I am to achieve my goal with evaluating and validating our training – an ongoing “project” but at least I have some good ideas on how to go about it. Thanks once more
Andy
Long-term evaluation
Andy
Long-term evaluation should be done by who are the most responsible people to do and in the relevant location – that is by the line manager after the learning event and in the workplace. It is not the responsibility or role of the trainer – surely you have quite enough to do without that (or you should have!). Trainers should concentrate on providing the most effective learning opportunities in line with the corporately agree specific objectives for the identified needs.
Leslie