I am conducting an evaluation of our companies training on the oracle IT system. I need to be looking at level three and four of Kirkpatricks so ensure that we are getting value for money. Any suggestions of any questionnaire questions i can use to gain this data using questionnaires a couple of months after the events took place?
craig mitchell
3 Responses
Level 3 & 4 Evaluation
Craig,
Assuming the training was set up with objectives then your level 3 questions should be phrased around the skill levels before and after the training for each objective. These could be completed by both line manager and delegate for comparison.
Do you have performance stats for the operators, if so Level 4 could be a comparator on these before and after.
NB Do not forget the CBA.
Level Four
What are your Level Four goals for the training?
If one of them is higher staff productivity in using the system for whatever it is intended, then you may want to follow one “best estimates” model of asking, how much more productive each user estimates they are, expressed as a percentage. Then you ask how confident they are with their estimation, again as a percentage.
When you analyse these two questions, you discount the first answer by the second to derive a “conservative” viewpoint.
For example, Joe says he estimates himself to be 20% more productive in using the system, given what he now knows. But he’s 50% confident in his estimation. So the net productivity improvement is 10% (50% of 20%).
As this method works on estimations, rather than “scientific data”, it’s worth getting your project sponsor’s buy-in to using this approach, before adopting it. But it’s an appoach that is used time and time again, where realism shows that a precise calculation is difficult, if not impossible.
Higher Level Evaluation
Hi Craig, I found “How To Measure Training Results” by Jack Phillips useful when things of ways to measure the results. I’d also be delighted to share our Five Principles of Brain Friendly Evaluation if it would be helpful. It may just give a slightly different perspective on the process.
Good luck with you quest!
Richard