Earlier this month, the great Donald Kirkpatrick announced that next year will be his last for public speaking. We revisit an interview with former HRZone.co.uk editor Annie Hayes.
His story begins back at the University of Wisconsin: "I was teaching and I thought to myself 'as long as I'm running seminars why don't I evaluate them'," says Kirkpatrick who humbly admits that all he really wanted to do, to begin with, was to evaluate the reaction. "I found though this wasn't enough, I wanted to know if they'd learned anything but then that didn't seem enough either. I needed to know if they'd taken anything back to their jobs, whether they changed their behaviour and got positive results," explains Kirkpatrick.
The seed of thought formed the basis of his now famous evaluation model based on four tiers:
- Reaction: what they thought and felt about the training
- Learning: the resulting increase in knowledge, skills and/or change in attitudes
- Behaviour: the extent of behaviour change
- Results: the impact on the business resulting from the learner's performance
"I was teaching and I thought to myself 'as long as I'm running seminars why don't I evaluate them'. I found though this wasn't enough, I wanted to know if they'd learned anything but then that didn't seem enough either. I needed to know if they'd taken anything back to their jobs, whether they changed their behaviour and got positive results."
"I got my PhD and five years later, the editor of the American Society for Training and Development (ASTD) asked me if I wanted to write an article based on the evaluation model. I'd forgotten about it by then," he laughs, "But I said sure and I wrote four articles, one on each level," says Kirkpatrick.
Still hanging on his words
It was a move that paid dividends – 40 years later and trainers and evaluators are still hanging on Kirkpatrick's words. In 1993 he published a book in which he profiled case studies to enable the readers to borrow concepts, forms and procedures as they wish. Dave Basarab provided one such case study showing how they had implemented the four levels at Motorola. He also wrote an introduction to the book which was called, 'Evaluating Training Programs: The Four Levels'. Book followed book and then Kirkpatrick's son, Jim, picked up the gauntlet with his own title, 'Transferring Learning To Behaviour'.
Simplicity is the key
So why does this model still hold currency? I asked Kirkpatrick, who told me the formula is very basic: "It's practical and simple." With that simplicity is the ability to translate it into organisations of different structures, languages and industries – it can be drip fed out whatever the culture or dynamics: "Every company has a concern about whether they are getting their money's worth from the training programmes. Trainers can evaluate levels one and two, but they need the support of managers to evaluate levels three and four," explains Kirkpatrick.
Over the years Kirkpatrick has witnessed a step change in the way organisations evaluate their own training: "When I wrote my first book most organisations were measuring reaction but very few were measuring learning. Over a period of time, more and more top executives - the jury that would approve the budget! - were asking for more evidence than just the number of training programmes that were offered and the reaction to those programmes. And more and more organisations were putting more pressure on the training department to justify their budget."
Since the original model was published Kirkpatrick has made some changes: "We decided that 'reaction sheets' weren't good enough, now we advocate getting people together in focus groups, give them reaction sheets and follow up on it." Kirkpatrick is also a stickler for staying with the rules: "No it's not good enough to skip level three, for example."
A legend and a guru
You get the impression that Kirkpatrick is amazed at the success that he is much trumpeted for. When the ASTD first called him a 'legend' and a 'guru' he laughed: "All I wanted to do was get my PhD done," he chuckles. It was a stroke of luck which he refers to as a 'miracle' and it has dominated his whole life: "Now practically everything I do is on evaluation. I have written a book on coaching but time after time they [organisations and conference organisers] still want me to do the same session on 'Evaluating Training Programmes: The Four Levels' because there are always new trainers attending the conferences and those who heard him before wanted to hear it again and pick up some practical ideas on levels three and four which they had missed the first time," says Kirkpatrick.
"When I wrote my first book most organisations were measuring reaction but very few were measuring learning. Over a period of time, more and more top executives were asking for more evidence than just the number of training programmes that were offered and the reaction to those programmes. "
A sense of humour...
Kirkpatrick is ever the practical man and he shares a story that conveys the good humour and ability to keep grounded that has kept him at the top: "I was conducting a session on 'Managing Change' at Dana Corporation in Michigan City, Indiana. Bill, the coordinator called me and asked if I was going to use the SMART approach. I wasn't quite sure what it stood for so I didn't want to say yes or no, so I humbly asked him to tell me what it stood for. He made the remark, 'I am surprised you didn't know'. So I told him I was going to use the PIE approach (I made it up just then and wanted to embarrass him a little.) After a pause he said 'I beg your pardon?' So I repeated it. And he said, 'Did you say the PIE approach?' So I said, 'I am surprised you aren't familiar with it.' So I explained that it stood for Practical, Interesting and Enjoyable. Both my son Jim and I use it today, except we have changed 'Interesting' to 'Interactive'," says Kirkpatrick.
But serious application
Another side shows Kirkpatrick to be very serious about the application of his model. As the father of evaluation, he sends the following stark warning to training departments about loitering around levels one and two. "If your jury is not asking for more evidence now, they will be and you better be ready with some evidence from 'evels three and four."
Whilst the world has changed around him, Kirkpatrick's model has remained solid, simple and easily digestible – a formula that has been translated across the world into Spanish, Polish and Chinese to name but a few. Despite its longevity the word continues to spread and will do so for the foreseeable future.
This article first appeared on TrainingZone in November 2008. Annie Hayes, MCIPD, is a former editor of our sister site HRZone.co.uk
