I have been tasked to create an online form to be used to evaluate the training undertaken by staff in my organisation. The brief I have been given is that it must incorporate Kirkpatrick's 4 levels of evaluation.
I appreciate that the form needs to be tailored to the specific item being evaluated (in this case training)and for the particular organisation, however, as I need to have this up and ready by Monday 25th February I hope that someone in the community may have an example that I could adapt.
In addition it has been suggested that the results should be easily transferred into a spreadsheet, preferably automatically. Quite a heavy task I guess.
Any assistance would be deeply appreciated.
Mike Styring
7 Responses
Four levels form
Mike
I think you have been given a duff brief. I wonder just how much the person who is commissioning this knows about evaluation – for a little knowledge is a dangerous thing. This sounds like evaluation is an afterthought and ‘let’s just do something’. I could contrive such an online form in about 10 mins; it is not difficult, just pointless.
I’d suggest you go back to them and explain that you need to know what you are trying to achieve and then work out the best way to achieve it. Kirkpatrick has his critics but his 4 levels is a popular framework for categorising some different aspects of evaluation. If, for arguments sake, you do go down that road, you will find you need to use different tactics to gather meaningful data at each level.
If you need to do something – however pointless – to satisfy your boss while you get chance to consider better options then just ask 4 questions and use a rating scale response (and maybe an open response option after each):
1. How good was the training? 1= Very good, 5 = very poor. (Please explain your response…).
2. To what extent were the learning objectives met? 1 = Fully, 5 = not at all. (Please state what you learnt…).
3. How much difference has this made to your work? 1 = Significant, 5 = None. (Please give details…).
4. What impact has this had on the business? 1 = Significant, 5 = None. (Please give examples or evidence…).
(please no comments fellow TrainingZoners about my hurriedly crude semantic differential rating scale!)
I am definitely not advocating this approach but because of your deadline I thought you might welcome something that will give you a breathing space. I should add that when you come to collate the data, if you get a sufficient response rate, you may just find something in it that is useful. But you may also find that the data is incomplete, unreliable and utterly misleading (so don’t rely on it).
Hope this helps
Graham
can I open a book?
Dear Mike
Notwithstanding Grahams excellent response to your issue (My only suggestion for changing Graham’s comment is replace “duff brief” with “poisoned Chalice”) can I propose opening a book on this?
My bet is that the commissioning manager is an accountant with limited HR expertise….any other bets?
Evaluation or Validation
An accountant – no worse a person that has read a book on training and thinks all training is ‘easy’…..
Sorry Mike – but apart from Grahams excellent ‘get out of jail’ solution you cannot do EVALUATION via web forms – VALIDATION yes.. Kirkpatrick… sorry wrong end of the training cycle.
Evaluation criteria needs to be defined BEFORE the training brief. In effect the evaluation is the TNA – but with a value on it…
There has been a lot of debate in this and many other forums about the practicalities of levels 3 & 4, what ever the outcome of those debates level 4 takes quite some time and is arguably not for single small scale interventions. And defiantly not for something that needs to be delivered next week.
I would go back to the person that has set this task as ask…
” please show me an example of where this was done previously to give me a benchmark of your expectations as I am having difficulty identifying appropriate level 3 & 4 factors”… sit back and watch them squirm or post the reply here for more feedback!!!
Good luck
Mike
http://www.rapidbi.com/bir
Kirkpatrick is obsolete
See a much simpler and more successful approach that has made Kirkpatrick obsolete https://www.trainingzone.co.uk/cgi-bin/item.cgi?id=122638
Evaluation – The Six Sigma Way
Now I realise that Mike has been tasked to use the Kirkpatrick model, but I thought I’d just mention this book I encountered a while ago, that looks at the development and evaluation of training programmes from the standpoint of Six Sigma.
I found it an interesting model and I hope I don’t oversimplify things by saying that basically it’s through the close partnership you have with the project sponsors at the start of the initiative that you arrive at your evaluation criteria and measures. And what I really like about this model is that ultimately and pragmatically, the criteria are whatever work best for the sponsors.
http://www.amazon.co.uk/gp/product/0787985333/026-4377281-4144418?ie=UTF8&tag=tech4learning-21&linkCode=xm2&creativeASIN=0787985333
some good advice already
There has been some good advice posted already. If you do want an online system that you could use and adapt and dump into excel you may want to consider using survey monkey. It allows you to create questionnaires for any use which you can then send via email link for people to complete. The data can then be analysed to whatever level you want including dumping to excel. A basic account is free but this has severe limitations. An unlimited account costs around £100 a year. Sign up for a free account, try it and see. I use it for sending out evaluation of learning events and found that it works well.
Did you get there?
Hi Mike
Got the evaluations sheet – did you ever find a system to collate and report on the findings?