Introduction
In the current economic environment where we are being asked to produce the same standards, with tightening budgets and less staff, we now need to look at our training methods and how to make them more effective.
Donald Kirkpatrick developed his 4 level evaluation model in the late 1950s and this is still used worldwide.
To make training worthwhile and productive organisations should introduce a structured strategy to training and introduce a plan of how to get the most out of the training on offer to all staff.
Kirkpatrick’s model was based on 4 levels of evaluation
Level 1 – Reaction
Level 2 – Learning
Level 3 – Behaviour
Level 4 – Results
This report will show an alternative method of evaluation, based on Kirkpatrick’s Model which is easy to introduce however it does mean that some Line Managers will have a little bit more work to do, but this is something Line Managers should be doing at the moment anyway.
In this growing climate of budget restraints staff are being asked to take on more responsibility and with that there will always be some form of training involved to up skill staff to carry out the duties of a new role. Whether it is observation, desk taught, e-Learning or face to face training delivery, organisations need to make sure that what we are asking or staff to do has a business benefit. In most instances, a staff member is selecting to do a job and then told when their training will be scheduled. Organisations now need to work a lot smarter and more efficiently to make sure that staff are properly trained to carry out the roles they are doing.
Pre-event Briefing:
Evaluation should start before a delegate has even applied to come on a training event.
If a member of staff is selected to carry out a specific role and some form of training is required they should have a briefing with their Line Manager to discuss what the role is and what it involves. At this meeting the Line Manager should lay out what is expected of the person and what form the training will take.
They should discuss what, if any, pre-event preparation work is required and make sure that the person is given the time to carry this out.
If this briefing takes place then the person, when they attend a training event knows why they are coming and what is expected of them. If a delegate knows why they are coming and what is expected of them then the training will be of more use to the delegate.
Attending the Event:
Trainers must make sure that at the start of every event they go over the aims and objectives of the event and discuss with delegates why they are there and make sure they have had their pre-event briefing with their Line Manager. If there is pre-event preparatory work this must be raised and explain how this work will assist them in the event they are attending and that if this work is not done how it will affect their understanding of the event and if necessary they may be turned away from the event if the preparatory work is essential to the training.
“A chain of evidence shows the organizational value of the entire business partnership effort through quantitative and qualitative data that sequentially connect the four levels and show the ultimate contribution of learning and reinforcement to the business”.
Level 1 – Reaction
This has in the past been all about the satisfaction of the delegate with the training they have received. Commonly called the “Happy Sheet” it was given out to delegates at the end of training and delegates quickly fill out the form before running out the door.
Using the Kirkpatrick New World Model this should now shift in emphasis to:
- The degree in which participants are actively involved in and contributing to the learning experience, and
- The degree in which the delegates will have the opportunity to use or apply their new found skills or abilities.
In this the Training Team will have to look at the end of event evaluation and shift the emphasis from Trainer-Centered questions to Learner-Centered questions.
- Was the delegate able to relate to the learning objectives
- Were they appropriately challenged by the material/tasks
- Was the course material essential to their role
- Were they able to immediately apply their knowledge to the role they will be doing.
- Was their learning enhanced by the Trainers experience/knowledge
- Was it easy for them to be actively involved in the session and was it relevant to their role.
- Were they comfortable with the pace of the event
- Did they find the atmosphere comfortable and contusive to their learning?
This is now a big change in the way we look at evaluation within our organisation and it should reflect on the delegate themselves (Delegate Led) rather than the trainer (Trainer Led).
Level 2 – Learning
This stage of evaluation should be done by the delegates Line Manager as soon as possible after they have attended any event. This was originally about what knowledge and skills they have acquired but this should also now include information about the confidence and commitment to their role they now have after the event.
This should take the form of a Post Event Evaluation and should include a discussion about what new skills or abilities they have learnt from the event and form a plan on how they will put these new skills and abilities into practice, how confident they are about doing this and gain a commitment for the delegate as to how and when they will put this into practice.
This should be considered as part of the delegates Continual Professional Development (CPD) and it should be noted in their Personal Development Plan (PDP) and their Personal Learning Plan (PLP) and should reflect in their own Objectives.
These discussions should also be noted in their In-Year and End of Year Reviews.
Level 3 – Behaviour
This stage is carried out by the Line Manager by conducting a briefing with them and noting what they have done since the new learning they have received and how they have put it into practice. This should be done 3-6 months after the training.
This is to look at how the delegate has applied the new skills or abilities into their role and how their reaction to the training has affected their role.
Level 4 – Results
This is the area that most often gets forgotten by most organisations as it looks at the operational impact on the organisation. Most departments find this a difficult stage to quantify and therefore do nothing about it.
At this stage the Line Manager should look at the results that are currently being achieved by their staff member and compare it with the results they were achieving prior to any form of training. This will track any positive (or negative) changes and results that are now being achieved compared to previous results. These figures can then be used by both the Line Manager and the staff member as evidence for their monthly and annual reviews and performance markings.
Conclusion
Currently the only form of evaluation for any training event that is carried out is the end of event evaluation (Level 1) which is sent, or given, to delegates after they have attended an event.
This evaluation is not mandatory and only a small percentage of delegates do not complete this. The evaluation forms that are returned mainly show a good to very good standard to the questions asked but these are Trainer-Centred and the results are slanted that way. In most instances there tends to be no action done with the results apart from the relevant trainer that delivered the event reads what has come back about them but unless there is something glaringly obvious, they take no action on the results.
This is a meaningless exercise and, as no action is done with them, there is no merit in sending them out to delegates apart from the illusion that we are interested in what they have to say about the event.
As most organisations have a reducing budget, all staff will have to work smarter and ensure that they are coming on events that are the right ones for them and the business. By changing to the recommendations highlighted below this should reduce the number of delegates who come on events not knowing why they are coming or what they will do with this new knowledge. Currently organisations have a history of people who come on events that fall into 3 categories; The Learner, The Prisoner, and The Holidaymaker.
The Learner tends to be delegates who have looked at the event description and decided that to do the job they are currently doing (or about to do) this is the most relevant event and they should learn what they need by attending this event.
The Prisoner tends to be the minority of delegates and they are the ones that are only coming along as they have been told to attend. They do not know what the event involves and do not know why they are attending or how they will put these new skills into practice. They have not had a formal pre-course briefing and only attend because they have been told to do so.
The Holiday maker is the delegate that is normally unsure of what the event is about or what use it will be to them but, it is a day out of the office.
The recommendations proposed will help to focus delegates to come onto events they require as part of their role, rather than see this as a day out the office.
Recommendations
My recommendation would be a vast overhaul of the evaluation strategy for any organisation.
Starting with the entry criteria of the events on “Events List”. These should be enhanced and include some form of Pre-event work or reading to be done by the delegates before they attend an event, thus preparing them for their event. This should also include a tick box on the application to say that they do understand that they will be required to undertake any pre-event work and arrange a pre-event meeting with their Line Manager.
For Delegates themselves, if they apply to come on an event, it MUST be forwarded to their Line Manager for approval prior to being submitted as a delegate, this is already done in some organisations, but not always if the training is in-house. This way Line Managers will see if there is any pre-event work to be done and will trigger an opportunity to do a pre-event meeting with the delegate to make sure that this is the correct event for them and they are attending as it is required for the role they are doing.
Level 1 evaluation - Reaction –
Produce a paper version of the event evaluation form and issue it at the start of each event. The evaluation form should be tailored to each and every event and the questions asked should be relevant for that event. The form should be “Learner-Centred” and be split up so that there are separate questions for each session of the event. That way delegates can be given a 5 minute reflection period at the end of each session and an opportunity to complete each section as the day progresses.
If event evaluations are issued at the end of each day delegates will rush through them in a hurry to get away and normally results from these are seen as “Happy Sheets” as there is normally little or no constructive criticism included on the forms. By splitting them up into different sessions, issuing them at the start, and giving reflection time at the end of each session there is a better likelihood of delegates giving more honest opinions. If need be, these evaluation sheets can then be scanned, and a copy e-mailed to the delegate and their Line Manager so that they can be used as a basis for Level 2 Evaluation.
Level 2 Evaluation - Learning –
This would bring a more structured approach to learning by asking line managers to conduct a Post-Event briefing as soon as possible after the event with their own staff member it gives the staff member an opportunity to take control of their own Learning and Development and be able to discuss with their Line Manager what they will do with the new skills/abilities they now have and how they will put that learning into practice. If the staff member is not a forward thinking person who does not want to push themselves then this will be an opportunity for the Line Manager to use this to discuss what they will be doing next and will give an opportunity to include these new skills as an objective in their Annual/Monthly Performance reviews.
The next 2 stages are more difficult to evaluate and will require input and commitment from Line Managers and staff members themselves.
Level 3 Evaluation – Behaviour -
This stage would require Line Managers to have a follow up meeting with their staff about 3 months after the event to discuss with them how they have put their learning/skills into practice, to discuss what has gone well and what has not gone so well and what they plan to do next, and action plan their proposed actions.
This is also an opportunity for staff to show how they are using the skills and planning their own development to identify any areas for improvement and to look at how they will show future improvement. If they have identified areas for improvement they can also look into what events or learning opportunities they may use to make these improvements.
If (as some staff are) they are not interested in career development for themselves this is an opportunity for their Line Manager to discuss what they feel should be the next steps for that staff member.
Level 4 Evaluation – Results –
This stage should be done 6 months to one year after the training.
This is the most rewarding, or the most frustrating, part of the process. This is where a Line Manager can look over the staff members results from the last year and look to see if there are any trends of improvements. Using reporting tools to see if they are using the skills they have learnt more efficiently.
By looking at stats and with regular input from the staff member this is where organisations can see how much the Return of Investment (RoI) the training has had on the staff members. If there has been a change in productivity by staff members this will be reflected in the objectives that have been set and noted in the end of year reviews and overall markings.
Staff who take control of their learning and development will tend to have better marking and performance as opposed to ones who come on training events as either a Prisoner or Holidaymaker, but Prisoners and Holidaymakers can be made aware of the benefits of the training on offer and how they can help their own development and help convert them to Learners.
