5 Responses

1. No simple solution
   This is always a tricky question. ‘Happy sheets’ do not give the type of arguments that you are looking for. My approach has always been to speak to each of the senior managers individually and ascertain what he/she sees as the desirable outcomes and changes from the training. You also need to try to assess the ‘cost’ of the problem (re-work, scrap, extended leadtime, overdues, complaints, etc.) Once you have this information you can then look at what you need to address and how to present your data and results. Good luck. Jenny (CompAir)

2. Get the learners to do it…
   Seriously … Build time into the programme for the learners to set their individual learning goals and then to work out a means of measuring their achievement of them (SMART objectives work well). This should also include examples of how they have transferred (or not) the learning to their work.
   This will: identify if the learning programme achieved their aims (not yours); whether it was practical and applicable; whether the organisation assisted or prevented the learning being used.
   All you need to organise is a mechanism to feed this back to the trainers and the line managers involved.

3. Effectiveness of training etc
   My approach is to set up the happy sheet at the beginning… though not the usual tick sheet.

   2 aspects –
   What are the OUTCOMES as opposed to outputs of the intervention – get them sorted – robustly and rigorously.

   Have some form of analogue tracking device against various criteria relevant to the intervention where it is more than a one-off sheep dip. (or even when it is a day course)

   This way you can track any strategic drift rather than find out at the end you’ve got lost in a fog.

   Steve Gorton
   Enabling Development

4. Look at a range of approaches
   You need to be looking at the purpose of the evaluation and the criteria you use to identify success.

   If you are looking for more funding you need to consider different levels of evaluation. There are a number of models but the following is a reasonable summary:

   Reactions – were participants satisfied – did they get what they felt they needed? The criteria is subjective and relates to learners perceptions. Methods include questionnaires, verbal feedback, games etc.

   Learning – did the participants achieve the objectives of the programme. The criteria is objective and this can be assessed at the end of the programme. Methods include practical, written or verbal testing in the form of case studies, exercises or quizes.

   Application to the effectiveness of the participant’s work – A review by the participant hopefully in discussion with their line manager, or by the manager, or by some objective measure, some time after the training event will enable you to identify whether learning has transfered to work. Easily measured in some industries less easy in others.

   This is more useful in making a business case for training but is more difficult to do because it involves the line managers who may feel this is an extra and unnecessary pressure. It works well if managers have identified the need for training and the effect training should have on the outcomes of their part of the organisation.

   Appilcation to the effectiveness of the team/department – This is a development of the above and requires a review of the effect of the newly developed skills and knowledge on the effectiveness of the whole team. Managers need to be involved in the TNA and Review process.

   Finally there is the level of ultimate value – cost/ benefit analysis. Some one else has asked the question and received a useful answer so I’ll leave it there!

5. Impact Evaluation by building evaluation criteria into your trai
   I think the nearest complete solution is to build your training management system around what you are trying to achieve.

   In brief- you take a recognised four level evaluation system such as Kirkpatrick – reverse the order to start with results or benefits to the organisation and decide at the outset, before any training is designed or purchased what you are seeking to achieve. Next step is to decide what behaviour you want to change ie ‘what can’t your people do now which you want them to do’ You then have to decide what evidence you would accept as being proof of behaviour change and benefits to the organisation. These become your impact measures. The system is an excellent way of involving operational managers in training.

   To go through the system in detail would take some time, but please ring me on 0113 2676845 and I would be happy to talk you through it

   Tony Morris