5 Responses

1. Benchmarks
   What were the benchmarks Jo? If I have some idea of what they were I might be able to suggest ways of tracking progress towards them.

2. Outline ideas
   I am just putting together a proposal to run a VRQ pilot, and have been looking at the same issue – how to evaluate. I have some ideas on what I want to achieve with the evaluation:
   ·Candidate’s feedback on programme
   ·Individual performance improvements
   ·Benefits/added value to organisation
   ·Feasibility of extending the pilot – we are aiming for maximum funding for the pilot, so cost benefit analysis will be important to us moving forward, as funding options change.

   I also have some thoughts on how I will go about this, but am still in the planning stage, with “Develop evaluation process” as a milestone on my project plan, so would be interested in others experiences and ideas.
   My initial thoughts are:
   ·Pre and post programme interviews/questionnaires with candidates
   ·Completion rates
   ·Training provider feedback
   ·Performance KPI’s
   ·HR stats- absence, attrition etc

   Equally, if you have any advice on other aspect of running a VRQ pilot, it would be gratefully received!

   Nicola

3. NVQ Benefits
   I suggest linking the evaluation to the benefits which this NVQ process was supposed to deliver. Hopefully (LOL) there is a Benefits Management process (ideally, as part of a larger Programme Management leviathan) which underlies the decision to go down the NVQ route. Find that & the evaluation should pretty much write itself.

   Good luck!

4. evaluation
   Hi Jo

   There is a generic evaluation tookit available. If you think it might be of interest, I can send you more details.

   Regards
   Jacquie Green
   IOL (UK) Limited
   t: 01884 821870
   jacquie@iol-uk.co.uk

5. What /how/who you ask depends on what you’re looking for
   Jo,
   There are six basic questions you should always ask: How have you applied the learning; What’s the impact of these efforts; What difference did this make in your unit/team; What’s the monetary value of the results; How much of this improvement was actually caused by the new skills/knowledge acquired; How confident are you in this estimate (0-100%). The last three help to isolate the programme effects from any other influences by asking the candidates and their line managers to estimate how much the new skills/knowledge have influenced the candidate’s work; whether they think the programme was value for money from a personal and business case and how confident they are in their estimates.

   Make use of data measures already there like the pilot objectives; the aims/objectives of the actual qualification even. Collection methods apart from questionnaires to consider are – a control group (to measure performance of the target group against an equivalent group who has yet to undertake the same qualification, turnover, employee satisfaction, organisational commitment etc.

   Know what your ‘before and after’ pictures are and what the data will inform. ‘Fully load’ costs if you need to show a monetary return on investment (ROI). Use measurements to assess change over a relevant timeline to illustrate impact of the whole process of learning/application. How does the analysis demonstrate that ‘change’? Remember to collect data at all evaluation levels for ROI study but keep lower level data simple.

   Only use credible sources to keep data results believable. Assume little/no improvement where no data and/or extreme/unsupported information received and omit from your report. Fully load costs – e.g. admin, travel, exam fees etc not just provider/venue costs for ROI. Intangible measures, those you can’t convert to money, can show valuable business impact and return on investment as well.

   Give me a ring if you want any further help.