We want to evaluate our expensive 5-day management course, to see how/if delegates have used the learning back at the workplace. We're thinking of telephone interviewing, in order to get a decent level of response. Has anyone any ideas of best questions? Does telephone interviewing work for this? Really interested to hear of experiences. Thanks.
5 Responses
Just a thought!
Hi Nicola
I’ve not done this with a 5 day management programme but I do get involved with it in relation to shorter and more targeted interventions.
One methodology that can work really well is to get each of the delegates to write a short essay/paper to reflect on what they were taught, what they learnt, what use that has been to them and where/how. Whilst this won’t give a numerical evaluation it will give a very personal and qualitative review of the of the outcomes of the training you have provided as well as providing a vehicle for the individuals to reflect and therefore revise.
Really telling would be if you were able to get the individuals’ bosses and direct reports to do the same…especially if they were asked to specifically identify differences before and after the training, a sort of 360 degree appraisal of the changes of behaviour and improvement in management skill.
I hope this helps
Rus Slater
http://www.coach-and-courses.com
In every case…
Hi Nicola, you need to go back to the needs of the learners to understand what they’ve learnt.
E.g. Before attending, Manager A had difficulty in Managing Difficult conversations. What have the quality of their return to work interviews been like? What effect has the course had in their ability to challenge underperformance? What is Manager A doing differently? What is the outcome of what Manager A is doing differently?
Peer and management review would be more effective here but are difficult to undertake if they haven’t been built into the programme at the design phase.
Ultimately, if you can’t measure what you’re doing, why are you doing it?
Evaluating after the event
Nicola
It is never easy trying to bolt on evaluation as an after-thought. I’d start by going back the the very premise of the course in the first place in particular, what was the definition of success (in business terms)? If that was never articulated, go back to your key sponsor or the management team and ask them to define the success criteria. You may need to help them to describe this in clear and unabiguous terms, to put it in the context of business outcomes, and, where possible, in a form that you can measure or get realistic evidence. Then you can decide what questions to ask, how to ask them and what performance data you may need to go alongside this. Telephone interviews are one possibility as long as you focus clearly on getting to the heart of the issues in your success criteria. If you just ask ‘how was it for you?’ – in a more sophisticated way, of course – that may give you the odd nugget or some information that may improve the content or running of the programme, but it may not give anything of real substance.
Alternatively, you can put the onus back on to the managers. I have run a follow-up day where each manager is given 10-15 mins to present to their colleagues what they have done differently, the impact on the organisation and, most crucially, they have to demonstrate that the value of the change they have made is at least equal to or greater than the cost of the course. Most managers don’t want to be the one that didn’t deliver a benefit and be shown up in front of their peers. As a consequence most managers get their finger out and really push to make some changes. And when they present they are not only justifying the value of the course, they are also sharing good practices with their colleagues which gives further learning and impetus. It takes a day, but I have found it works very well and sometimes reveals value that you might not have sought to achieve or ask about.
Hope that helps
Graham
Programme Close Webinars
In a previous role, I introduced so-called Programme Close Webinars that took place three months after the main workshop.
At this webinar, each attendee had to deliver a short five-minute presentation that answered the following three questions and then had to take questions from the other group members:
These webinars were always an upbeat end to the programme, as people usually had lots of positive things to share – both in terms of personal and business impact. The other group members would often add further support (and sometimes challenge) to their co-trainees.
For the trainers, hearing first-hand about the impact the learning had had provided a powerful (and sometimes moving) end to the programme.
Thank you!
Thanks for all of these excellent comments. They’ve given me plenty of things to think on!
N