No Image Available

Claire Savage

TrainingZone

Editor, news

googletag.cmd.push(function() { googletag.display(‘div-gpt-ad-1705321608055-0’); });

Diary: Metrics that don’t measure up

default-16x9

diaryTraining manager Josie Roberts is under pressure to invent feedback forms that show clear metrics to be plotted on graphs. But she wonders what will the feedback really tell her?


Every now and then the subject of evaluation raises its head. One of the first courses that I wrote and delivered here was on an area in which I have years of experience. The evaluation forms came back giving me positive scores and all was well.... Until the day someone scored me as “Below Expectations” on knowledge of my subject.

I have extensive industry experience, I pride myself on keeping up to date with what’s happening inside and outside the company so what was that all about? Had there been progress I should know about? Did this individual expect me to talk about some aspect I was unaware of? Was it something I knew but hadn’t included? Was it something I covered but delivered with less than conviction?

"It’s frustrating when the desire to measure overcomes the wish to be able to organically improve courses as feedback builds."
The score on its own was completely unhelpful. It told me that I had room to improve but gave no idea how. Being open to feedback and keen to continually improve, I found this somewhat frustrating. Therefore I changed the style of the forms so we have moved from a series of tick boxes to one overall score for the course and then specific questions. For example, “What will you take away and use?” – which I can check this against the course objectives. They might have loved the course and learned heaps but if they haven’t come away with what the course was designed to teach then we need to re-think.

We actively invite suggestions for improvement. For example, “What could the trainer have done that would have enhanced your learning?” We get a lot of “N/A” type responses but also plenty of comments. Specific feedback that we can take away and use. And we do.

We now have a very clear picture of what people are learning and what they are liking. We are also able to answer the question of how people would like to see our courses change. Keen to get better and better I love these forms.

Sadly people that like graphs and reports don’t. Other than overall grade you can’t generate a clear line reflecting the ongoing improvement and I am being asked to move to a more complex form that will return to the style of having general grades and not much space for specifics.

It’s frustrating when the desire to measure overcomes the wish to be able to organically improve courses as feedback builds. I’m not a detail person and struggle to understand the benefits of what is being proposed. I have had to overcome my first inclination to refuse point blank as I realise that doesn’t reflect well on my professionalism. I am working on a compromise.

In addition to the work focussed training I run a personal development group at lunchtimes. I enjoy being able to try more unusual training exercises and deliver on topics that I personally find interesting without having a show a specific work place benefit.

We cover things like personal goal setting, communications etc, so I believe the company benefits in the long-run too. It has a few incarnations running with the oldest group having been meeting for eighteen months now. They are very positive about the sessions and rejected my suggestion of reducing the frequency when I came to the end of topics I know about (that take roughly an hour to explain). So I have given myself an additional challenge of learning new and interesting things to share on a regular basis. One way of ensuring I keep up to date with what’s going on in the world of adult learning!

Josie Roberts is a pen name for a training manager in the private sector

Read more of Josie's columns:
Time waits for no-one
A rookie mistake
Taking targets
What to do about non-attenders?

One Response

  1. Feed back agreement
    A good article and one which helped clarify my thinking.
    I agree the tick box approach doesn’t work well for trainers (It does for participants as generally they want to mark the form and go). I wonder if staff outside of L&D appreciate the need for clear feed back as much as L&D demand.

    In my organisation I have worked on the feed back forms (on and off) for 2 years and we are more focused on open questions aiming to elicit specific feed back. “None”, “N/A” and “No” are still common answers though!

    I think the focus should move towards what the learner does back at work and this is where we will start to re-focus our efforts. I think we will have the “why do you want to know/need to know” response from participants when exploring this. Particularly as they are busy with their everyday jobs. But this is now the challenge for me!

No Image Available
Claire Savage

Editor, news

Read more from Claire Savage