In a previous blog I outlined the four stages in Kirkpatrick’s model of evaluation:

1 Learner’s reaction to the learning (its relevance, level of interest and involvement, and perceived value and transferability to the workplace).
2 Learning results (did learning actually take place?).
3 Behaviour change (did performance actually improve in the desired way?)
4 Business impact (has this improved performance led to measurable improvements in organisational performance?).

One of the key objectives for any trainer has to be ensuring that learning took place, and that means some form of assessment. But assessment can do more than measure if learning took place, although that’s important enough – it can also enable learning and transfer learning to the workplace. I’ll try to explain what I mean by these three and, in the process, establish some criteria that trainers should use to determine the effectiveness of any assessment they develop or use.

Measuring learning is the commonest function of assessment. Did learners actually learn what they were supposed to learn? Last summer my son was in the midst of his A Levels. He worked out what he needed to learn to be able to answer the sort of questions he was likely to be asked in order to get the grades that he needed for his university of choice. He did this well enough to get the grades he needed (and create a couple of proud parents, as a useful side benefit.)

But that’s not all the assessment did – as well as showing that he had learnt what he was supposed to learn, he also found that the revision he had to do in preparation helped him to understand what he had been taught. However, that wasn’t the point of the assessment, and it wasn’t the assessment itself that did it, but the preparation for assessment. A really well designed assessment should enable learning to take place, but formal exams aren’t a particularly good vehicle for doing that. Projects and other work based assessment activities, and diaries, learning logs and reflective reviews all have learning built into the assessment process.

Let’s look at one example. Team leaders who have been taught about continuous improvement and the tools and techniques needed to do it have only learnt about the topic (what is sometimes called procedural knowledge, or knowledge that). What team leaders need to do is to apply what has been learnt by leading their teams through continuous improvement activities, using the tools and techniques in the process. In doing this they will learn how to do it (what’s sometimes called procedural knowledge).

A report on the process, or observation by an assessor (a rather expensive alternative), provide opportunities for the learners to demonstrate what they have learnt and how they can continue to develop and improve. The assessment has made it possible to measure the learning that has taken place, has enabled further learning to take place, and has helped to transfer learning to the workplace. What’s more, by building in an action plan, it can enable further, continuous improvement to take place.

This is not a plea for learning to be accredited, but to suggest that assessment needs to be seen as being a part of the learning process and something that can be used to enable learning transfer (Kirkpatrick’s third level) as well just measuring learning has taken place (Kirkpatrick’s second level).

The beauty of the new Qualification and Credit Framework (in England, Wales and Northern Ireland) is that small units of assessed learning can be accredited, and credit accumulated towards a qualification. This makes it far easier for training to be accredited without needing to be a substantial programme. However, it’s only worth doing if the assessment process is fit for purpose. That means asking:

Does it measure learning? (If it doesn’t do that, then it really isn’t worth doling.)
Does it enable learning? (Assessment activity can deepen, extend or reinforce learning, sometimes all three.)
Does it enable learning transfer? (Learning transfer means that what has been learnt is used in the workplace to improve the person’s performance and, ideally, organisational performance.)

These three criteria are quite demanding but should form the basis for any judgement about the appropriateness of assessment of learning in any development activity.

Oh, and there’s a fourth one – is it cost effective? Assessment can be a time-consuming activity, and time is money. This doesn’t mean looking for the cheapest way to do the assessment – after all, online multiple choice tests are a very cheap way of assessing large numbers of people, but they only measure learning. (The high set up costs of truly effective multiple choice tests means that they are best used with large numbers.) To have assessment that enables further learning and transfer into practice demands more sophisticated approaches. By contrast, workplace observation is good at assessing learning transfer, but poor at measuring learning, not ideal in enabling learning unless combined with workplace coaching (which raises possible role conflicts), and relatively expensive.

The sort of work based assessment that I’ve outlined above is a good compromise between these competing factors. It can satisfy all three of the criteria above, and do so relatively cost effectively. However, to be truly effective you need the buy in from line managers that I was talking about last month. After all, one of the most depressing things any trainer can hear is “My manager won’t let me do it!”
David Pardey