If you've been involved in commissioning or delivering training programmes, then you will hopefully have thought about their evaluation. Has the training made a difference? Did the participants find it useful? Did they learn anything? What was the return on investment (ROI)?

When we evaluate training solutions we have to do more than answer the questions above. In fact, there are other even more important questions:

What was it in particular that worked and how? What impact was created? Did it work for all participants or just some? What would have made it work even better? How do we know that it was the training that created the impact and not something else?

Traditional forms of evaluation, based purely on statistical analysis, can make statements along the lines of: x % of participants considered the training a good use of their time; y % of trainees were able to demonstrate that they had learned the key behaviours; or the KPI measured showed a z % increase for those trained compared with a control group.

Critically, these forms of evaluation cannot state for certain that it was the training that caused the effect – only that it correlated with the effect. They are interesting conclusions but in our view evaluation should deliver more than this. We all know that correlation isn’t necessarily causation.

We want our clients to get real value from evaluation, and in a way that can inform the next step of their strategy. In order to meet this need we apply Rob Brinkerhoff's Success Case Method.

Identifying groups

Firstly, let’s step back a bit and think about what happens after a training session. There are broadly three types of trainees:

A.  Those who get the training, use it and create a business impact as a result;

B.  Those who get the training but then either don’t use it or try it but don’t get a result and so stop using it;  

C.  Those who don’t get it at all.

All results, and therefore the ROI, will come from group A. But what happens to the other two groups? If we learn to understand that, then we could easily make a huge difference to business results. If we could move just a few of those in groups B and C into group A, then the ROI would increase significantly.

Brinkerhoff's Success Case Method uses a two-step approach to answer these questions. Step one identifies which of the three groups each participant falls into. This usually takes the form of a simple online questionnaire. Step two is to interview a representative sample of each group to understand in detail the key questions relevant to each of them.

Post-training embedding and coaching are as important in terms of impact as the workshop itself

For group A – what exactly did they do? How did the training help them? How exactly did their use of the training create a business impact and what was that impact, quantitatively and qualitatively?

For groups B and C – what was it that prevented them from using the training or creating the impact that group A had achieved? Typically the answers to this question tend to fall into one or more of three categories:

1. Either there was something about the person (e.g. they were not ready for the training or it was inappropriate for them) or there was an issue with the training itself;
2. The delivery method was ineffective or was not at the depth needed, or the content did not address the desired impact; or finally – and most commonly
3. There was something in their environment that inhibited their ability to make use of what they had learned (e.g. the pressure of the day job over-rode the learning, their manager was unsupportive or the systems and process they were required to use got in the way of applying what they had learned).

When we have the answers to these questions, it leads to a clear set of initiatives for the organisation to take that will improve future trainings and remove the barriers to creating the impact that the business is seeking.

Our evaluations have consistently shown that post-training embedding and coaching are as important in terms of impact as the workshop itself, and it helps participants to apply the formal training to their own real-world context.

The power of coaching

A recent evaluation of a sales training programme for a major client demonstrated that our focus on key behaviours was effective in producing the desired results. The programme involved discovery, workshops and post-workshop support by way of coaching and web-based follow-up.

The evaluation found that the key determinant for creating impact was the post-workshop embedding and support delivered through personal coaching. We found that participants who had not taken up the coaching offered created significantly less business impact than their colleagues who had. This finding enabled the client to adjust future rollouts of the programme to ensure even greater impact.

Many companies tend to race into training programmes without figuring out what it is that they are really trying to change, which makes evaluation challenging at best. Our experience shows that those kinds of companies are more likely to be susceptible to the next hot subject and waste time, resources and energy that could be better used refocusing on their original objective.

R A C E

While organisations understandably focus on outputs such as pipeline growth, pipeline speed, revenue increase, we believe that inputs are just as important.

We believe that results are a function of attitude, competence and execution. The transformational formula R=A+C+E^® (developed by Whitten & Roy Partnership’s co-founder Scott Roy) helps to separate sales inputs from outputs while putting an appropriate focus on each aspect. It also helps our clients to identify what it is that really works to create the strategic and sustainable outputs that they are looking for.

Results are a function of attitude, competence and execution

We use the R=A+C+E^® formula to identify the inputs most important to the team’s sales success: for example, how easily somebody learns a new behavior (an attitude input), how well a salesperson listens to customers in order to understand their needs before offering a service or product (a competence input), or how effective a salesperson is in identifying and avoiding activities that squander their selling time (an execution input).

Our evaluation process considers all of these parameters carefully and provides an answer not only to the question “did it work?” but also to the questions of “how did it work?” and “what can we do to make it better?”

In today’s busy world it is easy for organisations to rush to solutions without first clearly defining the issue that they are trying to resolve. At the end of the day, what matters most is to work together to uncover the real problems and deploy the most appropriate solutions so that clients can make the very best decisions for their businesses.