Comparing learning to a course of drugs and training professionals to doctors, Paul Kearns questions how often the 'medication' is working. He argues that evaluation is a test of professionalism in L&D.
Training is a drug. Training managers are dealers, line managers pushers and users can become junkies. Then there are the consultants, always pushing the latest wonder drug - is NLP the new LSD? It is all about quick fixes and short-lived highs that last about as long as it takes to organise the next course. It’s a vicious cycle and it’s incredibly difficult to kick this nasty habit.
Learning is a drug. In the right, highly skilled hands learning managers become ‘GPs’, they prescribe the right courses of treatment that are most appropriate for line managers and employees. Here the consultants are exactly that - consummate experts to whom the GP can make referrals when needed. It’s about developing positive habits, a virtuous cycle that, over time, ensures people are in the rudest of health.
Reading the debates on TrainingZone recently, on the merits or otherwise of using ROI studies and calculations:
Any impartial observer could be forgiven for thinking that this is a highly emotional subject that quickly produces polarised views. The two scenarios painted above also represent two entirely opposing perspectives on what training/learning looks and feels like today in most modern organizations. So which is the closest match to your own and what really distinguishes one from the other?
The answer is surprisingly simple, in theory. The theory in question is evaluation and for ‘trainers’ it is an unwanted burden and a chore. For learning professionals it is a mark of their professional pride. ‘Training’ organisations do not subscribe to the concept, theory or practice of proper evaluation. Learning organisations see evaluation, and yes ROI when appropriate, as integral to a holistic, learning culture.
Defensive
Unfortunately though, words like ‘evaluation’, ‘return on investment’ and even ‘measurement’ appear to offend the frail sensibilities of trainers who think training, by definition, must be good and so puts them on the defensive. So instead let’s refer to questions such as - How successful would I like learning to be?; What might successful learning look like?; Who should determine what is success?; Is it worth bothering to check; and even - Will this help or hinder the learning process?
The key difference between ‘training’ and learning is that one is an input and the other is an output and all organisations, profit or non-profit, exist to produce outputs and hopefully beneficial outcomes. In practice, ‘drug-addled’ trainers measure their success by the number of fixes, sorry, the number of people who attend courses. Managers, who never seem to tire of quick fixes, reinforce this irrational behaviour. Only when they both come off the drugs do they realise where they have been going wrong. On occasions, just like those who give up smoking, some even become the fiercest critics of those who still indulge. Polarisation seems to be inevitable in this emotive subject; there is no happy medium.
Holistic
Learning ‘GPs’ however, are only interested in outputs. They judge success simply by observing the health and well-being of their patients. The best GPs will take a holistic view and undertake a rigorous and extensive diagnosis, using their judgement, to get to the root causes of the problems. They will never prescribe training until the full diagnosis is complete. They will take time to check behaviours, willingness to change, the organizational environment and many, many other contributory factors. Moreover, any training prescribed will always be just one part of an overall solution. The most professional GPs will also do something else - they will ask you to come back in a week to check and make sure the treatment is working. The whole of the medical profession is predicated on a principle of evaluation; a closed loop feedback system that starts with diagnosis and ends when the patient’s health is checked and hopefully assured.
Now, if you are a learning professional who finds yourself stranded in an organisational ‘drug den’ and you want to get out, what do you do? Some trainers have argued that it doesn’t matter too much what they do as long as they keep their ‘customers’ happy. Adopting this minimalist, survival stance might keep you alive in the short term but it won’t do anything to aid your own, professional development and will contribute nothing to organisational health.
Even if trainers want to change there are two things stopping them. First, the conventional theories on evaluation do not work. The vast majority of trainers use the four-level (Kirkpatrick) model of reaction, testing, transfer and results. But this only ever did a limited job on one-off, training courses, it was never designed as part of an iterative, organisational, learning model. Second, even if you try to use a better model your ‘crack addict’ managers and ‘junkie’ employees, who can’t get enough of courses on their CV, will not want to come off with you. So you will have to manage their expectations and this is where ROI can be used as therapy.
ROI
Based on my own experience, over 15 years, I can see why Don Taylor suggested that senior managers are not the ones asking for ROI studies. On the rare occasions that they do (e.g. the CEO of HSBC asked his head of training some years ago what the bank was getting for the £x million they were spending every year) it is usually a challenge to the training team to justify their existence rather than a challenge to themselves. If this CEO, a banker, didn’t already know the answer then why had he signed off the training budget of £x million every year?
Very rarely does anyone see ROI as a positive, constructive, learning-enabling, experience probably because they follow Jack Phillips five-level model, which puts ROI at the end, after the event, not at the beginning. In doing so, even supposedly intelligent people can so easily make themselves look pretty stupid. CEO’s need to understand not only that training should aim to get an ROI but also the best return possible. ROI is about focus and commitment to learning from the top.
So of course ROI matters but it has to matter to you first, if you want to be a true learning professional. Look what happens when it doesn’t. Four years ago I was invited by the BBC, an existing client, to help them evaluate a ‘leadership programme’ that was costing £35 million for 7,000 managers.
Tough choices
Those of you who profess to be learning professionals have to stand up and be counted, like Garry Platt. You have some very tough, personal choices to make and how you make them defines the nature of your ‘professionalism’. Those of us fortunate enough to work with many clients can choose who we work with.
It might not be so easy for training managers to jump ship but being totally compliant with line managers’ wishes is neither professional nor likely to lead to respect. Your own personal integrity and reputation will be made or broken on the back of your conviction that the theories, models and methods you use are not only right but the best available. Garry constructed an impassioned yet objective and lucid argument to rebut Don’s take on the subject of ROI.
In my opinion, Phillips model is not based on a robust theory, it is not best practice in the UK (even though promoted by the ASTD) and it very unpopular to boot precisely because it can be bureaucratic, time consuming, chore and may fail to convince CEO’s.
Being a learning professional should have the same status as the best in the medical profession but if we don’t take our own work seriously enough, through evaluation, why should anyone take us seriously?
I have been called many things due to my dogged persistence in asking evaluation and ROI questions. Many trainers do not like this approach because it means they have to raise their game significantly and many managers do not like being asked awkward questions. I do not ever remember, however, anyone questioning my integrity or professionalism for doing so. Those who only want to keep the line happy will achieve neither.
About the author: Paul Kearns is the author of Evaluating the ROI from Learning (CIPD) and his latest book The Value Motive (Wiley) shows how to link human values to hard monetary value. Anyone interested in joining Paul's Baseline Model Users Group should contact paulkearns@blueyonder.co.uk for further information.
