Learning Technologies Conference plenary speaker and expert on measurement and evaluation Dr Jack Phillips, talks to TrainingZONE about the ROI methodology.
TrainingZONE: While many training professionals say that they are coming under pressure to prove the ROI of learning and development, a smaller proportion seem to actually do it. Why do you think that is?
Jack Phillips: Barriers to implementing a measurement and evaluation process include fear of negative results which will threaten a programme or department’s budget (if not existence); misunderstanding the purpose of the process as personal rather than programme evaluation; using evaluation as a post-programme activity rather than planning for evaluation at a programme’s design phase; lack of knowledge of measurement and evaluation – how to do it – jeopardising credibility for the results; and lack of support by key stakeholders who need critical communications to support their buy-in to the process.
TrainingZONE: There's a sense that ROI best kept for larger and more expensive training programmes. Is that fair?
Jack Phillips: First, it must be used selectively – to conserve resources. Reserving evaluation at Level 5, ROI, is not only fair, but it is also the wiser business practice. Criteria for selecting solutions and programs for Level 4, Business Impact, and Level 5, ROI, are life cycle of the solution, linkage of the solution to organisational goals and strategic objectives, management’s interest in the evaluation, cost of the solution, visibility of the solution, and size of the target audience. Smaller, inexpensive programs may be evaluated at lower levels – providing data that will contribute value for planning and decision-making.
TrainingZONE: At the most basic level, what in your opinion is the minimum that a training professional be doing with regard to evaluation?
Jack Phillips: At the very least, all programmes should be evaluated for reaction, satisfaction, and planned action (Level 1). About 40-60% of programmes should be evaluated at Level 2, Learning; and 20-30% at Level 3, Application. Ideally, evaluations should move to 10-20% at Level 4, Business Impact and 5% for Level 5, ROI. One critical piece is that evaluation be planned for during a program’s design and development phase.
TrainingZONE: Why do you feel that employers are putting pressure on L&D departments to prove the value of their training?
Jack Phillips: Because L&D is expensive, time consuming, and often disruptive for workflow, managers want to see value for their expenditures of time and money. The growing demand for accountability in all fields and all sectors has led to the “show me the money” trend moving from capital expenditures through materials acquisition and now into the service aspects of organisational life. L&D executives have failed in the past to show credible value, although it has been promised many times.
TrainingZONE: While there is more pressure to conduct evaluation, to what extent do you think that training professionals are responding to that challenge?
Jack Phillips: We are seeing a positive response in tremendous numbers of organisations – corporate, governmental, and non-profit – all around the world. As the push for accountability grows and as there are increasing numbers of case studies and success stories, training professionals are seeking credible and sustainable methods for proving the value of their programs and solutions and for communicating their value to executives in the language that executives often want to hear – money and ROI.
TrainingZONE: With the threat of outsourcing, do you feel that strong training evaluation and metrics will help corporate L&D departments demonstrate their value – and keep their jobs?
Jack Phillips: Yes. Executives want companies that are financially sound and profitable and they want to know the value of all departments within the company in terms that are meaningful for corporate planning and budgeting. When L&D departments can demonstrate the value – the business impact and ROI – of their programs, they demonstrate their value to the company and have better chances of surviving budget cuts, downsizing, and outsourcing.
TrainingZONE: Is there one organisation that has used this approach to evaluation that you would single out as a beacon of how to make ROI work and why?
Jack Phillips: Over 2000 organisations (and more than half of the Fortune 500 companies) are enjoying great success using the ROI Methodology for measurement and evaluation. Over 200 are considered best practice organisations – having used the methodology for over five years. Singling out one or even a handful as particularly exemplary would be unfair to the rest since all are using this methodology effectively to demonstrate the value of programmes and solutions.
TrainingZONE: Looking into the L&D crystal ball, what are your predictions for 2006?
Jack Phillips: I expect that in this year we will see increasing focus on aligning L&D activities with business objectives as well as a continuing rise in the requirement for accountability, including ROI and growing volume and refinement of e-learning to maximise use of technology to improve the effectiveness of L&D programs and solutions.
TrainingZONE: Are there any trends you are spotting in the US that you expect to cross to the UK?
Jack Phillips: Use of the ROI Methodology to show value is now well established in the US as a credible and reliable process. The UK waited to see if this is a viable tool. Now that the proof is in, UK organisations are selecting this methodology as the measurement and evaluation process of choice.
TrainingZONE: Apart from your own, is there a particular session that you would recommend to this year's conference delegates?
Jack Phillips: "Persuading the board to invest in learning technology" and "Delivering business results with Government funding".
* Dr Jack Phillips is chairman of the ROI Institute and author or editor of more than 30 books and more than 100 articles. His expertise in measurement and evaluation is based on more than 27 years of corporate experience in five industries. Dr Phillips’ PhD in human resource management is from the University of Alabama. For more information on Learning Technologies 2006 click here.
