Few would dispute how important learning programs are for business results. It’s the number one thing executives want to see from learning. That business connection must be there. Fortunately, there is credible business data available in any organization, government or business to prove learning’s impact.
As the value of learning unfolds, five levels of evaluation are possible: reaction, learning, application, impact and ROI, driven by objectives at each level. At the impact level, specific impact data in the organization have actually changed; this is usually reported in the records or in reports. The challenge is to show how much of the improvement is connected to an actual program. This requires that learning leaders separate learning’s effects from other potential influences.
This action does not mean that learning is a process implemented in isolation. On the contrary, all processes must work together. For example, consider a sales scenario where a sales learning program’s objective is to increase sales from existing customers by 10 percent in six months. In the evaluation, you track the impact measure, “sales from existing customers.” You know that advertising, special promotion, pricing strategies, market growth and competitor actions, among other things, could contribute to the sales increase. If you don’t take that extra step to indicate the amount of the sales increase connected directly to the learning program, you lose credibility with senior executives. After all, the marketing team — for instance — is already showing how many sales are connected to a particular promotion, market growth or pricing strategy. That particular business function has been doing this for years. Unfortunately, we learning leaders rarely tackle this issue.
The ROI methodology addresses the challenge of isolation very clearly. It’s a part of the definition, a required step in the ROI process model, and it’s in our standards as a guiding principle. Some 95 percent of this methodology’s users voted to adopt this as a standard, which means that it’s always done.
Critics often ask, who really does this? Well, we know that almost 5,000 people have taken this step and become Certified ROI Professionals. For individuals to earn this designation, they must conduct a study in the workplace showing a program’s effects, including the ROI. Similarly, to measure business impact, learning leaders must take an extra step to show the amount connected to the program — always.
How do you do this? There are many strategies, but here are a few simple tactics to get you started:
- Compare an experimental group — who takes the program — with a control group — who does not take the program. If the groups are matched, the difference can be attributed to the learning program.
- Trend the pre-program impact data to the post period, and compare the actual data to the trend. If nothing else has entered the process, and trend continues, this difference — the actual versus the trend — shows the amount attributed to the learning program.
- Estimate the effects. The most credible individuals, who understand the process best, indicate the percentage of the improvement that is connected to the learning program. When these estimates are collected, the error of the estimate is removed to make it credible. This step discounts the amount of the allocation by the amount of error that is present. This technique can be used if nothing else works, but it is actually very credible. When estimates are compared to other methods, such as an experimental versus a control group, estimates are often more conservative because they are provided by people who know their work best, who are actually driving that impact measure. They can sort it out, and they do.
Other techniques are available. To learn more about how to isolate the effects of your programs please let us know, and we will send you a briefing paper.
The long and the short of it is, to connect learning to the business, you must sort out a program’s effects from other influences. Otherwise, you’re wasting your time. It can be done. Thousands of people are doing it. It’s up to you to make the effort.
Jack J. Phillips is the chairman, and Patti P. Phillips is president and CEO of the ROI Institute. To comment, email editor@CLOmedia.com.
