In the past two decades, the ROI Institute has had the opportunity to help thousands of organizations connect learning and development to performance. In that time, 10,000 professionals have participated in a return on investment certification process with 4,000 completing an ROI study to earn the designation of Certified ROI Professional, or CRP. And we have conducted research to support and ambitious publications program that has generated 75 books, including more than 400 case studies documenting the success of connecting learning to performance.
From this research and practice, we have reached some conclusions about the connection of learning to performance, which is often misunderstood. To see this connection clearly, it is first helpful to think about the five levels of evaluation: reaction, learning, application, impact and ROI.
Check your understanding by choosing generally true or generally false for each of the statements:
- A positive reaction to learning usually predicts positive learning.
- Certain content reaction measures can predict application.
- Learning usually predicts application for a group of participants.
- Research shows that about 60 to 90 percent of learning is not used on the job.
- ROI analysis contains too much mathematics for it to be useful.
- Increases in learning will usually correlate with increases in impact measures.
- Application always correlates with impact.
- If executives see one ROI study in detail, they will want ROI studies for all programs.
- When there is improvement in business impact, there will usually be a positive ROI.
- An e-learning program usually produces more application and impact success than facilitator-led learning.
Now let’s see how you scored.
Myth No. 1 is generally false. It is difficult to have correlation between reaction and learning unless the reaction measures are all job-related and content-focused.
Myth No. 2 is generally true. Certain types of reaction often predict application, such as “relevance to my needs,” “important to my success,” “I would recommend to others” and “I intend to use.”
Myth No. 3 is generally false. Just because someone has learned the content, doesn’t mean they will apply it.
Myth No. 4 is generally true. Depending on whose numbers you examine, that is about right. Ph.D. dissertations continue to show the failures of learning transfer.
Myth No. 5 is generally false. Most ROI studies can be conducted using only fourth-grade mathematics, a ratio of net benefits to cost, a measure understood by most managers.
Myth No. 6 is generally false. Learning correlates with impact only on rare occasions. Learning does not usually correlate with application, and application is a precondition for an impact.
Myth No. 7 is generally false. Application does not necessarily lead to impact. If learning is not the proper solution, the program likely won’t generate impact, though there may be application.
Myth No. 8 is generally false. When executives see the resources required to conduct these studies, they will help the chief learning officer use this process selectively.
Myth No. 9 is generally false. A positive impact does not always mean a positive ROI. If the solution costs too much, the ROI will usually be negative.
Myth No. 10 is generally false. Unfortunately, our studies of e-learning, particularly when compared to the same program in an instructor-led format, produce less application and impact data. What is often missing is application and impact objectives, a lack of design emphasis on application and impact, and the connection with the participant’s manager, who has much influence on results.
How did you do?
9-10: You should be a consultant on evaluation.
7-8: You should read the 100-page book, “The Bottomline on ROI.”
5-6: You should read the 400-page book, “The Value of Learning.”
3-4: You should attend a five-day ROI Certification program.
1-2: Perhaps you should consider a new occupation.
Much has changed with learning and performance in the past two decades, and the mysteries about learning driving workforce and organizational performance are being unraveled by a substantial number of learning functions. Where do you stand?
