Many still view the measurement of training effectiveness as a black art. But in five years of research, I have identified key philosophies, techniques and best practices that are practical and really work.
The goal of training measurement is to deliver useful information for decision-making. You must collect consistent, repeatable data so you can make meaningful comparisons of programs, audiences, instructors, delivery methods and other factors. The most practical approach is through standardized, end-of-program assessments for learners and their managers. At Eaton University, KPMG and AT&T, learners must complete standard, end-of-course assessments that include satisfaction and effectiveness measures. The organization can then plot trends over time to identify well-designed programs and problem areas.
KMPG Canada and Randstadt each ask their learners standard questions such as, “How much did this training program help you increase your job performance?” Although they’re subjective, responses are clear indicators of relative effectiveness. Another technique is to send two assessments: one at the end of the program and another 90 days later. Although the second assessment will typically have a lower response rate, it provides a better understanding of job impact.
Use Job-Impact Indicators
One of the most important measures of a well-designed program is direct impact on job performance (called Kirkpatrick Level 3). A simple and powerful approach to measuring job impact is to use a series of standard assessments such as: How much impact did this training program have on your job? (standard numeric ranges); how well can you apply the skills and knowledge you received in this program on the job? (standard numeric ranges); what are the biggest obstacles that will prevent you from using these skills and knowledge on the job? (multiple choice answers).
These indicators will give you significant understanding of how the program is used and applied. As KPMG Canada discovered, many learning programs rated high for learning impact do not score nearly as well on job impact.
As you apply standardized assessments, you might find one program with a slightly lower satisfaction or effectiveness rating than another. Don’t jump to conclusions based on a single score. You must do additional analysis to understand which programs succeed, which fail, and why.
If you want to quantify the business benefit of training, you must collect quantitative data upfront. Randstadt uses a simple approach. Every organization that requests a learning program must submit a training request form. If a business manager has a $5 million problem in processing errors and a training program scores 80 percent in effectiveness, the program likely returned approximately $3 to $4 million in savings.
Collect Business Measures
If your organization collects consistent business metrics, you can use this information for trend analysis. The data will not tell you the precise impact of training, but when analyzed over time, you can draw inferences about training impact. For instance, Merck measures sales-rep effectiveness through fieldtrip reports managers complete. The reports are used to identify field problems and training needs. Subsequent reports indicate whether behaviors change after training.
Assign a Dedicated Resource
Organizations that excel at measurement have assigned individuals to own the measurement process and supervise reporting. Program managers and consultants cannot be expected to create a companywide approach, develop the right assessments or collect and analyze data on a consistent basis. A dedicated employee can do the research, evaluate tools and identify best practices.
If you focus on building a consistent, reliable and easy-to-analyze process using the principles above, you will find yourself with valuable, credible information that will support important decision-making.
Josh Bersin is the principal and founder of Bersin & Associates, and has more than 25 years of experience in corporate solutions, training and e-learning. He can be reached at jbersin@clomedia.com.
