Don't get lost in the void of measurement. (Image courtesy of Wikimedia Commons.)
In my 2008 book “The Training Measurement Book,” I presented a framework for an integrated approach to measure corporate learning. In those early days of e-learning, companies struggled to figure out how to apply the Kirkpatrick model to their online learning programs, and we found a hugeinterestin trying to measure return on investment. Despite all that interest, we found only 3 to 4 percent of companies were measuring it.
Fast forward to 2015 and the world of training measurement continues to be a challenge, yet the landscape is radically different.
We are just completing an update to our learning measurement research, and there are some interesting findings.
Less than half the companies we surveyed measure informal learning, and about 20 to 30 percent measure usage of videos, self-study materials and communities of practice. About 97 percent measure classroom training and 72 percent measure e-learning — I wonder what the other 28 percent do. The No. 1 challenge they still cite is “inability to understand business impact of learning.”
What holds these companies back? The same issues we encountered in 2008: 62 percent say lack of integrated data from their LMS, 56 percent say lack of automated tools and analysis systems and 59 percent say “lack of tools that link learning to businessoutcomes.”
While the world of learning technologies is far more integrated and advanced than it was back then, these data clearly show the biggest problem we have is not getting the data but rather figuring out what to do with it.
How we measure the effect of training is not really a technical problem; it’s a business process and design problem. Training, like every other nonrevenue generating activity in business, is a support function. Support functions like IT, training, humanresources, finance, security and operations can only be measured by looking at three fundamental things:
Efficiency: How efficiently are we doing these nonrevenue generating activities, as they are expense items? We want to make sure we are not wasting money. In training, we must look at simple volumes: How many people are we training at what cost, and what volume of activity are we generating? Unfortunately, our research shows that 80 percent of most content in big companies is not widely used, so there’s lots of room for rationalization and efficiency work.
Effectiveness: How effective is the training at producing the outcome? Here the critical issue is not to try to measure the ROI of training — which is never a very convincing project because all training has positive ROI, and there are always other factors involved — but rather to measure how effectively did the training solve the problems we set out to solve.
Every training program can be viewed this way. If we take the time upfront to quantify and clearly define the problem we want to solve, training programs improve revenue, reduce turnover and improve quality, as well as many other positive things.
Alignment: The third, and probably most important, way we measure a “nonrevenue generating” activity like training is to look at alignment. How well aligned is this program with the company’s strategic priorities? We have to measure alignment because learning and development budgets are covered by line executives who want to make sure the training they’re paying for is what’s needed.
Only when you continuously realign — revisit the learning and development portfolio against business priorities — can you make sure you are truly working on the most important problems in the company. The “black hole” of measuring training will probably always be with us if we don’t think about things this way.
While analytics is getting better, and we have more tools that tell us what high performers do and how they succeed, ultimately the only way to measure a support function like learning is to use this approach. If we do this well, we can more easily prioritize learning investments and continuously work to refine and improve spending, program design and resource allocation to drive more value.
