Despite the many models available — Kirkpatrick, Phillips ROI, Success Case Method, Bersin IMPACT Framework and others — companies continuously struggle to build comprehensive measurement solutions for the many types of learning in their organizations.
The most common measures companies track continue to be learner satisfaction, course completions and course hours. Bersin & Associates research shows fewer than 7 percent of organizations regularly measure the impact of their learning programs. Even worse, this number hasn’t risen in the past five years.
But we know the way learning occurs has changed dramatically. Today, learning involves myriad formal and informal experiences, all types of content, strong managerial involvement and a focus on long-term development and talent management.
I’m excited to say learning measurement concepts and principles are changing too. Here are three important changes taking place and what they mean to a learning and development leader.
Measuring informal learning is now easy. If you have an informal, social or collaborative component to your learning program, there are a lot of new tools and approaches available. We have found three major components to measure here: activity — the percentage of learners contributing and how frequently they are contributing; contribution — the percentage of learners offering new content and how frequently they do so; and feedback — how well users like content, what content they read and view the most, which authors are the most highly rated, and who is accessing which content most frequently.
Most new collaboration tools and websites have standard features such as rating scales, feedback links, leader boards — lists of authors who are the highest in ratings or contributions like most emailed and most downloaded by date range — tools that let you view all contributions by a single author or expert authors.
Nearly all new collaboration systems have analytics tools that let you look at usage patterns and analyze activity by division, group, manager and employee. Most LMS systems have yet to implement all these features, but if you use Jive, SharePoint, Lotus, Telligent or nearly any other collaboration system, this type of reporting is readily available. New sharing tools like those available in Blackboard, Jambok, Saba and many other learning systems have these built in.
Measurement standards are arriving. Watch for news about the Talent Development Reporting Principles (TDRP). Pioneered by a small set of companies and thought leaders, and written by Dave Vance, the former CLO of Caterpillar, the TDRP is an attempt to build the generally accepted reporting principles available in other business sectors for talent development. The TDRP draft specification is now being circulated and has been endorsed by BP, Microsoft Learning, PWC, Cigna, Lockheed Martin, Sprint, Sodexo, Lowe’s, Qualcomm, ADP, Deloitte and a variety of industry thought leaders.
Talent analytics systems are on the rise. I spent almost eight years in the development and marketing of data warehousing and analytics systems. Early in my career as a learning analyst, I believed the market was desperately in need of a focused learning and development analytics system, and I tried to work with several LMS vendors to build such products. Today these types of systems are available. KnowledgeAdvisors, Oracle, Plateau, Saba, as well as talent management software providers such as Taleo and SuccessFactors are now delivering analytics platforms which let training managers quickly analyze completion, compliance, activity and utilization of learning programs by division, group, manager or individual.
I spent a day with NetApp’s learning team a few weeks ago and saw the complete implementation of a learning analytics system. CLO Sundar Nagarathnam has a team with extensive experience in these systems. However, after talking with them I learned that only two people worked on the system, and they built it using off-the-shelf software from Oracle. Any reasonably sized learning team can now implement such a solution without tremendous risk or expense.
Training measurement has evolved. If you are a learning leader, now is the time to revisit your measurement strategy and take advantage of new tools and methodologies.
Josh Bersin is the principal and founder of Bersin & Associates. He can be reached at editor@CLOmedia.com.
