Just as social learning and technology have modernized the way learning occurs in companies today, they also have made a mark on the key performance indicators learning leaders should capture and measure.
David Wentworth, senior learning analyst for Brandon Hall Group, an independent research and advisory firm specializing in learning and development and talent management, said the overall measurements for social and informal learning should not deviate much from the measurements an organization is taking for standard learning modalities. Where the changes show up is in the data points companies analyze in conjunction with social and informal learning.
“Data such as participation rates for discussions, questions asked and answered, and ratings of submitted content become important with social and informal learning,” he said. Wentworth said these pieces of information allow organizations to identify people who are hubs of informal knowledge, along with those who are simply listening or not participating at all.
The need for well-suited key performance indicators, or KPIs, in connection with social and informal learning will only grow along with the popularity of these learning interventions. A 2012 Towards Maturity benchmarking study (Figure 1) showed that:
• 95 percent of companies surveyed want to use technology to increase best practice sharing, but only 25 percent are achieving this.
• 92 percent seek to use learning technologies to respond faster to business change, but only 25 percent achieve this.
• 94 percent want to speed up the application of learning back into the workplace, but only 23 percent achieve this.
The KPIs learning leaders put in place for social and informal learning can be used to address some of the utilization gaps — what companies want to do and what they can actually achieve — organizations experience when they implement new learning technology, as well as help them more accurately evaluate the effectiveness of their learning interventions.
New Measures for a New Process
Wentworth said social learning is all about usage, participation and contribution, and these factors should be measured. Further, the traditional Kirkpatrick measurement levels still apply. “Do learners like the tool? Are they gaining the necessary knowledge? Are they able to turn that knowledge into action? Are outcomes improving because of these learning methods? The goal is to measure the effect of these types of learning on learning goals, behaviors and business objectives,” he said.
KPIs for social and informal learning should examine:
• How many people use the system.
• How often they use it.
• The types of activities they engage in.
• How many posts, questions, resources, etc., people contribute.
• User satisfaction with the technology.
• Individual improvements to competencies and skill sets.
• Effects on their jobs and daily work.
Sodexo, a global food services and facilities management company, launched Expertise in Action collaborative learning groups last March. It implemented this more social and collaborative learning option in response to learners’ development needs within the company’s broader Spirit of Mentoring system, which comprises both formal mentoring, called IMPACT, and informal mentoring including peer-to-peer managed through its employee business resource groups. The Expertise in Action groups leverage social learning techniques to help managers engage regarding topics of mutual interest and development needs, such as communication, strategic leadership and influencing, among others.
During the first three months of the Expertise in Action program, there was a 62 percent increase in social learning technology site usage. In addition, 85 percent of participants said they can use the learning they gained through the experience on the job.
“In the past, we would need to actively pursue people to participate in mentoring,” said Jodi Davidson, director of diversity and inclusion initiatives at Sodexo. “Today, business partners within our organization are initiating opportunities to leverage mentoring and Expertise in Action groups to advance their areas of focus.”
What many learning leaders may find interesting is what Davidson’s executive sponsors told her about the formal IMPACT mentoring program. After six consecutive impact analysis studies showed a 2-to-1 benefits to cost ratio result for the program, executive sponsors suggested it was no longer necessary to conduct future ROI studies, saying they “got it.”
Leading companies constantly re-evaluate their processes, analyzing what and how they measure, and recalibrating organizational objectives. Davidson said instead of the ROI studies conducted in the past, now Sodexo will conduct a second longitudinal study to ensure progress is being made related to pipeline development and other programmatic objectives.
Learning KPIs are a moving target, which can be frustrating, but when results are gathered and analyzed, the impact can be profound. Brandon Hall’s Wentworth said it’s important to understand that the nature of measurements should match the nature of the learning. “For a formal course, you would use formal assessments and other measurements; for informal learning, look for informal methods of measurement.”
Kent Barnett, CEO and founder of KnowledgeAdvisors, a global talent development analytics firm, agreed. He said to gather meaningful data from social learning, leaders need to poll learners while they are consuming content. Micro-polls and pop-up surveys for point-of-interaction measurement can help.
“This will give you a snapshot view of what participants are experiencing, what they are learning, how they are using their new knowledge, etc.,” he said. “Companies can also utilize standard surveys for mentoring, performance support, on-demand resources and community of practice programs to give a deeper look at how social learning is impacting the participants and the company.”
Getting Results
Companies can quickly begin to see results from learning KPIs focused on social and informal learning. For example, among 70 companies using one type of social learning software, monthly metrics from within the tool show that during September there were 25,282 posts generated, 22,067 replies given, 12,921 welcomes given to new participants, 648 polls created or completed, 3,440 events created or attended, 1,147 tasks created or completed and 3,413 likes given.
These data points show that employees in those 70 companies are engaging one another in conversations centered on specific learning goals and objectives, providing support and know-how to their colleagues and completing tasks associated with their learning engagements, and they like what they see.
“A social platform or network is only as good as its level of adoption and use,” Wentworth said. “If learners don’t like using the software, it’s important to find out why. A participant survey will give you quick, honest feedback as to what’s working and what isn’t. It can also be helpful to uncover successful stories about how these tools benefited individual learners.”
One such story from health care company Humana proves that many people can be affected by one learning engagement within a social learning tool. Humana’s social learning platform, The Knowledge Exchange, is designed to build online communities to help people learn from one another, and that is just what happened with a group of colleagues who came together in 2013 to learn about data visualization.
More than 100 people identified a common learning need focused on data visualization, and they found one another and joined together in the Knowledge Exchange to learn about this topic. These employees came from a variety of departments, business units, locations and roles. They collaborated with one another, learned from one another and then put their new skills into action to show how they could apply them to their daily work.
Participants completed a self-generated survey, then put those responses into an actual data visualization product. They took the skills they learned about data visualization and created a tangible output — hard evidence showing the success of their learning endeavor. The group is still active and growing.
The diverse nature of this learning group — across business units, locations, etc. — is a common occurrence these days as technology makes it easier for people to connect. Some 68 percent of organizations are using social learning to build networks inside companies, according to the aforementioned 2012 Towards Maturity study. The study also showed 55 percent are using it to support learning generation and sharing of user-generated content, like the group at Humana. Another 49 percent of companies are using social learning to support personal professional development, which Sodexo and Humana have accomplished.
To help learning leaders get started, companies can use the following questions to measure the effectiveness of their social and informal learning activities.
1. How often do you engage in social learning activities, such as ask a question, reply to a post, access content or participate in meetings?
2. Have you been able to find people to help you address your learning needs?
3. Did the people you interacted with consistently contribute to discussions or other knowledge-sharing activities?
4. How helpful were the knowledge and resources shared within the social learning program?
5. Have you been able to apply the knowledge on the job? How?
6. Why did you participate in the social learning program?
7. Do you feel your participation has been worthwhile?
8. Would you recommend this program to others?
Companies can even take things a step further and ask if people feel they are more engaged now because of the social and informal learning experience, or if they feel more productive since their experience. “This can then be tied back to individual performance to see how their social learning experience is impacting behaviors,” Wentworth said.
He said as with measuring any type of learning, it can be difficult to isolate impact in light of other variables, so “it’s important to take baseline measurements prior to the use of the social or informal learning tools.” This can provide a clearer picture of the effect social and informal learning are having on employees, and learning leaders can then evaluate if their new KPIs work.
Randy Emelo is president and CEO of Triple Creek, an enterprise social learning and knowledge sharing software company. He can be reached at editor@CLOmedia.com.
