Just a couple of years ago, when EMC Corp.’s Tom Clancy talked about hiring a data specialist for his learning organization, he’d get strange looks from his industry colleagues. Today, it’s a different story. The learning profession has learned how crucial such information professionals are.
“Now if you say ‘data scientist,’ everybody jumps up,” said Clancy, the head of education services at the Boston-based information technology company.
What’s changed? In a nutshell, big data. Just like their counterparts in marketing, finance and product development, talent managers are grappling with the information age, sharpening their analytics chops and sifting through massive data sets to deliver results better, faster and cheaper.
But for talent managers, long steeped in the softer side of management and inexperienced or uncomfortable with hard data, big data is requiring a big stretch, particularly when it comes to employee learning and development.
Some are feeling pretty good about their prospects. According to a 2013 survey of business and HR leaders conducted by the U.K.-based Chartered Institute of Personnel and Development, a healthy 63 percent of HR leaders think they are able to draw insight from data.
But in a sign of just how difficult the transition to a more quantitative mindset will be, business leaders had more misgivings about HR. Only one-fifth of them (21 percent) trusted HR leaders to draw lessons from data.
The Data Deluge
Even if you don’t realize it, a digital trail follows wherever you go. That text message to a friend, pit stop at the ATM, selfie posted to Facebook and swipe of the debit card at Starbucks leave a traceable record of your daily activities. With hardly an effort, you generate a wealth of digital information that indicates where you were, what you were doing and whom you were doing it with.
The sheer size of the collective data generated and the technical terms used to describe it tend to cloud our view of the scale of what’s happening. For lack of a better term, it’s big, and this explosion of data is not limited to our personal lives. At the office, swiping your key card shows what time you arrived, your email inbox stores a record of your communication with colleagues and contacts with clients, and every file and document you touch comes loaded with metadata about where, when and how it was created and edited.
This proliferation of information has given rise to big data, roughly defined as massive amounts of data that traditional database tools struggle to capture and analyze. Analysts identified three aspects of big data — volume, variety and velocity — which has stuck as a working model (See “The Three Vs and L&D”).
“The definition of what is data has changed,” said David Dietrich, advisory technical education consultant at EMC and one of those data scientists Clancy talked about hiring. “It used to be rows and columns and numbers if you think about tables in Microsoft Access or worksheets in Excel. Now we’re including everything from clickstreams from Web analytics, text to do natural language processing, or images to do machine vision and classification in medical imagery.”
Given HR’s relative inexperience with sophisticated analytics, it’s fair to question if the data collected about employees is really big data, particularly when it comes to learning and development. Elliott Masie, industry analyst, founder of The Masie Center and co-author of the book “Big Learning Data,” argues it is.
“All day long a tremendous amount of data is being created in what we call data exhaust,” he said, pointing to the digital trail left when a worker attends a class, reads an e-book or takes an e-learning course.
While most organizations only store four pieces of data about e-learning — for example, including who learners are, when they took it, how long they took it for and what their final score is — the depth of available information is much greater. Look at it from an enterprise-wide perspective and it grows even larger.
“The reality is there’s probably somewhere between 10,000 and 120,000 pieces of data in the data exhaust,” he said. “How long did that person spend on each question? How long did they mouse over the wrong answer before going to the right one?”
Multiply that by the large numbers of people taking courses and the increasing speed at which that information is generated, and learning data quickly approaches the realm of truly big data. Combine it with other sources of employee data and it becomes even bigger.
“If you start looking at the social graph and looking at the actions that people are doing in the system, potentially we’re collecting hundreds of data points per individual per day as opposed to a salary action once a year,” said Nick Howe, vice president of learning and collaboration at information technology company Hitachi Data Systems. “We’re talking several orders of magnitude more data.”
Rise of the (Learning) Machines
In learning and development, big data is showing up in new ways, such as machine learning and sophisticated, Amazon-like recommendation engines. It’s also reshaping established practices like instructional design and mentoring.
At Hitachi Data Systems, the company’s 7,000 employees use software by Jive to communicate and collaborate on projects. But it’s also a learning and development data engine that collects employee profile information, analyzes actions they take — documents reviewed or discussions participated in — and recommends additional information or introduces a co-worker who might be helpful. And it learns from the data it collects to make better and more targeted recommendations, thus the term “machine learning.”
If “our chief finance officer goes into our collaboration platform and searches for something and I go into the collaboration platform using the exact same search term, we will get back two different sets of results because the system is taking into account the ecosystem in which we participate,” Howe said.
Adding in performance data makes employee learning even more individualized. At Farmers Insurance, the claims division created the Professional Development Center, a learning and development system with a big data engine that crunches data from 360 assessments, employee engagement surveys and individual performance ratings to find individual leaders’ strengths and weaknesses.
It’s simply too costly and time consuming to build curriculum around every one of the company’s 57 identified leadership skills, said Jeff Losey, head of professional development for the claims division. Nor would it be useful for leaders. “It doesn’t make sense for us to take all of that information and shotgun it out there and put it on a website and say, ‘Here it is available to you in case you need it,’” Losey said.
Instead, through the center, leaders get personalized development in areas that will have the greatest effect, such as constructively delivering feedback and developing others. Farmers also used the data to start a mentoring program and included a “find a mentor” button feature in the system that matches leaders with others who can help. “The reason why big data is important is because it helps us determine exactly what we need when they need it,” Losey said.
Building the Professional Development Center, which began in May 2012 and rolled out in November that same year, also had an additional benefit: uncovering overlooked high-potential people. The center provides a more objective data-centric picture of potential leaders than traditional, subjective succession planning by generating detailed individual talent portraits searchable by skill. “There were many top performers that the executives didn’t know — had never heard of, had never seen,” Losey said.
That discovery led Losey’s learning group to develop a three-day simulation program that gave 60 high-potential employees broader business experience and put them in front of Farmers’ executive team. Twenty of them have moved into formal succession planning. “It takes executive exposure to get many of them into management positions and move up in the organization,” he said.
Beyond programs and measures, big data is also prompting learning and development departments to develop deeper analytic skills. In the past, learning organizations mostly recruited instructional designers, classroom trainers and subject matter experts.
“Now I need people that understand analytics, data mining and the business itself so they know what context to put the data into,” Losey said. “They also need to know where the data came from and what it means, because a lot of data is worthless.”
Thinking that big data analytics is just traditional HR analytics on a larger scale is a fatal error. In traditional HR analytics, organizations have a question they want to answer such as how many people are planning to retire in the next five years or the number of women among the top 100 leaders.
Big data analysis takes disparate data from diverse sources in different formats and uses statistical techniques such as regression analysis and predictive modeling to allow the data to speak for itself, Howe said, and give rise to interesting correlations and outliers.
That sort of sophisticated thinking about data has left many in learning and development behind, said Josh Bersin, founder of analyst firm Bersin by Deloitte. “The world has come and gone and left them in the dust,” he said. “They spent a lot of time focused on trying to measure this one thing that they do which is training … but without measuring anything else it doesn’t have a lot of meaning.”
Bersin recommended creating a talent analytics center of excellence that collects data from the various functional silos, including learning, and use it to examine business problems such as sales productivity, turnover or compliance. That kind of correlated talent management data has the potential to provide real breakthroughs, Masie said.
“If I actually look at the performance changes of an individual — whether it’s cars rented, complaints handled, patients kept alive, whatever you want to use — and you correlate that data back to learning, it gets really interesting,” he said.
Big Data Just Big Hype?
Within L&D, big data is approaching the pinnacle of the hype curve. Like social learning before it, the topic is the theme of industry conferences and subject of countless blogs and articles like this one. Vendors and software providers have seized upon it to tout the capabilities of their applications and services.
It’s easy to listen to the buzz, engage in magical thinking about the potential of big data and say, “Let’s collect a whole bunch of data and see what it says,” Bersin said. “Maybe something beautiful will happen.”
Collecting and analyzing data of this magnitude is enormously difficult, he warned, and requires not only a way to bring together data from multiple systems but also to clean up “dirty” HR data that is old, inconsistent and full of errors. It also requires thoughtful preparation and asking the right kinds of questions.
“Data lies when you ask it the wrong questions,” said Mike Loukides, vice president of content strategy at O’Reilly Media, a technology publisher and media company, and someone who has written on the topic. “If you are sloppy with the questions that you are asking when you start your data analysis, then you’ll get answers that may well look good but don’t really mean anything.”
Bersin recommended that companies start with a distinct problem, such as identifying the characteristics of a successful hire or the factors that lead to employee turnover. “Eventually you’re going to build a bigger and bigger database but you’re going to be much more effective if you start with a small number of problems and try to get the data that will help you solve that problem,” he said.
Above all, it’s important to understand that data — big or little — is only one factor to be considered when determining a course of action. Data science is not about “turning a crank and getting out a big hammer that you can bash your opponents over the head with,” Loukides said. “It’s not about putting an end to the conversation. Data is the basis for the conversation. It’s not about saying we did our numbers and this is the answer.”
Big data can’t remove the ambiguity that is a reality of business. What it can do is shed light on alternative perspectives and uncover possibilities that hedge rather than entirely remove uncertainty.
“You are able to reduce it by doing certain things, by having good-quality data and using appropriate methods to analyze that data,” said EMC’s Dietrich. “Insofar as you can do that, you can make better decisions with less risk.”
In addition to better decision-making, big data can turn up correlations that demonstrate learning’s relevance in potentially new and meaningful ways. At EMC, Clancy, whose organization delivers training to customers as well as employees, wanted to know the effect a customer investment in education had on subsequent EMC business. “A newbie data scientist came back after four months and said I have the answer for you,” he said. “For every dollar that a customer buys from us, they will buy an incremental $10 of EMC product.”
Further analysis showed that EMC’s Net Promoter Score, a measure of customer loyalty, was three times higher for customers who used his group’s training versus those who didn’t. That sort of big data science addresses the age-old credibility problem, giving talent managers more objective information to make more effective recommendations to business leaders.
“Executive-level data science allows you to have executive-level conversations for strategic direction, funding and everything else that goes along with it,” Clancy said.
