In an effort to identify the world’s elite learning organizations, Chief Learning Officer magazine established the LearningElite benchmarking program last year. The intent is to provide learning and development practitioners with a transparent benchmarking program that recognizes the efforts of learning organizations across the globe.
The LearningElite designation recognizes organizations of any size, in any industry and in any country and does not focus on specific learning initiatives, but rather the strategic contribution of the overall L&D function. The application to be considered is an open process which involves completing a detailed form that provides a view of the L&D function and an opportunity to assess strengths and areas in need of improvement.
Every organization that applied for recognition received a customized scorecard highlighting performance across the program’s evaluation dimensions. That scorecard also included a comparison among applicants, giving them an opportunity to assess their strengths and areas for improvement based on the benchmarks provided.
The program was developed by the Human Capital Media (HCM) Advisory Group, the market research arm of Chief Learning Officer magazine, and featured a research-based evaluation model to identify the world’s elite learning organizations. As part of that development process, magazine editors and researchers recruited learning executives, academics and government agency officials to participate in a survey, focus group and in-person development event to build the five-dimension model and application. Next, program developers established the criteria-referenced cut score for the five dimensions based on empirical research and feedback from subject matter experts.
The Five Dimensions of the Elite
The Chief Learning Officer LearningElite program evaluates L&D organizations in these five dimensions:
• Leadership commitment.
• Learning strategy.
• Learning execution.
• Learning impact.
• Business performance results.
Leadership commitment: An aligned strategy effectively executed may not produce results if it’s not backed by leadership. When executive leadership is clearly behind learning and development, it resonates across the organization. When learning is viewed as a key strategic endeavor, barriers are removed, results are expected and positive outcomes are delivered. When learning is central to an organization’s values and goals, it will have leadership commitment and empower the learning and development function to deliver results.
Learning strategy: The strategy of the modern learning organization must be linked to and embedded in the core organizational strategy. Any learning organization that is separated from the broader organizational strategy is functionally disconnected and ultimately will not be able to meet its customers’ needs.
Learning execution: An effective learning strategy means the right plan is in place with the right people to get the right things done, and it is assessed along the way. Sound learning execution is rooted in a technology foundation to support the people who execute a standardized process to develop and deliver learning solutions when and where learners need them. Measurement of the plan allows for course correction and change.
Learning impact: Although leadership, strategy and execution are crucial, they do not necessarily mean L&D leaders know what they are doing adds value. Effective learning organizations have a systematic method to measure learning strategy, execution processes and workforce performance to demonstrate they are adding value. Meaningful internal metrics must be established and communicated and must act as the driver for change to demonstrate that learning procedures are embraced and effective at yielding proven results.
Business performance results: Traditionally, learning and development organizations have been held accountable for producing internal results. The modern organization is now responsible for proving contribution to overall business performance. Making this connection is not simple, and showing the linkage is sometimes nothing more than a leap of faith. However, the most advanced learning organizations show how their efforts improve corporate performance and benchmark against others.
Modern learning organizations exist in an interconnected, complex business environment. As good stewards of resources, they must deliver value to the broader organization, external and internal customers, partners and suppliers, as well as to the learning and development field. Learning and development professionals are not merely consumers of learning content — they are active participants in this self-generating field, benchmark against the best and provide lessons learned to support the continued advancement of the L&D industry.
Evaluating and Judging the Elite
The LearningElite model employs Robert Brinkerhoff’s success case method evaluation technique, whereby applicants describe and document the efforts of their L&D organizations and back them up with evidence. This approach uncovers results, accomplishments and practices and requires a combination of qualitative and quantitative data.
Once data was collected, researchers from HCM Advisory Group used statistical techniques to determine which application items were functioning best and to establish item performance across the five dimensions. After this analysis, items that were not functioning as expected were removed from final calculations for all applicants. Applications were then vetted by magazine editors and researchers to confirm they met the standards and criteria outlined. Applications that failed to meet the criteria were removed from the process and the corresponding organization was notified.
The application data were then entered into an online system and each application was sent to multiple judges. These judges applied through an online system, and those who were reviewed and determined to be senior learning leaders and contributors to the L&D field were selected to judge applications. The judges reviewed applying organizations that were outside of their primary industries and were able to recuse themselves if there was a conflict of interest.
Program administrators assessed inter-rater reliability — the degree of agreement among judges — by correlating scores for each applicant across judges. They also assessed intra-rater reliability — the degree of agreement among multiple rating sessions by a single judge — by correlating the scores for each judge. Any ratings provided by a judge that didn’t demonstrate reliability were eliminated and applications were submitted to be judged again.
The judges were given a scoring rubric detailing the criteria and standards by which they should rate each application item. In alignment with the goal to create a transparent benchmarking program, that scoring rubric can be found in the 2011 LearningElite Annual Report, available through the Chief Learning Officer website.
When judging was completed, results were reviewed and assessed for consistency, and the ranking of LearningElite organizations was based on a calculation of the final score. The selected organizations are diverse in geography, industry, type, size, revenue and investment in learning and development. They are headquartered in six countries and represent 25 industries.
Characteristics of the 2011 Elite
Overall, the 2011 LearningElite organizations are a disparate group of small- to large-size organizations headquartered across the globe, representing all types of organizations from a range of industries with varying levels of investment in learning and development.
Among the organizations, 70 percent are small- to mid-size (up to 100,000 employees) and 30 percent are large (more than 100,000 employees). Further, 70 percent of the organizations generated between $1 billion and $50 billion in annual revenue last year, and 12 percent reported annual revenue of more than $50 billion.
Looking at the aggregate results, it became clear that a large budget, while helpful, was not essential to being recognized as a LearningElite organization. According to analysis, 31 percent of these organizations had an L&D budget of more than $50 million; in contrast, one-third of the organizations had a budget less than $10 million.
Additionally, 14 percent reported their L&D budget was greater than 5 percent of their annual operating budget last year, while the majority (86 percent) stated their L&D budget was less than 5 percent of their annual operating budget.
Among the organizations, 24 percent reported annual learning and development investment represented less than 1 percent of their total annual revenue, and 24 percent reported annual learning and development investment represented greater than 1 percent of their total annual revenue. Further, 43 percent spent $1,000 or more per employee on learning and development, while 38 percent spent less than $500.
The diversity of the 2011 LearningElite organizations indicates that exceptional learning and development can be achieved regardless of size, geography or level of investment.
Stacey Boyle is vice president of the Human Capital Media Advisory Group, the research and business intelligence arm of Chief Learning Officer magazine. She can be reached at editor@CLOmedia.com.
