An elementary school teacher, a university professor, a company owner, an industry researcher and now an award-winning corporate learning leader, Brenda Sugrue has spent her career in learning. She says working in so many roles has allowed her to identify principles to guide the development and delivery of effective learning regardless of audience, content or context. She said those principles are the anchor from which she makes decisions in her role as global chief learning officer for EY, a $35 billion professional services organization with more than 260,000 people worldwide.
In her first few months as EY’s CLO, Sugrue presented her vision for learning at the company: to increase the alignment, effectiveness and brand of learning at EY. This strategy accelerated the development of the skills needed to achieve the company’s Vision 2020 plan, which aims to make EY the leading global professional services organization by 2020.
“I am very interested in the science of learning and how to apply it at scale to engineer effective and efficient learning experiences,” Sugrue said. “The more we know about how the mind works and the best ways to build expertise, the more confident and successful we will be in designing systems and content to develop knowledge and skills in any audience.”
When Sugrue presented the strategy to the company’s highest management body, EY’s Global Executive, she promised to run learning like a business. “That means having a well-defined budget, clear strategy and priorities, efficient execution and accountability for results,” Sugrue said. “Defining, tracking and reporting those results is critical.”
The bookends of Sugrue’s strategy were design and measurement. An enhanced design increased the effectiveness of the learning and generated business outcomes, while a focus on measurement showed the business impact and increased executive support and investments in learning.
Purposeful Design
Sugrue said design determines the quality of the learning. Learning designs that include realistic practice activities with mechanisms to monitor and correct individual and group errors, for example, are more effective than designs that focus more on information and demonstrations, she said. “A design checklist such as the one we use at EY ensures that all the essential elements of effective instruction are designed into the solution so that it has the best chance of being effective.”
When Sugrue joined EY four years ago, she began her strategy to improve design. One of the first things she did was to simplify and standardize one indicator of the learning function’s performance: the feedback survey that participants complete after learning, often referred to as level 1 evaluation. Sugrue led an effort to reduce level 1 survey questions to the four that are most highly correlated with impact: perceptions of value, relevance, confidence and intent to apply.
“If learners are not feeling good about the learning experience, that can decrease learning, application and business impact,” Sugrue said. “We are now able to report on these four variables across all our learning and identify groups and content with lower scores.”
Sugrue’s team also added two open-ended questions to the survey and began using machine learning to identify what people like most and what they would change. Sugrue said this analysis showed that learners want more opportunities to practice and reflect, the activities that are proven to have the greatest impact on learning.
Calculated Measurement
The second prong of Sugrue’s strategy — measurement — shows how a company is doing and how it can do better. “We can confirm if a learning solution as designed is having the impact we expected, and if not, what is causing the gap,” Sugrue said. “We can correlate patterns of consumption with performance and provide better guidance to learners.”
She said measurement can also show that the cost of training is less than the cost of hiring for particular skills.
Sugrue created a four-pronged measurement strategy: reporting, program evaluation, analytics and special studies. Reporting focuses on costs, consumption, satisfaction and delivery channel mix. With these metrics, Sugrue said they have been able to show that over the past four years, the company delivered 40 percent more learning hours (from 9 million to 13 million hours) to 30 percent more people and increased average hours per person by 15 percent (from 47 to 51 hours), while keeping costs flat at $500 million.
For program evaluation, there is a standard methodology including level 1, 2 and 3 surveys for measuring satisfaction, learning, application and qualitative impact. Sugrue said they supplement surveys with interviews to document business impact cases. Quasi-experimental studies are conducted for quantitative results, using sampling to equalize control and treatment groups. Sugrue said they’ve done 22 business impact studies over the past four years and have consistently doubled business metrics such as revenue, client satisfaction and retention.
Sugrue said their level 1 through 4 results are in the top quartile compared with the results of more than 200 similar studies in other large organizations. For example, EY’s average application rate is 84 percent compared with 66 percent in benchmark studies.
The analytics component of the strategy uses big data to correlate learning with other talent and business metrics such as engagement and gross margin. Finally, Sugrue said the special studies are done internally and externally to identify performance differentiators and validate trends and best practices.
Sugrue said investing in measurement has been a cornerstone of the learning transformation at EY. “It has changed the conversations we have about learning, moving us from opinion-driven to data-driven discussions,” she said. “It has validated our evidence-based approach to learning design and delivery [and] has allowed us to link upfront business alignment of skills and content with business impact.”
Further, she said investment in measurement has guided the company’s configuration of new technology and increased the credibility in an organization of auditors and analytical professionals.
Next for EY
In addition to a focus on design and measurement, EY’s learning transformation has standardized learning processes across the world and increased its use of shared services and automation. Sugrue said they are implementing an integrated talent technology suite which includes learning. “We are fortunate to be part of a larger global talent transformation and to be part of an organization where ‘strengthen global and empower local’ is one of the pillars of our business strategy and how we operate,” Sugrue said. “The challenges of our size and complexity are overcome by our commitment to teaming and collaboration.”
The company is also reshaping its workforce to deliver new and transformed services. Last year, Sugrue’s team launched a badging program for skills such as analytics, blockchain, artificial intelligence and transformational leadership. Sugrue said their upskilling programs rapidly develop the specialists they need to enable growth in particular markets. “We have redefined our approach to career, development and performance with frequent conversations and feedback to guide development goals and activities so that people can stay at the top of their game in their current roles, and at the same time pursue their personal career interests and aspirations,” Sugrue said.
She said her role and the role of her team is to anticipate the skill needs of the business and build those skills as rapidly and effectively as possible. She said she and her team keep people relevant in a changing world and give people the opportunity to transform who they become. “That is the role of the CLO,” she said. “To engineer a learning environment that supports the growth of the business and the personal growth of each individual within it.”
