C-suite executives almost universally value ROI when they look to the CLO to account for not only where learning dollars go and to whom, but also to know that learning investments have a significant impact on the business. Learning dollars must be justified and results clearly illustrated to warrant future investments and to establish the correct direction learning activities should take in order to illicit the greatest impact on the enterprise. How exactly senior learning leaders can do that is not so well defined, particularly because all learning returns are not equal. Evaluating business objectives, workforce development needs, and the technical and administrative infrastructure of a company are some of the ROI basics to consider when developing a learning measurement strategy.
Return on effort (ROE) emerged briefly in the metrics landscape as an alternative to ROI, but like many hot, new learning and development ideas, ROE proved to have visible flaws. It is not the most concrete measurement method available, and there is some skepticism that it has any value at all. “I don’t see any advantage that it has over ROI,” said D. Verne Morland, managing partner, ROI Learning Services. “In fact, I see several disadvantages. First, tough minded business people generally say that any effort can be reduced to a dollar value. So if you’re reducing the effort to a dollar value, why don’t you talk about it as an investment and as return on investment? Secondly, because ROE in traditional financial terms means return on equity, it’s going to be a little bit of an education process to explain to executives that we’re talking about a different ROE. Finally, I think even the word effort has a bit of a bad connotation. It reminds me in education settings of talking about giving someone an A for effort. In business terms you don’t get an A for effort. Effort is not where it is. It’s the return.”
There are also different definitions of ROE. For some the E stands for expectation not effort, and the question of the term’s exact meaning is part of the problem with its potential use in a learning and development measurement scenario. Morland said that unless the term is measuring learning activity that can’t be reported in any other way, creating a new term and educating people on what it means, how it’s used, how to interpret it and what’s a good or a bad number probably won’t offer much additional metric value. Instead, consider the ROI basics. “All returns are not equal,” Morland explained. “The cost of a program, once you’ve done your homework and you’ve accounted for all of the costs, becomes known but the returns can be multiple. If you’re creative and you look at all the possible, positive impacts on the business that a (learning) program could have, you can calculate multiple returns on the same investment, and those returns add up.”
When evaluating the most beneficial strategy to approach the measurement issue, Morland said a traditional place to start is with an organization’s business objectives. “Let’s say that the main objective of the business for a particular period in time is sales growth. Obviously, training that focuses on improving close rates or order values or total orders would be measures that would resonate with the top management of the firm. If there is a different primary objective, and let’s take one that sounds similar but is in fact quite different, profitability or profit growth, then the goal would be to have training that focused on the sale of higher margin products or that showed people how to reduce the cost of sales through things like automated prospecting, faster selling cycles and so forth.
“Some companies are driven by growth,” Morland said. “You could have a company, for example, that might eventually become a conglomerate specializing in different product lines because it wasn’t interested in the product so much as it was interested in the growth of revenues. Other companies are interested in profitability, and everything they do as a company protects and reinforces profitability. Some are motivated by customer service, some by innovation. I bring this up because some objectives change over time. A company might emphasize sales growth for awhile, and then when it gets growth but its profits are slipping, it might emphasize profitability for a time. CLOs need to be well tuned to the current winds in the executive office as to what are the objectives for the next accounting period or the next fiscal period.”
Different overarching business strategies usually result in different training strategies, thus metrics strategies also will vary. “Most businesses also have a companion set of goals with regard to their infrastructure,” Morland said. “Employee retention tends to be a common one. It would be important, for example, to have learning collaborate with HR to determine based on the employee perspective, the importance of learning, professional development and career pathing to employee satisfaction. If that can be used to measure reduced turnover costs, then that’s another return that is being driven by the same course or set of courses. In this case it might be necessary to almost take your entire learning budget and compare it against the returns that are obtained in multiple parts of the business. But a single course can have multiple areas of return.”
A second point of consideration centers on the workforce’s development needs. “This is something that people always talk about,” Morland said. “Sometimes it’s lip service, and sometimes people follow through with a needs analysis. Often you get management who says ‘I know that our workforce is not skilled enough in X, therefore, we need training on X.’ My argument would be, that can be a very astute observation on the part of management, but at the same time it’s always important to ask the prospective learners what they need.”
To support the need for learner input, Morland pointed to the Hawthorne Effect an industrial psychology case study based on the Western Electric Hawthorne Works plant in Cicero, Ill. The company did a series of experiments to improve the productivity of workers on an assembly line. Leaders talked to workers and asked, for instance, if better lighting would help their work, then gathered their input, made some improvements and productivity went up. The two groups talked again, made more changes and productivity went up even more. Leaders ultimately concluded that it wasn’t the changes they were making so much as it was the fact that they were taking an interest and asking people about their work, caring for workers and asking “What would make your work life better?”
“That carries over into training as well,” Morland explained. “Taking an interest in people is going to have returns on the training that you subsequently provide them because you’ll have more engaged students and better learning outcomes. I’m a great believer in needs analysis not only for practical reason of discovering what people need but also for the ancillary benefit of getting greater buy in from the learning audience.”
Another practical consideration for learning metrics is what in your organization actually can be measured? “It doesn’t do any good to come up with an elaborate ROI formula if you then go to your administrative department and say, ‘We need these numbers,’ and they say, ‘We can’t get those numbers.’ Do a little homework with your administrative and technical departments before you develop your ROI strategy to find out what kind of information is routinely being gathered and reported today. The path of least resistance in any measurement effort is to use measurements that are already being gathered because that affords you the opportunity to establish a baseline prior to the training’s implementation.
“If something is really necessary to measure training effectiveness, you could potentially augment your reporting system to gather additional data. But my own experience in large corporations is that’s a long shot,” Morland said. “It’s better to modify them, if you’re not compromising your objectives, to accommodate the data that is already being collected rather than specifying something that may be a better measurement but in fact is not something that you’re not likely to get with any statistical reliability.
“Most things in business can be reduced to some dollar value,” Morland said. “That may sound harsh but even if you look at something outside of the training world, say life insurance, for example. In a sense, life insurance is in some ways putting a value on human life. We don’t like to think of it that way, but it does. There are a lot of intangibles in business like employee satisfaction or employee goodwill that we say, ‘Well, you just can’t put a number on that.’ In many respects that’s taking the easy road out. The good will that a business has in terms of establishing a name and a marketplace over the years, accountants actually assign a value to company goodwill that goes into the company’s balance sheet. It’s basically what makes the balance sheet balance. Emphasis should be to try and bring things down to a dollar value but to also be willing to look outside the general financial realm at other dimensions.”
Morland said that a balanced scorecard concept can help learning leaders think beyond traditional financial measures that come from the accounting department and help them to assign values to business processes such as the recruiting process, employee retention, employee productivity and product quality. “These can be assigned a value, and at least part of that value, depending on the training that’s gone on, can be attributed to the training.
“One of the most common mistakes that CLOs tend to make is to think that this kind of rigor isn’t needed or that it costs too much,” Morland said. “Typically CLOs come from a learning background. Because they’ve devoted some potentially significant portion of their lives to the learning process, they obviously believe in it. They are the choir, the preacher, the ones who in their heart of hearts believe that training is good. It’s almost hard for many of them to understand the psychology that says training may be good, but it’s expendable if push comes to shove. They have to understand that it’s a competitive world out there.There are other departments, other functions that are after corporate dollars. They cannot rest on the logic that learning is good, that learning makes a difference, that learning makes people happier and more productive. All of these things may be true, but when push comes to shove, they don’t carry the day.”
