Several years ago, I read a study by the American Society for Training and Development concerning the value of training evaluation that has particular relevance for the challenges of evaluating all types of diversity ROI interventions, including diversity training interventions. The study focused on the notion of “making training evaluations more effective” and queried organizations regarding how well training evaluation was meeting business needs. Responses to the 26 questions led to disturbing conclusions. “The pursuit of excellence in learning evaluations continues, but so far few organizations think they have mastered them,” the report stated. “Only about one-quarter of respondents … agreed that their organization got a solid bang for the buck from their training evaluation efforts.”
The Accountability Trend
Over the years, I have heard similar comments made by diversity practitioners, CDOs and unfortunately by many CEOs, key leaders, stakeholders and sponsors about diversity ROI evaluation efforts. Yet, the only way to determine that a diversity initiative or skill development effort and diversity training are having the desired effect is to use formal evaluation processes and cost-benefit analysis methods. The results of these activities can confirm the positive effects of the diversity intervention and identify improvements to make it better.
It is imperative to “master” the processes to measure the impact of diversity initiatives to have any credibility with the organization. Finance, marketing, sales or operations and the like cannot get a “pass” on showing their results and effectiveness, and neither should diversity if we want to be taken seriously. Mastering diversity measurement and evaluation strategies can contribute to maximizing an organization’s overall return on investment (ROI or return on mission), thus showing its value to the bottom line.
For years, diversity managers convinced top executives that the impact of diversity couldn’t be measured, at least at the monetary contribution level. However, today many executives are aware it can and is being measured in many organizations. Top executives are subsequently demanding the same accountability from their diversity training functions. With the acceptance of ROI as a mainstream measurement tool for diversity functions, the debate has shifted from whether ROI should be conducted to how it should be conducted on a consistent, standardized basis.
Barriers to Diversity ROI Implementation
ROI is a familiar term and concept for managers, particularly those with business administration and management degrees. Today’s chief diversity officers who are aligned as strategic business partners are more aware of bottom-line issues in the organization. They are pleased to see the ROI methodology applied to the evaluation of diversity performance improvement interventions.
Many enlightened business managers often take a professional business approach to diversity, with ROI being part of the strategy. Top executives who watched their diversity budgets continue to grow without appropriate accountability measures have become frustrated with this approach. In an attempt to respond to the situation, they have turned to diversity return on investment (DROI). Top executives are now demanding DROI calculations from diversity departments where they were not required previously.
So, what factors prevent us from mastering diversity ROI measurement? Here a few that are consistently challenging for diversity practitioners:
1. Lack of skills and orientation: Many diversity staff members neither understand ROI nor do they have the basic skills necessary to apply the process within their scope of responsibilities. Diversity ROI measurement and evaluation is not usually part of the preparation for the diversity job or taught as part of a university education focused on diversity. Also, the typical diversity training program or intervention does not focus on results, but more on diversity awareness concepts, activities or other issues. Staff members attempt to measure results by measuring learning only instead of the full range of diversity performance intervention outcomes (at all six levels). Consequently, this is a tremendous barrier to implementation that must be changed such that the overall orientation, attitude and skills of the diversity staff member is focused on results, impact and outcomes.
2. Faulty needs assessment: Many existing diversity interventions are not based on an adequate needs assessment. Some diversity interventions have been implemented for the wrong reasons based on management requests alone or efforts to chase a popular fad or trend in the industry. If the intervention is not needed, the benefits from the program will be minimal. A DROI calculation for an unnecessary program will likely yield a negative value. This is a realistic barrier for many diversity interventions.
3. Fear: Some diversity departments do not pursue DROI measurement implementation due to fear of failure or of the unknown. Fear of failure appears in many ways. Designers, developers, facilitators and program owners may be concerned about the consequences of a negative DROI. They fear that the DROI measurement process will be a performance evaluation tool instead of a process improvement tool. Also, the DROI process will stir up the traditional fear of change. This fear, often based on unrealistic assumptions and a lack of knowledge of the process, becomes a real barrier to many DROI measurement efforts.
4. Discipline and planning: A successful DROI evaluation implementation requires much planning and a disciplined approach to keep the process on track. Implementation schedules, evaluation targets, DROI analysis plans, measurement and evaluation policies, and follow-up schedules are required. The diversity change management team may not have enough discipline and determination to stay on course. This becomes a barrier, particularly if there are no immediate pressures to measure the return. If the current senior management group is not requiring a DROI evaluation, the diversity change management team may not allocate time for planning and coordination. Also, other pressures and priorities often eat into the time necessary for an effective DROI evaluation implementation. Only carefully planned implementation efforts succeed.
5. False assumptions: Many diversity staff members have false assumptions about the DROI process that keep them from attempting DROI. Typical assumptions include: (a) The impact of intervention cannot be accurately calculated, (b) Operating managers do not want to see the results of diversity expressed in monetary values. They won’t believe it, (c) If the CEO does not ask for the DROI, he or she is not expecting it, (d) CDO denial – “I have a professional, competent staff. Therefore, I do not have to justify the effectiveness of our programs,” (e) Learning or this type of intervention is a complex but necessary activity. Therefore, it should not be subjected to an accountability process, etc. These false assumptions form perceptible barriers that impede the progress of a DROI evaluation implementation.
What to Do About Them
To overcome these barriers, it will be critical to:
1. Build DROI skills and measurement orientation: Don’t wait until you are asked about the DROI of your diversity intervention to gain competency and business acumen in this area, start learning about DROI today!
2. Learn the detailed steps to conduct a comprehensive needs assessment: Needs analysis is the cornerstone of any diversity performance analysis effort. It provides you with appropriate justification for either developing or not developing your diversity intervention. You must conduct a needs analysis, no matter how abbreviated, before any diversity intervention takes place.
The objectives of a needs analysis are to:
• Describe the exact nature of a performance discrepancy.
• Determine the cause(s) of the discrepancy.
• Recommend the appropriate solution(s).
• Describe the target population.
3. Overcome fear by taking action: The best way to overcome fear is by taking action, generating results, evaluating the outcome and implementing improvements. Fear is often based on a lack of knowledge, so the antidote is to learn and master the DROI skills and processes.
4. Build DROI discipline and planning focus: There is really no substitute for implementing a thorough approach to a DROI evaluation process. It must be implemented using effective project planning and management skills as well as following the DROI methodology according to each step in its design.
5. Eliminate any false assumptions: Let’s face it; the DROI evaluation process and its associated analytics are here to stay. It’s only realistic that diversity practitioners eliminate any false assumptions, wishful thinking and outdated measurement paradigms. In the future, there is likely to be even more demand for DROI analysis feedback, demonstrated credibility and intervention performance value ties to the bottom line.
Benefits of a DROI Evaluation
The DROI methodology is the most accurate, credible, and widely used process to show the impact and results of a diversity intervention. The diversity change management team will know the specific contribution from a select number of programs. Preparing a DROI study will determine if the benefit of the program, expressed in monetary values, has outweighed the costs. It will determine if the program made a contribution to the organization.
Calculating DROI in different areas can show which interventions contribute the most to the organization, allowing priorities to be established for high-impact performance. Successful programs can be expanded into other areas – if the same need is there – ahead of other programs. Inefficient programs can be redesigned and redeployed. Ineffective programs can be discontinued. Using this process has the added benefit of improving the effectiveness of all diversity interventions we conduct. Only those diversity practitioners who can operate as full strategic business partners will survive for the long term. We can ill afford to let the aforementioned barriers trap and derail us.
