Four Ways to Assess Learning

When it comes to learning design, it’s best to assess an issue early. Defining the 
problem, checking assumptions and gathering data rapidly can avoid many 
program execution delays and mistakes.

CLOs are often expected to perform miracles: solve deep-seated learning deficits; design transformational learning programs for all levels; take risks so their organization’s learning processes are cutting-edge; be able to put a new program or process in place at hyper-speed; all while cutting budgets to the bone and delivering a return on investment.

Learning and development teams need to be educators, innovators, financial wizards and master jugglers. Organizational diagnosis and needs assessment tools can help accelerate the design and delivery of targeted, high-impact solutions.
Even a short or partial needs assessment has the following benefits:

• Brings what is working well into focus.
• Prioritizes gaps and needs to enable triage.
• Provides real-time feedback and increases positive engagement from employees.
• Helps the HR and learning team gather valuable, objective, decision-enabling data.
• Builds the competency and understanding of internal staff for diagnosing organizational issues.

There are four steps that comprise a rapid needs assessment process that can immediately be put to effective use.

1. Define the problem: Get specific about outcomes. It may seem obvious to clearly define the need before designing a solution, yet many learning-based projects fail to meet expectations due to hastily or ill-defined issues. Executives often describe business needs in broad terms, which is understandable given their strategic perspective. It’s critical, therefore, that the person responsible for designing learning programs to address these broadly stated goals fully understands the business context and can accurately translate the business need into tangible solutions that will deliver the expected results.

Broad phrases such as “strengthen leadership skills,” “improve teamwork” or “increase innovative thinking” are too vague to make a meaningful difference when attempting to drive change. By asking probing questions and listening intently, these broadly stated objectives can be turned into tangible actions.
Suppose one has been asked to design and deliver a half-day workshop on “influencing skills” as part of a three-day retreat. Before diving into the development of workshop materials, this individual could conduct a quick needs assessment by asking questions, such as:

• What roles do the participants have, and whom do they need to influence?
• What is a typical scenario in which the participants are having difficulty influencing the other party?
• What underlying problems may be contributing to this challenge?
• What organizational factors might help or hinder their ability to successfully influence their key stakeholders?
• How well do the participants understand their stakeholders’ needs?

This type of conversation would likely uncover more specific learning needs than the original request. For example, it might become apparent the real needs are stakeholder management in a matrix structure and managing competing demands. When pressed for time, it is easy to shortcut this important first step; however, skipping it could have a negative impact in the long run.

“You have to understand what the real need is,” said David Emanuel, senior vice president, human resources at CIT Group in Livingston, N.J. “We often look at a symptom and think it’s the need. Diagnosis gives us a modicum of hope that we’ll get it right. It’s the only way that you know you are solving for the right ‘X,’ and even then you may discover that there is more than one ‘X.’ You can always speed up the diagnosis process, but you absolutely cannot skip it. By strengthening our skill sets in this area, we can truly seize our seat at the business table.”

2. Check assumptions: Slow down to speed up. Companies could save millions each year if they developed a team of people to check assumptions. Assumption checkers are highly skilled in critical thinking and recognizing cognitive biases. These are not devil’s advocates or people who are good at saying no. Assumption checkers are internal experts at listening to how a problem or issue is defined or framed to ensure that proposed solutions do not overlook a major barrier or trap, or that the team is not missing a more obvious and less expensive solution. Good assumption checkers listen carefully, avoid jumping to conclusions, understand cognitive biases, reflect on the ramifications of an idea or plan and ask astute questions.

Imagine that the CLO at a large high-tech company that lives and breathes speed is asked to develop a one-day senior management meeting. As a relatively new member of the organization, this CLO thinks he has a good idea of how the management team needs to be shaken up. He designs a day that is fast-paced and confronts senior executives with the ways he believes they are falling down on the job. He then asks a senior person on the team, who has been in the organization much longer, to look at the design and to check his assumptions.

One of the CLO’s assumptions is that these hard-driving executives need to be confronted with their demeaning and sometimes bullying attitude toward their subordinates. The assumption checker listens carefully and points out that the design seems to model the exact thing the CLO wants these executives to stop doing. The CLO subsequently realizes that his frustration and anger has gotten in the way of efforts to design a day that is likely to be productive.

3. Gather data: Keep it brief and focused. Many learning and development professionals say, “We don’t have time to collect data, do interviews, run a focus group or send out a survey.” Data gathering does not need to be elaborate, lengthy or complicated. Valuable data can be gathered in brief, focused conversations. The key is to find ways to gather some data rather than jumping over this step and racing to the design phase.

Imagine how much time the aforementioned CLO could have saved if he had gathered a little data first. Perhaps the CLO could have interviewed some of the senior management team about past management conferences and asked what worked and what did not. Further, if the CLO asked each member of his senior team to informally interview two people, he would have had a more solid basis on which to design the one-day meeting. Informal interviews, questions asked at the end of a meeting and hallway conversations can help gather data rapidly and facilitate plans, designs and decisions.

With a growing emphasis on evidence-based management, learning leaders are in great need of tools that can help them make evidence-based decisions. Gathering data may seem like a daunting and time-consuming task. The words “needs assessment” and “organizational diagnosis” often conjure up images of numerous consultants occupying a conference room for months, conducting hundreds of interviews and dozens of focus groups and sending out surveys. But accelerated needs assessment and diagnosis tools can take the place of traditional methods to help senior learning leaders in their everyday, pressured decision-making lives (see sidebar, page 48).

Incorporating targeted data collection into the learning leader’s work often involves a change in mindset: a shift from seeing data collection as a long and elaborate process, involving survey designers, trained interviewers and focus group facilitators to a shorter, more ad hoc process. While robust data and analysis is valuable, CLOs should consider ways to collect data rapidly when faced with an all-or-none option.

Learning team members can be trained to do rapid data collection. Training in learning inquiry skills and how to conduct informal interviews and ad hoc focus group sessions is available from many sources. These team members can then be deployed on short notice to find out, for example, what is causing an acute problem in a leadership development program, or to understand a seeming misalignment between coaching practice and organizational values. There are now many methods of rapid data collection, some involving technology and others that can be done at a meeting.

4. Develop recommendations: Drive business results with targeted solutions. Recommendations are an important part of any needs assessment process, whether it is rapid or not. The advantage in the three steps outlined in the sidebar is that by the time one comes to developing recommendations, they will be based on a clear statement of the problem or issue, grounded on a foundation of data rather than untested assumptions and targeted to drive business results.

When CLOs train their staff to conduct fast needs assessments, especially in rapid data collection tools, they are not only building their staff’s competence, they are moving toward evidence-based decision making.

Decisions based on data and evidence are more likely to avoid obstacles and to be successfully executed. Many companies pride themselves on fast execution, even though it carries many challenges and sometimes catastrophic errors. Defining the problem, checking assumptions and gathering data rapidly can avoid many execution delays and will almost guarantee avoidance of debilitating mistakes. Gut instinct works sometimes, but it is no basis on which to build a state-of-the-art learning function.

Dana Kaminstein is an affiliated faculty member in the Organizational Dynamics Graduate Studies Program at the University of Pennsylvania. Helen Materazzi is president of Corporate Leadership Resources Inc. and an affiliate with Schaffer Consulting. They can be reached at editor@CLOmedia.com.