With an identified need for leadership training at the store-manager level, Gap Inc. implemented a leadership-training program called Leadership Training for Supervisors (LTS). This program focused on supervisory and leadership skills—how to influence and interact with store employees. The business goals that LTS would ultimately seek to impact were increasing sales, reducing turnover and reducing inventory loss. The initial rollout was for general managers, area managers, district managers and regional managers for Gap, GapKids, Banana Republic and international divisions of Gap. In order to evaluate the effectiveness of this program, Gap’s corporate training department developed an evaluation strategy that used all of the Kirkpartick levels and followed many of the guidelines listed in Figure 1.
Level 1: Reaction: Because this was an initial rollout of LTS, it was important to gather data from the participants for program improvement. After completing the program, each participant was given an LTS program evaluation to complete that focused on how well learner expectations were met, the relevance of the program to their jobs, the overall presentation of training experience, as well as LTS’s skill level and pace. The items in the survey were rated primarily on a 1 to 5 scale with 1 being “entirely ineffective” to 5 being “very effective.” Yes/No and fill-in-the-blank items were also included in the form. Overall, results were positive and gave the evaluation team the data that it needed to determine what was working and what adjustments to certain aspects of the LTS program were necessary.
Level 2: Learning: After the instruction, was there an improvement in the knowledge and skills of the participants? To answer this question, each participant in the LTS program was evaluated after the instruction with a fill-in-the-blank test with 55 possible answers that allowed the learners to demonstrate their grasp of the concepts taught and their ability to apply these concepts in different management situations. Because this was new material to the majority of the learners, there was no pre-assessment. The results of the post-assessment indicated that significant learning did occur during the program, with an average score for all divisions of 87 percent, indicating that new information was gained in order to respond successfully to the items in the assessment.
Level 3: Behavior: The question asked at this stage was: Did the managers change their behavior on the job as a result of the training? To answer this question, the designers created a short- and long-term evaluation strategy. Shortly after the instruction, a post-program survey was developed that measured the degree to which the managers’ behavior changed in the 12 skills areas of the LTS program: diagnosing, leadership styles, flexibility, direction, support, contracting, receiver skills, sender skills, goal setting, positive feedback, constructive feedback and providing recognition. Managers who completed the program were asked to rate themselves on their LTS skills. Their direct reports were also asked to rate their manager on the same skill set. The manager/direct report surveys used a 6-to-1 scale (6 = much better, 5 = somewhat better, 4 = no change, 3 = somewhat worse, 2 = much worse, and 1 = don’t know). This scale assessed perceived competency in the 12 skills from the two respective points of view. Results indicated that managers were at an average of “much better” to “somewhat better” in their on-the-job application of their new skills. How did these results hold up over time? This was the question asked during the longer-term part of the evaluation strategy. Approximately one year after the program was administered, the managers’ direct reports were again asked to rate their managers’ LTS skills. The results indicated that the managers were not only continuing to apply their skills but were also improving in the application of those skills.
Level 4: Results: Prior to the evaluation, the stated business goals that LTS would ultimately seek to impact were increasing sales, reducing turnover and reducing inventory loss. How did the program measure up to these goals? The evaluators looked at stores that had increased sales, reduced turnover and reduced inventory loss as related to the number of managers that showed increases in LTS post-assessment scores. They found that there was a strong relationship between high LTS scores and increased achievement of the stated business goals. Between 50 percent and 80 percent of the time, managers of these successful stores had also increased their LTS scores.
By using all of the Kirkpatrick levels along with a logical set of evaluation guidelines, Gap’s corporate training department was able to create a reasonable and progressive link between each level of its evaluation program, which culminated in the achievement of its stated business goals.
