Article
Lanigan, M. L. (2008). Are self-efficacy instruments comparable to knowledge and skills tests in training evaluation settings? Performance Improvement Quarterly, 20(3-4), 97–112. doi: http://dx.doi.org/10.1002/piq.20005
Synopsis
Ajzen’s theory of planned behavior states that an individual’s actual behaviors can be predicted by that individual’s behavioral intentions. The most significant component of behavioral intentions is self-efficacy, which is the confidence the individual feels in his or her ability to perform a behavior successfully. The author, building on her previous research on the role of self-efficacy in evaluating training outcomes, investigated the relationship between self-efficacy scores and knowledge/skill test scores. Is there a correlation between a learner’s sense of confidence in his or her ability to use the new knowledge or skills and the learner’s tested level of knowledge or skills?
Research Questions
The author poses three primary research questions:
- What is the relationship between self-efficacy and knowledge exam scores?
- What is the relationship between self-efficacy and skills test scores?
- What is the relationship between self-efficacy scores collected both pre- and post-training, and self-efficacy scores collected only post-training (but which also measure the learner’s pre-training level of self-efficacy)?
The goal of this research was to look for a direct relationship between a learner’s self-efficacy and that learner’s knowledge or skills test scores. Does the learner’s perception of his or her ability match the learner’s measured knowledge or skills? The author also looked at the timing of self-efficacy assessment. Is it better to measure self-efficacy before training begins, and then measure it again at the end of training to see how the learner’s confidence in understanding the course content has changed? Or is it better to ask the learners to assess their confidence levels at the end of training, and at that time ask them to rate how confident they were before training started as well as after training concluded?
Methods
The study involved a class of 98 individuals in a certification program who attended a 4-hour workshop on how to write a cover letter. The author created three assessment instruments for this study. The first instrument measured the learner’s self-efficacy regarding the task of writing a cover letter. The second instrument was a skills test requiring the learner to write a cover letter based on given information. The third instrument measured the learner’s knowledge about the correct structure and content of a cover letter. The three instruments were administered in order before the training class started, and again at the conclusion of class. The self-efficacy assessment administered before class started only asked learners to measure their pre-training level of self-efficacy; the self-efficacy assessment administered post-training asked learners to rate both their pre-training and post-training levels.
Findings
In this study, the author found a significant link between self-efficacy levels and knowledge scores, both pre-training and post-training. The link between self-efficacy and skills scores, both pre- and post-training, were also significant but not strongly so. There was a strong correlation between the learners’ self-efficacy scores measured pre-training and how they rated their pre-training self-efficacy after training was concluded.
From this, the author drew two main conclusions.
- For training topics which are low-risk, evaluators may be able to substitute self-efficacy measures for knowledge or skill measures. Although the author did not discuss this further, it seems possible that self-efficacy assessment may be a better predictor of future behavior than knowledge quizzes when the training topic is behavior-oriented, such as customer service skills. The author acknowledges that for higher-risk training programs which require precise knowledge or skills, such as pilot or medical training, much better evidence of a link between self-efficacy and knowledge or skills would be needed in order to substitute self-efficacy assessment for knowledge/skills assessment.
- Since there was no significant difference in how learners rated their level of pre-training self-efficacy when asked before training started or after it ended, the author concluded that this assessment could be given at either time. However, the author noted that this may be dependent on the training subject; learners may not understand more complex training objectives before class begins and thus may not yet be able to accurately rate their self-efficacy.
Implications for HPT
ASTD’s 2009 study on the usage of training evaluations found that 81% of the organizations in the study evaluated training at the knowledge level; this level of evaluation is usually done at the end of the program by the training department, which can collect and analyze this data. Only 55% evaluated their training programs by examining the actual post-training job behaviors of the learners; this level of evaluation requires more extensive data collection and a greater need for resources and organizational support which may not be available. By incorporating the concept of planned behavior into the evaluations conducted at the end of training programs, training departments could collect data which may indicate potential changes in the learners’ on-the-job behaviors. While this is no substitute for conducting a true behavior-based evaluation, it may give training departments some useful guidance on the effectiveness of their programs that they otherwise would lack.
Discussion for OPWL-N Members
Do you include questions about self-efficacy when you conduct reaction-level evaluations (Kirkpatrick level 1) of your training programs? Have you used self-efficacy assessment as an adequate substitute for knowledge or skills assessment? If so, what was the topic of the training program? About what should practitioners be careful when substituting self-efficacy assessment for knowledge or skills assessment?
Workplace Oriented Research Central (WORC)
Prepared by OPWL Graduate Assistant, Perri Kennedy
Directed by OPWL Professor, Yonnie Chyung
Posted on February 22, 2012