- This event has passed.
Graduate Defense: Douglas Hutchinson
January 21 @ 3:00 pm - 4:00 pm MST
Title: Examining Response Labels and Response Order in the Likert Scale
Program: Master of Science in Organizational Performance and Workplace Learning
Advisor: Dr. Seung Youn (Yonnie) Chyung, Organizational Performance and Workplace Learning
Committee Members: Dr. Crystal Han, Organizational Performance and Workplace Learning and Dr. In Gu Kang, Organizational Performance and Workplace Learning
Performance improvement practitioners value evidence-based practices, which include data-driven decisions. Data can be obtained through survey questionnaires designed with closed-ended questions and response scales. The Likert scale (Strongly Disagree, Disagree, Neutral, Agree, and Strongly Agree) is one of the most commonly used response scales. Whether the 5-point Likert scale, as a verbal descriptor scale, should be treated as an ordinal or interval scale is an on-going debate, as different types of statistical analyses are applied to ordinal and interval data. The author of this thesis, conducted this study to examine if survey participants would perceive a 5-point Likert scale close to an interval level measurement when an adverb such as Moderately, Somewhat, or Slightly is added in front of Agree and Disagree. This information could be used by researchers who wish to construct interval level scales in order to allow for use of additional statistical analysis such as parametric tests.
The author of this thesis conducted the study with a convenient sample of performance improvement practitioners, including master’s degree and graduate certificate seeking students, graduates, and faculty in the Organizational Performance and Workplace Learning department at Boise State University. For this study, the author developed a web-based survey instrument using a horizontal slider format. The first screen of the survey instrument contained eight partially-labeled Likert scale sliders, each of which presented three anchors in ascending order (Strongly Disagree on the far-left side, Neutral in the middle, and Strongly Agree on the far-right side) along with their numerical values (-2, 0, and +2, respectively). The slider bar was initially placed on Neutral (0). Using the eight partially-labeled slider, participants were instructed to move the slider bar to locate each of the following eight anchors on the slider; Disagree, Moderately Disagree, Somewhat Disagree, Slightly Disagree, Agree, Moderately Agree, Somewhat Agree, and Slightly Agree. To test the response order effect, the second screen of the instrument asked the participants to repeat the above procedure using another set of eight Likert scale sliders presented in descending order. The third screen of the instrument asked for participants’ gender, age group, and native English speaker status.
The data was collected in October of 2020. The web-based survey system (Qualtrics) recorded data rounding to two decimal points and provided summary report data including mean, standard deviation, variance, and minimums and maximum response scores for each item. A total of 109 participants submitted the survey. However, the initial data screening detected 31 datasets with responses where more than 10% of responses were incomplete or used the incorrect side of the slider continuum, which were excluded. Two additional responses from non-native English speakers were also excluded due to the linguistic aspect of the study. This left 76 responses available for analysis (55 females, 20 males, 1 “do not want to report”).
95% Confidence intervals were constructed for each of the 16 items. The anchor being tested would be considered useful for constructing an interval measurement if it’s corresponding confidence interval included the value -1 or +1. Response order effects were tested by performing a paired sample t-test in excel comparing the average scores of each response option when presented in ascending versus descending order.
The results showed that, when presented in ascending order, both Disagree and Moderately Disagree and both Agree and Moderately Agree were closely aligned with -1 or +1 on the continuum, respectively. However, adding other adverbs Somewhat and Slightly to Agree and Disagree made the 5-point Likert scales to be clearly ordinal scales. When tested in descending order, similar results were found except that Agree was rather far from +1 and moved toward +2. Therefore, the study concluded that when one needs to collect interval data from a 5-point Likert scale, Moderately Agree and Moderately Disagree can be used in either ascending or descending order of the scale, while Agree and Disagree can be used only in ascending order of the scale.
Although Somewhat would not be a good adverb to be added to Disagree and Agree when the 5-point Likert scale is expected to generate interval data, an unexpected interesting finding was that Strongly Agree, Somewhat Agree, Somewhat Disagree, and Strongly Disagree in descending order can be used as an interval-level 4-point Likert scale.