Matthew May, PhD, Senior Research Associate
Mcallister Hall, Research Associate
Vanessa Crossgrove Fry, PhD, Interim Director
May, M., Hall, M., & Crossgrove Fry, V. (2021). Idaho literacy intervention program evaluation. Idaho Policy Institute, Boise State University.
In 2020, the Idaho Legislature authorized an independent, external evaluation of the state’s literacy intervention program (Program) that will consider: (a) program design, (b) use of funds, including funding utilized for all-day kindergarten, (c) program effectiveness; and (d) an analysis of key performance indicators of student achievement, as well as any other relevant matters. For the third year, Idaho Policy Institute (IPI) was contracted to conduct the evaluation.
Performance data traditionally used in this evaluation is unavailable because of the COVID-19 pandemic. As a result, IPI administered online surveys to teachers (n=494) and administrators (n=101) and conducted in-depth interviews with teachers (n=11) to understand the function and perceptions of the Program across the state. This report also includes data from the 2019/20 (LEA) literacy plans, budgets, and expenditure data.
Teachers surveyed are moderately confident in the IRI by Istation’s ability to accurately identify student performance. Any lack of confidence may be attributable to issues associated with test structure. This can include student unfamiliarity with technology, poor quality of test audio, and timed questions.
Use of Funds
On average, LEAs use a majority of their funds each year to hire more or increase pay of current personnel. The most common personnel employed with literacy funds are support staff who help lead intervention groups and provide more opportunities for small group or one-on-one instruction. Administrators indicated that if their LEA were to receive more literacy funds, they would increase funds dedicated toward personnel.
Survey and interview data were used to conduct an analysis of the Program’s strengths, weaknesses, opportunities, and threats (SWOT). Strengths reflect positive impact of the Program and opportunities identify potential growth for the Program.
Strengths include the ability to target specific learning gaps among students. As a result, students receive instruction designed for their growth. Additionally, Program funding is being used for items that can be traced directly to the students
Opportunities identified include allowing the funds to be directed toward kindergarten readiness education. This would require a change in state statute to include non-school age children as funding eligible. Funds could also be allocated to students in grades 4-6 who are performing at a K-3 level. Some respondents indicated that a statewide literacy professional development opportunity would also enhance the Program. Additional Program funding could allow all students the opportunity to attend all-day kindergarten.
According to survey data, 37% of LEAs surveyed in the state are using at least some of their state literacy funds toward a version of a free all-day kindergarten program. Examples of the structures discussed in interviews are provided in Table A.
Table A: All-Day Kindergarten Types
|Part-time Individual needs-based||All-day with teacher (alternating)||Limited||Students scoring 2 attend all-day two days a week, students scoring a 3 attend all day four days a week|
|Individual needs-based||Half-day with teacher/ Half-day with paraeducator||Limited||Students scoring lowest on kindergarten screening|
|School needs-based||Half-day with teacher/ Half-day with paraeducator||Open||All kindergarten students attend|
|Free kindergarten||All-day with teacher||Open||All kindergarten students attend|
|Fee-based*||All-day with teacher||Limited||First-come, first-serve|
*Charging fees/tuition is not allowed per state statute, but it still occurs..
COVID-19 continues to impact instruction. Many teachers have had to change their instruction this year to include more review. As a result, practice time, engaging activities, and content depth are sacrificed. Teachers are optimistic that students will produce average IRI scores in the Spring. Teachers also report providing multiple versions of support for students needing literacy intervention during virtual learning sessions.