Counselor Education Department Evaluation Report 2021-2022

Overview: Program Evaluation

Stakeholders, including current students, faculty, site supervisors, program graduates, and community employers, are involved in the evaluation process. This section outlines (Section 4. A.1) the data that will be collected, (Section 4.A.2) a procedure for how and when data will be collected, and (3) a method for how data will be reviewed or analyzed (Section 4.A.3). The process of evaluation consists of:

1. University reports on current students’ academic progress.
2. Faculty review of professional, personal, and academic dispositions and development (PPADD) and evaluations of student achievement as related to the key performance indicators (KPI) of the program and specialty areas.
3. Departmental surveys of current students, program graduates, site supervisors, and employers. Masters’ students in their 3^rd year and 1-, 3-, and 5-year program graduates are asked to provide feedback regarding their experiences in the counseling program through an exit survey and program graduates survey. Respondents are asked to rate their level of preparedness on professional identity standards, program objectives, and specialty area program objectives. The exit survey and program graduates survey also contain questions regarding program satisfaction. Site supervisors of 3^rd year students and employers of 1-, 3-, and 5-year program graduates are asked to provide feedback regarding preparedness of their supervisee/employee on professional identity standards, program objectives, and specialty area program objectives. Surveys include quantitative and qualitative measures.
4. Compilation and analysis of data from the multiple evaluation methods.
5. Annual Faculty Work Meetings to review findings, assess current status of all aspects of the programs and suggest changes/modifications in the curriculum, coursework, departmental functioning, faculty activities, student selection and retention activities, student monitoring and other aspects of existing programs.
6. Generation of Annual Evaluation Report.
7. Sharing findings and suggested changes with students, administration, site supervisors, advisory board members, program graduates and others interested in the Counseling Masters’ and Doctoral Program at Boise State.

The Program Evaluation Process is overseen by the Chair and the CACREP Coordinator (or Co-Coordinators).  All department faculty are participants in the evaluation process. The Evaluation Plan is systematic and ongoing from year to year. Multiple methods of assessment are used throughout the academic year. Annual assessments include evaluations of current students’ academic, professional, and personal dispositions and development, level of learning based on students’ accomplishment of key performance indicators, as well as demographic and other characteristics of applicants, students, and graduates. All faculty members evaluate the programs, curriculum, coursework, admissions process, and current student functioning. Site supervisors evaluate current students and program outcomes. Graduates are evaluated by assessing alumni knowledge of program objectives and employer evaluations.

The Logic Model that guides the overall evaluation process is depicted in Figure 1.

Figure 1. Logic Model

The assessment and transition points for short term outcomes for the MA Program and PhD Program are depicted in Figures 2 and 3.

Figure 2. MA Programs (School and Addiction) Assessment and Transition Points

Figure 3.  Ph.D. Program Assessment and Transition Points

Table 1 presents the timeline used to complete the assessment.

Table 1. Evaluation Procedure

Process Evaluation
Data Collected (A.1)	Procedure and When Data is Collected (A.2)	Methods for Review or Analysis (A.3)
# Students Enrolled (B.2)	Chair reviews enrollment data prior to Summer (May), Fall (August) and Spring (December) Semesters	Compare expected vs. actual enrollment
Student Demographics (B.2)	Chair and Advisor review during admissions (application self-report) and in September	Review trends and comparison to overall State demographics
Student Course Evaluations (B.2)	Faculty review in December, May and August	Review University generated output and make determinations for change based on patters. Discuss with the Chair and/or peers as needed
Student Supervisor Evaluations (B.2)	Practicum and Internship Instructors in December and May	Review student evaluations make determinations for change based on patters. Discuss with Practicum Coordinator and/or Chair as needed
# Staff; # Faculty, # Adjuncts	Chair reviews contracts in May and December	Chair determines instructional needs based on grant buyouts, CACREP caps and ratios
Review of Mission, Goals, and Objectives (B.1)	All Counselor Education Core Faculty and Part-Time Lecturers review in August	Revisions are made based on aligning with CACREP standards, community needs, and Boise State’s, College of Education, and Department’s Strategic Goals
Review of Curriculum Matrix (B.1)	CACREP Coordinator reviews matrix and syllabi prior to Summer (May), Fall (August) and Spring (December) Semesters	CACREP Coordinators checks that syllabi include standards and KPIs and facilitates a conversation with all Core and part-Time Lectures as needed regarding curriculum placement for standards and KPIs
Review of Syllabi (B.1)	All Counselor Education Program Faculty review syllabi prior to Summer (May), Fall (August) and Spring (December) Semesters	Faculty update syllabi based on most current practices and readings and reviews standards and KPIs are included based on matrix and review and update assignments, expectations, and delivery methods as needed
Review of Assessment Process (B1-3)	CACREP Coordinator and Chair in August and January	CACREP Coordinator and Chair review assessment activities and outputs including surveys that need to be collected and reported for the academic year
Outcome Evaluation
Data Collected (A.1)	Procedure and When Data is Collected (A.2)	Method for review of Analysis (A.3)
CPCE pass rate (B.3)	CPCE Course Instructor reviews scores in January – March	If students score below 1 SD below the national mean, students retake the section(s). Students complete a multiple-choice exam (70% to pass). Procedure is described in detail in MA Handbook
NCE pass rate (B.3)	CACREP Coordinator assesses pass rates through Exit surveys of graduating students in May	CACREP Coordinator and Chair review data and report in Evaluation Report
Key Performance Indicators (B.1)	All Counselor Education Faculty submit completed rubrics and CACREP Coordinator reviews in May, August, and December	CACREP Coordinator facilitates a discussion as needed during faculty meeting to review scores and reports KPI scores in Evaluation Report
Supervisor Evaluations (B.3)	Practicum and Internship Instructors in December and May	Review student evaluations and make determinations for change based on patterns. Discuss with Practicum Coordinator and/or Chair as needed
GPA (B.1)	Advisor reviews in May and December	Advisors review and discuss any concerns with students
# Admission to Candidacy (B.3)	Advisor reviews by October	Advisor reviews and approves all candidacy applications and reports to the Chair
Licensure Rates (B.3)	CACREP Coordinator reviews in May	CACREP Coordinator sends survey to program graduates and reports in Evaluation Report
Employment Rates (B.3)	CACREP Coordinator reviews in May	CACREP Coordinator sends survey to program graduates and reports in Evaluation Report
Exit Survey (B.3)	CACREP Coordinator and Internship Instructor ask students to complete during Internship class (or alternatively identify class) in May	CACREP Coordinator reviews, discusses with faculty during faculty meetings, and reports in Evaluation Report
Program Graduates Survey (B.3)	CACREP Coordinator reviews in May	CACREP Coordinator sends survey to program graduates, discusses with faculty during faculty meetings, and reports in Evaluation Report
Supervisor Survey (B.3)	CACREP Coordinator reviews in May	CACREP Coordinator sends survey to program supervisors, discusses with faculty during faculty meetings, and reports in Evaluation Report
Employer Survey (B.3)	CACREP Coordinator reviews in May	CACREP Coordinator sends survey to employers, discusses with faculty during faculty meetings, and reports in Evaluation Report
Program Development Review (B.2)	Advisor reviews with student in Fall and Spring as needed	Advisor works with students during Fall 1 to complete the Program Development form and then works with students as needed to make revisions. This form is used by the student to apply for candidacy during year 3
PPADD (B.1)	All Core Counselor Education Faculty and Part-Time Lectures conduct the assessment in April	Advisor reviews scores and sends a Letter of Concern or Remediation Plan if a student scores less than 2 on any item
Portfolio (B.1)	PhD Advisor reviews in during Spring 3	Advisor applies rubrics to review and grade Portfolio and assess KPIs
Doctoral Comps (B.1)	Dissertation Chair and Student during Spring 2	Student has options described in PhD Handbook and Chair mentors student and submits grade
Dissertation (B.1)	Dissertation Chair beginning as early as Fall 1 and completed by Spring 3	Student has options described in PhD Handbook (article based on traditional dissertation) and Chair mentors student and submits grade

Through individual instructor review and analysis, CACREP Coordinator and Chair review and analysis, and all counselor education core faculty and part-time lecturer review and analysis of measures described above, decisions are made for curriculum and program improvement (Section 4. A.4) including, but not limited to modifying KPIs, course curriculum and sequencing, course delivery, as well as recruitment and retention efforts, training implementation for supervisors.

Evaluation of Program Inputs

Faculty, Adjunct Faculty, Staff

The Department of Counselor Education has 5 full-time faculty members and 1 part-time faculty member. Faculty workload consists of teaching, research, service, and administrative activities, with 50% of time typically devoted to teaching. Of the 23 required courses and 4 elective courses in the MA Program, core faculty taught 15 required courses and 1 elective course and non-core faculty taught 8 required courses and 3 elective courses. Additionally, we hire professional counselors licensed and registered as supervisors in the state of Idaho to provide practicum lab live supervision. Doctoral students under the supervision of Counselor Education faculty taught 2 courses.  For the Doctoral program, Counselor Education Faculty taught all of the Counselor Education and Supervision core courses. The Department has one full time Administrative Assistant.

Site Supervisors and Advisory Board

Internship Site Supervisors continue to play an invaluable role in the education and development of our students. Site supervisors provide ratings on skill-based measures during the 3^rd year of the MA Program, as well as completing a survey assessing Professional Identity Standards and Program Objectives.

Advisory Board members participate in the selection of MA students each year by reviewing applications and participating in applicant interviews. The Advisory Board also met to discuss the addiction and school emphasis curriculum, internship experience, and other issues specific to the addiction and school programs.

Site Supervisors and Advisory Board Members, along with current students and program graduates, were are also called upon to review and provide feedback on the revised department mission statement and program objectives. Input from these key stakeholders was used to modify the mission statement and program objectives.

Evaluation of Program Outputs

Program Activities

The Counselor Education Department offered a MA in Counseling Program with a School Counseling cognate area and Addiction Counseling cognate area. The Department also accepted a student for Fall 2021 to the PhD in Counselor Education and Supervision program.

The MA and PhD curriculum were reviewed during bi-weekly faculty meetings. Curricular offerings are aligned with CACREP standards and KPIs and associated measurements have been placed throughout the program offerings.

The assessment and evaluation procedure were reviewed this year. In August 2021 faculty, with the input of all program stakeholders, reviewed the program objectives, key program indicators, and key program indicator measures. Minor revisions were made to the program objectives. When surveyed, 100% of the stakeholder respondents (n=36) reported they agreed or strongly agreed with the Program Mission and Program Objectives.

Program Applications and Enrollment

In 2021-2022, we received 70 completed applications for our MA program (38 with the expressed interest in the school cognate, 32 with the expressed interest in the addiction cognate) and 2 completed applications for our doctoral program. There are 71 students enrolled in the MA Counseling Program and 3 students enrolled in the Doctoral Program for fall 2021. Table 2 presents number of students enrolled by cohort year and program. Demographic diversity is presented in Tables 3 – 5.

Table 2. Enrollment MA Program and PhD Program

	MA Program			Ph.D. in CES Program
Year	School	Addiction	Total
2019	13	10	23	1
2020	14	10	24	0
2021	15	9	24	1
TOTAL	42(59%)	29(41%)	71	2

Table 3. Student Demographics – School Program

Cohort	Gender			Ethnicity
	Male	Female	Did not disclose	White	Hispanic	Asian-American	African- American	Native American	Other or Not Known
2019	1	12	0	12	1	0	0	0	0
2020	1	12	1	12	0	1	0	0	1
2021	2	13	0	14	1	0	0	0	0
TOTAL	4(10%)	37(88%)	1(2%)	38(90%)	2(5%)	1(2%)	0(0%)	0(0%)	1(2%)

All Enrolled School Students – Ethnicity by Gender
	White	Hispanic	Asian-American	African- American	Native American	Other or Not Known
Male	3	1	0	0	0	0
Female	35	1	1	0	0	0
Did not disclose	0	0	0	0	0	1

Table 4. Student Demographics – Addiction Program

Cohort	Gender		Ethnicity
	Male	Female	White	Hispanic	Asian-American	African- American	Native American	Other or Not Known
2019	1	9	9	0	0	1	0	0
2020	1	9	7	1	0	0	0	2
2021	3	6	8	0	0	1	0	0
TOTAL	5(17%)	24(83%)	24(83%)	1(3%)	0(0%)	2(7%)	0(0%)	2(7%)

All Enrolled Addiction Students – Ethnicity by Gender
	White	Hispanic	Asian-American	African- American	Native American	Other or Not Known
Male	3	0	0	1	0	0
Female	19	1	0	0	0	1

Table 5. Student Demographics – Doctoral Program

Cohort	Gender		Ethnicity
	Male	Female	White	Hispanic	Asian-American	African- American	Native American	Other or Not Known
2019	0	1	1	0	0	0	0	0
2020	0	1	1	0	0	0	0	0
2021	1	0	1	0	0	0	0	0
TOTAL	1(33%)	2(67%)	2(100%)	0(0%)	0(0%)	0(0%)	0(0%)	0(0%)

All Enrolled Doctoral Students – Ethnicity by Gender
	White	Hispanic	Asian-American	African- American	Native American	Other or Not Known
Male	1	0	0	0	0	0
Female	2	0	0	0	0	0

Summary of Findings

The MA program has an average of 20 – 25 students per cohort with about 59% of students in the school cognate and 41% in the addiction cognate. Students are predominantly female and White; however, we have minority representation that is representative of our state. The Doctoral program currently admits one student per year. Eight of the students admitted have been female, two males, and all have been White.

The MA program aims to graduate approximately 20 students per year. In general, 22 – 24 students are admitted as faculty anticipate a 10% attrition rate from orientation to fall enrollment and an additional 10% attrition rate from fall orientation to graduation.

Student Satisfaction with Program

Student program satisfaction is measured and the Student Exit Survey and Program Graduates Survey. Results are presented in Table 6.

Table 6. MA and Ph.D. Program 3^rd Year Students Satisfaction with Program (Percent Satisfied or Very Satisfied)

	MA Program*	Ph.D. Program**
Program Area	Mean and % Satisfied or Very Satisfied	Mean % Satisfied or Very Satisfied
Faculty
Faculty Expertise	3.6 – 68%	4.0- 100%
Faculty Availability	3.9- 79%	3.0- 0%
Quality of Instruction	3.4- 68%	4.0- 100%
Quality of Advising	3.1- 47%	3.0- 0%
Curriculum
Content Coverage	3.6- 68%	4.0- 100%
Course Sequencing	3.8- 84%	4.0- 100%
Number of Electives	3.0- 42%	4.0- 100%
Clinical Courses
Practicum Quality	4.1- 84%	5.0- 100%
Internship Availability	4.0- 79%	5.0- 100%
Internship Quality	4.1- 79%	4.0- 100%
3rd Year Student Overall Satisfaction	3.7- 79%	4.0- 100%
Program Graduates Overall Satisfaction+	4.3- 100%	3.3- 75%

Note. *N = 19, Masters Program; ** N = 1, Doctoral Program; ⁺N= 11 (MA); N=4 (Doc)

Summary of Findings

Quantitative data indicate overall satisfaction with the program. Quantitative findings from the Exit Survey indicate current 3^rd year student were most satisfied with aspects of their clinical courses and least satisfied with the number of electives offered and the quality of advising. Overall, quantitative data indicate average to above average levels of satisfaction with the Masters Program.

Evaluation of Program Outcomes

Professional, Personal, and Academic Dispositions and Development Review

All students are reviewed at least once a year to assess professional, personal, and academic dispositions and development. All core faculty members teaching and advising students participate in the review. Students are required to meet a standard of professional ethical behavior, and appropriate personal behavior, as well as participate in professional and personal growth and development activities.

Faculty concerns regarding individual students were discussed at faculty meetings through the academic year and students were reviewed by faculty using the Professional, Personal, and Academic Development form (PPADD).  The PPADD was developed by faculty in 2013 and reviewed annually and updated as needed. Table 7 indicates average scores on the PPADD in the areas of professional, personal, and academic development by cohort.

Table 7. Faculty Ratings of Students’ Professional, Personal, and Academic Dispositions and Development (PPADD – 1-3 scale)

	MA 2019 Cohort	MA 2020 Cohort	MA 2021 Cohort	All PhD Students
Compliance with ACA Standard C.5	100%	100%	100%	100%
Compliance with ACA Standard F.8.a	100%	100%	100%	100%
Professional Development	3.0	2.1	2.0	3.0
Personal Development	3.0	2.0	2.0	3.0
Academic Development	3.0	2.2	2.0	3.0
Total PPAD	3.0	2.1	2.0	3.0

There was 1 student who received a score of < 2.0 on one or more of the PADD areas.  Table 8 indicates the cohort, area of concern, PPADD rating, and action taken by faculty in response to the area of concern.

Faculty also review student issues at faculty meetings on an ongoing basis as needed. These discussions may also result in areas of professional, personal, and academic development and remediation/dismissal from program. There were two additional students who on remediation plans in 2021-2022.

Table 8. Professional, Personal, and Academic Dispositions and Development Problems

Cohort	Area of Concern	PPADD Rating	Action
2020	Professional	1.0	Student was sent a letter of concern, met with advisor, and was on a remediation plan.
2019	Professional	—	Student was sent a letter of concern. Concerns were addressed with support of school counseling cognate coordinator and cohort advisor, and the student successfully graduated in May 2021.
2018	Professional/ Academic	—	Student was placed on a remediation plan at the end of spring 2021. In the fall of 2021, student worked closely with a faculty member to complete remediation plan and successfully graduated in December 2021.

Students are also required to maintain a GPA of 3.0 or higher, achieve grades of C or better in all graduate level courses, and achieve a B or better in COUN 505 (Counseling Skills) and COUN514/516 (MA Practicum I and II), COUN614 and COUN 616 (Doc Practicum I and II), and a Pass in COUN526/528 (MA Internship I and II), COUN626 and COUN628 (Doc Internship I and II), and COUN592/692 (MA and Doc Portfolio).  Doctoral students also complete a Program Development Form with the Doctoral Advisor every year to determine expected progress in academic development.

Students are also sent a letter of concern when they receive a C in any of their coursework, a B- or lower in a clinical course, and Incomplete, or dismissal from the program if retention and remediation planning are not successful and this pattern continues. Table 9 indicates the term, cohort, course where problems occurred and action taken by faculty in response to the academic problem.

Table 9. Academic Development Problems

Term	Cohort	Course	Grade	Action
Fall 2021	2020	COUN 514	I	Student met with practicum supervisor and cohort advisor to discuss concerns. Student successfully completed COUN 531 Practicum Intensive, resolving concerns.
Fall 2021	2020	COUN 514	I	Student met with practicum supervisor and cohort advisor to discuss concerns. Student successfully completed COUN 531 Practicum Intensive, resolving concerns.
Summer 2021	2020	COUN 541	I/F	Student received an “I” in COUN 541 that eventually turned into an “F” when student did not complete course requirements by deadline. Student paused program and will retake the course if the student reenrolls in the program.

Summary of findings

One MA student was identified by faculty regarding personal, professional, or academic areas on the PPADD. Additional students were identified over the course of the academic year as having academic, professional, and/or personal concerns. In all cases, the advisor, course instructor, and/or practicum supervisor discussed concerns with the students. Students placed on remediation plans were monitored and ultimately all students successfully completed remediation plans. There were no problems identified for our doctoral students in the areas of professional, personal, or academic development.

MA Program

CACREP Professional Counseling Identity Standards

CPCE scores were reviewed to assess knowledge and performance on Professional Identity Standards for MA students. Spring 2021 CPCE pass rates by specific identity standard area are shown in Table 10. All students who did not pass the original CPCE area were given the opportunity to take an exam in Spring 2021. All students passed all sections.

Table 10. CPCE Pass Rates by Professional Counseling Identity Standard Area

Professional Counseling Identity Standard	Initial Pass Rate	Final Pass Rate
CPCE
Professional Orientation and Ethical Practice	100%	100%
Social and Cultural Diversity	95%	100%
Human Growth and Development	100%	100%
Career Development	100%	100%
Counseling and Helping Relationships	100%	100%
Group Counseling and Group Work	100%	100%
Assessment and Testing	95%	100%
Research and Program Evaluation	95%	100%

CACREP Professional Counseling Identity Standards for MA students are also assessed through Exit, Supervisor, Program Graduates, and Employer Surveys. Survey results for MA students are shown in Table 11-13.

Table 11. MA Program Objectives (1-5 Scale)

MA Program Objectives	Mean
	Mid-Term Outcomes		Long-Term Outcomes
	Exit Survey(N = 18)	Supervisor Survey(N = 22)	Program Graduates Survey(N =23 )	Employer Survey(N = 2)
Core Average
Students will develop a strong identity as professional counselors who embrace ethical practice, advocacy, and reflection within the context of a multicultural and pluralistic society.	4.4	4.4	4.4	5.0
Students engage in self-reflection while learning theories and models that facilitate effectively working with a diversity of clients. Students consider elements of power and privilege as they adapt their counseling practice to meet the needs of their clients.	4.4	4.7	4.2	3.0
Students will learn how biopsychosocial factors influence development and functioning across the lifespan and integrate a developmental perspective in their counseling practice.	4.6	4.5	4.4	5.0
Students learn the significance of career development across the lifespan. Students will learn strategies (e.g., assessment, resource identification, and advocacy) to help individuals develop a career plan and address career-related challenges.	3.6	4.1	4.1	5.0
Students will learn theories and skills that promote a personalized approach grounded in evidence-based practice for working with clients. Students will learn common factors underlying ethical and effective counseling practice.	4.6	4.7	4.3	5.0
Students will learn theories and processes related to group formation and facilitation in a variety of settings.	4.4	4.5	4.4*	5.0
Students will gain the knowledge needed to administer, interpret, and/or utilize assessment tools in an ethical and culturally appropriate manner to guide clinical and educational decisions.	4.0	4.4	3.9*	5.0
Students will acquire knowledge and skills related to research methodology, statistical methods, and the use of qualitative and quantitative findings to guide data-informed decision making and evaluation of counseling practice.	3.9	4.1	4.0*	4.5

*N=22 (vs. N=23 for other questions- one program graduate stopped answering questions half way through).

Table 12. School Program Objectives (1-5 Scale)

School Program Objectives	Mean
	Exit Survey (N = 13)	Supervisor Survey (N = 14)	Program Graduates Survey (N = 16)	Employer Survey (N =1)
Students will demonstrate an understanding of the multiple roles school counselors have as leaders, advocates, and systems change agents in P-12 schools and be able to evaluate components of a comprehensive school counseling program.	4.6	4.6	4.3	5.0

Table 13. Addiction Program Objectives (1-5 Scale)

Addiction Program Objectives	Mean
	Exit Survey (N = 5)	Supervisor Survey (N = 8)	Program Graduates Survey (N = 6)	Employer Survey (N = 1)
Students will identify roles, theories, and models related to addiction counseling and apply empirically supported approaches to work with clients.	4.6	4.8	4.3	5.0

Findings from the surveys (ratings on a 5-point scale) indicate MA students are achieving Program Objectives, with an average performance from 3.6-4.7 for mid-term outcomes and 3.9 – 5.0 for long-term outcomes for MA Program Objectives, 4.6 for mid-term outcomes and 4.7 for long-term outcomes for School Program Objectives, and 4.7 for mid-term outcomes and 4.7 for long-term outcomes for Addiction Program Objectives. The lowest scores relative to all outcomes (all scores average above a 4.0) were in areas related to working with diverse clients, career development, and research.

Current and former students (program exit survey, program graduates survey) were asked to respond to open prompts inquiring about positive aspects of their program experience and areas that need improvement. Responses related to positive aspects included appreciation for faculty members and supervisors (e.g., flexible, supportive, knowledgeable, helpful, top notch), appreciation for relationships built with peers (e.g., cohort model), appreciation for personal and professional growth opportunities, and appreciation for resources and networking opportunities. Responses related to areas of improvement related to the impact of the pandemic (e.g., disliked online courses, remote instruction), turnover in faculty member positions, desire for more advising support (e.g., more support related to licensure and certification, more meetings), suggestions for additional courses (e.g., human sexuality, neuroscience, attachment theory, ASAM, mindfulness, solution focused therapy, private practice, and so forth), and improvement of some existing courses (e.g., teach more contemporary theories in theories course, improve career, assessment, and research courses).

MA COUNSELING – Enrolled student outputs

CACREP 2.F.1. Professional Orientation and Ethical Practice

Program Objective:

1. Students will develop a strong identity as professional counselors who embrace ethical practice, advocacy, and reflection within the context of a multicultural and pluralistic society.

Key Performance Indicator:

1. Students will demonstrate knowledge of the multiple roles and responsibilities of the professional counselor including the role of advocate, interdisciplinary team member, and ethical practitioner.

KPI Measures:

• COUN 501 Advocacy Activity – 100% of students earned an A on assignment
• COUN 501 Ethical Decision-Making Model – 96% of students earned an A on assignment; 4% of students earned a B on assignment
• COUN 526 Supervisor Evaluation – 100% of students earned a Pass on assignment
• COUN 690 Comprehensive Examination – 100% of students Passed comprehensive exam

CACREP 2.F.2. Social and Cultural Diversity

Program Objective:

2. Students engage in self-reflection while learning theories and models that facilitate effectively working with a diversity of clients. Students consider elements of power and privilege as they adapt their counseling practice to meet the needs of their clients.

Key Performance Indicator:

2. Students will demonstrate cultural competence that enables them to meet the individual needs of clients in a multicultural society.

KPI Measures:

• COUN 509 Exam/Paper- 90% of students earned an on assignment; 10% earned a B on assignment COUN 514 Case Conceptualization – 64%of students earned an A; 36% earned a B
• COUN 526/528 Supervisor Evaluation – 100% of students earned a Pass on assignment
• COUN 690 Comprehensive Examination – 100% of students Passed comprehensive exam

CACREP 2.F.3. Human Growth and Development

Program Objective:

3. Students will learn how biopsychosocial factors influence development and functioning across the lifespan and integrate a developmental perspective in their counseling practice.

Key Performance Indicator:

3. Students will identify biopsychosocial factors that influence development and functioning across the lifespan and apply developmental theories to work with individuals.

KPI Measures:

• COUN 526 Supervisor Evaluation – 100% of students earned a Pass on assignment
• COUN 690 Comprehensive Examination – 100% of students Passed comprehensive exam

CACREP 2.F.4. Career Development

Program Objective:

4. Students learn the significance of career development across the lifespan. Students will learn strategies (e.g., assessment, resource identification, and advocacy) to help individuals develop a career plan and address career-related challenges.

Key Performance Indicator:

4. Students will demonstrate an understanding of career theories, assessments, career resources, and the interrelationship between career and other life roles.

KPI Measures:

• COUN 507 Career Report – 100% of students earned an A.
• COUN 690 Comprehensive Examination – 100% of students Passed comprehensive exam

CACREP 2.F.5. Counseling and Helping Relationships

Program Objective:

5. Students will learn theories and skills that promote a personalized approach grounded in evidence-based practice for working with clients. Students will learn common factors underlying ethical and effective counseling practice.

Key Performance Indicator:

5.a. Students will demonstrate an understanding of counseling theories and models for client conceptualization.

KPI Measures:

• COUN 502 Personal Theory Paper – 92% of students earned an A on assignment; 8% of students earned a B
• COUN 528 Integrated Theory Paper – 100% of students earned an A on assignment
• COUN 690 Comprehensive Examination – 100% of students Passed comprehensive exam

CACREP 2.F.5. Counseling and Helping Relationships

Program Objective:

5. Students will learn theories and skills that promote a personalized approach grounded in evidence-based practice for working with clients. Students will learn common factors underlying ethical and effective counseling practice.

Key Performance Indicator:

5.b. Students will demonstrate the ability to establish and maintain the therapeutic alliance and utilize case conceptualization and treatment planning skills.

KPI Measures:

• COUN 505 Skills Rubric – 100% of students earned an A on assignment
• COUN 514 Client Conceptualization Paper -64 % of students earned an A; 36% of students earned a B.
• COUN 526/528 Supervisor Evaluation – 100% of students earned a Pass on assignment

CACREP 2.F.6. Group Counseling and Group Work

Program Objective:

6. Students will learn theories and processes related to group formation and facilitation in a variety of settings.

Key Performance Indicator:

6. Students will demonstrate knowledge of group theories and stages, as well as demonstrate group facilitation and leadership skills.

KPI Measures:

• COUN 513 Exam – 71% of students earned an A on assignment; 29% earned a B
• COUN 513 Leader Plan Assignment – 100% of students earned an A on assignment
• COUN 690 Comprehensive Examination – 100% of students Passed comprehensive exam

CACREP 2.F.7. Assessment and Testing

Program Objective:

7. 7. Students will gain the knowledge needed to administer, interpret, and/or utilize assessment tools in an ethical and culturally appropriate manner to guide clinical and educational decisions.

Key Performance Indicator:

7. 7. Students will demonstrate knowledge of the selection, implementation, and interpretation of different types of assessment tools used within counseling.

KPI Measures:

• COUN 504 Assessment Report – 67% of students earned an A on assignment; 33% of students earned a B on assignment
• COUN 550 Biopsychosocial Report – 100% of students earned an A on assignment
• COUN 690 Comprehensive Examination – 100% of students Passed comprehensive exam

CACREP 2.F.8. Research and Program Evaluation

Program Objective:

8. Students will acquire knowledge and skills related to research methodology, statistical methods, and the use of qualitative and quantitative findings to guide data-informed decision making and evaluation of counseling practice.

Key Performance Indicator:

8. Students will demonstrate the ability to design a counseling program evaluation and the ability to collect, analyze, and use data to evaluate their counseling practice.

KPI Measures:

• COUN 512 Program Evaluation Assignment – 100% of students earned an A on assignment
• COUN 514 Counseling Evaluation Assignment – 57% of students earned an A; 43% of students earned a B
• COUN 529 Counseling Practice Evaluation Poster – 100% of students earned an A on assignment
• COUN 690 Comprehensive Examination – 100% of students Passed comprehensive exam

CACREP 5.G. School Counseling Specialty

Program Objective:

9. Students will acquire knowledge and skills for competent practice in their counseling specialty area.

Key Performance Indicator:

9.b. Students will demonstrate an understanding of the multiple roles school counselors have as leaders, advocates, and systems change agents in P-12 schools and be able to evaluate components of a comprehensive school counseling program.

KPI Measures:

• COUN 533 Paper and Project – 100% of students earned an A on assignment
• COUN 526 ASCA Paper – 100% of students earned an A on assignment
• COUN 526/528 Supervisor Evaluation – 100% of students earned a Pass on assignment

CACREP 5.A. Addiction Counseling Specialty

Program Objective:

9. Students will acquire knowledge and skills for competent practice in their counseling specialty area.

Key Performance Indicator:

9.a. Students will identify roles, theories, and models related to addiction counseling and apply empirically supported approaches to work with clients.

KPI Measures:

• COUN 545 Exam or Paper – 100% of students earned an A on assignment
• COUN 526 Clinical Assessment and Treatment Planning Paper – 100% of students earned an A on assignment
• COUN 528 Supervisor Evaluation – 100% of students earned a Pass on assignment
• COUN 548 Evidenced Based Approach Paper and Presentation- 100% of students earned an A on this assignment

Summary of Findings

Overall, CPCE data for MA students indicate lower levels of knowledge in Research and Program Evaluation, Assessment and Testing, and Social and Cultural Diversity relative to other core areas. On the surveys assessing program objectives, the lowest relative scores (all still averaging above a 4.0) were in areas related to working with diverse clients, career development, and research. KPI scores on Key Assessments were generally high with most students earning As on most assignments. The most variability was seen in Group and Assessment.

PhD Program

Doctoral Professional Identity Standards and PhD Program Objectives for PhD students are assessed through Exit, Supervisor, Program Graduates, and Employer Surveys. Survey results for PhD students are shown in Table 14.

Table 14. Doctoral Program Objectives (1-5 Scale)

PhD Program Objectives	Mean
	Mid-Term Outcomes*		Long-Term Outcomes*
	Exit Survey (N = 1)	Supervisor Survey (N = 2)	Program Graduates Survey (N = 4)	Employer Survey (N = 1)
Counseling
Prepare advanced professional counselors who demonstrate clinical skills in counseling grounded in empirically supported, theory-based approaches to helping.	4.0	5.0	3.5	5.0
Supervision
Students learn theories and models of clinical supervision and have opportunities to practice ethical and culturally appropriate supervision.	5.0	5.0	4.0	5.0
Teaching
Students understand the various roles of the counselor educator related to teaching and mentoring and learn how to apply culturally sensitive, relevant, and developmental instruction within the field.	5.0	5.0	4.0	5.0
Research and Scholarship
Students will demonstrate the ability to design and implement quantitative and qualitative research and to disseminate research through professional conference presentations and publication.	4.0	5.0	4.0	5.0
Leadership and Advocacy
Prepare advanced clinicians, supervisors, and educators who serve as leaders and advocates in their respective communities.	4.0	5.0	3.75	5.0

Ph.D. in Counselor Education and Supervision – Enrolled Student Data

CACREP 6.B.1. Counseling

Program Objective:

Prepare advanced professional counselors who demonstrate clinical skills in counseling grounded in empirically supported, theory-based approaches to helping.

Key Performance Indicator:

Students will demonstrate ethical and culturally relevant proficiency in counseling practice that allows for case conceptualization, theoretical integration, and application of empirically supported approaches to helping.

KPI Measures:

• COUN 602 Theory Paper – 100% of students earned an A on assignment
• COUN 614 Case Conceptualization – 100% of students earned an A on assignment
• COUN 616 Supervisor Evaluation – 100% of students earned an A on assignment

CACREP 6.B.2. Supervision

Program Objective:

Students learn theories and models of clinical supervision and have opportunities to practice ethical and culturally appropriate supervision.

Key Performance Indicator:

Students will identify purposes, roles, and approaches to clinical supervision and demonstrate the development of a personal style of clinical supervision that incorporates attention to legal, ethical, and culturally aware practices.

KPI Measures:

• COUN 624 Supervision Paper – 100% of students earned an A on assignment
• COUN 624 Evaluation of Supervision – 100% of students earned an A on assignment
• COUN 624 Practicum Supervisees’ Evaluation of Supervisor – Student earned a 5/5 average “Course Instructor” rating.

CACREP 6.B.3. Teaching

Program Objective:

Students understand the various roles of the counselor educator related to teaching and mentoring and learn how to apply culturally sensitive, relevant, and developmental instruction within the field.

Key Performance Indicator:

Students will demonstrate knowledge of the roles and responsibilities related to educating counselors and be able to apply ethical and culturally relevant andragogy to counselor education.

KPI Measures:

• COUN 603 Instructional Theory Paper – 100% of students earned an A on assignment
• COUN 613 Observation of Teaching Evaluation – This course was not offered in 2021-2022
• COUN 626 Course Evaluations (Students) – student received evaluations of 4.9/5 which is a passing score
• COUN 626 Faculty Evaluation – 100% of students received an A on evaluation.

CACREP 6.B.4. Research and Scholarship

Program Objective:

Students will demonstrate the ability to design and implement quantitative and qualitative research and to disseminate research through professional conference presentations and publication.

Key Performance Indicator:

Students will demonstrate the ability to formulate research questions, design research methodology to investigate those questions, collect and analyze data, and disseminate results through professional conferences and peer-reviewed journals.

KPI Measures:

• COUN 620 Conference Presentation Proposal – 100% of students earned an A on assignment
• COUN 620 Journal Article Submission – 100% of students earned an A on assignment
• COUN 693 Dissertation – Dissertation in progress.

CACREP 6.B.5. Leadership and Advocacy

Program Objective:

Prepare advanced clinicians, supervisors, and educators who serve as leaders and advocates in their respective communities.

Key Performance Indicator:

Students will demonstrate knowledge and skills of effective leadership and advocacy in the counseling profession and process of educating counselors.

KPI Measures:

• COUN 610 Advocacy Event Participation – 100% of students earned an A on assignment.
• COUN 610 Leadership Paper – 100% of students earned an A on assignment.
• COUN 592 Documentation of Leadership Position – Student provided documentation of leadership roles (Pass)

Summary of Findings

Supervisor and employer data on Program Objectives indicates consistently high scores on all KPI areas. Student data on Program Objectives indicates higher scores in the areas of Supervision and Teaching and relative lower scores in the area of Counseling and Leadership and Advocacy. All doctoral KPIs collected through course assignments in 2021-2022 were satisfactory.

Graduation, Program Completion, Licensure, and Job Placement Rates

Program and university records were used to determine number of graduates and completion rates. Number of graduates and completion rates for MA and Doctoral Program students are shown in Tables 16 and 17. Licensure and job placement rates are shown by program in Table 18.

Table 15. Number of Graduates by Program

	Number of GraduatesAY 2021- 2022	Number of Graduates from Diverse Racial or Ethnic BackgroundsAY 2021-2022
School	13	2
Addiction	10	0
PhD	0	0

Table 16. Completion Rate by Program

Cohort	MA School Program		MA Addiction Program		Doctoral Program
	Completion in 3 years	Completion in 5 years	Completion in 3 years	Completion in 5 years	Completion in 3 years	Completion in 5 years
2017	94%	100%	88%	100%	100%	100%
2018	82%	82%	40%	80%	0%	–
2019	88%	–	75%	–	0%	–
Average*	88%	91%	68%	90%	33%	100%

* For MA program, average for 3-year completion is over 5 years and average for 5-year completion is over 3 years.

Table 17. NCC, Licensure, and Job Placement Rates MA Students

	NCE Pass Rate 2019 Cohort*	Licensure Rate*2017, 2019, 2021 Graduates	Job Placement as a Counselor**2017, 2019, 2021 Graduates
School	100%	88%	88%
Addiction	100%	100%	100%

*Data from NCE Report survey (n=11); **Data from Program Graduates survey (n=22; 6 Addiction and 16 School)

Summary of Findings

The masters program exceeded the target of graduating 20 students (23 students graduated in 2021-2022). The NCE pass rate for graduating students and the reported licensure rate for program graduates is high – only two students surveyed reported they were not licensed and these students were school counselors (only a certificate is required for school counselors in the state of Idaho). Most graduates are employed as counselors.

Use of Findings to Inform Program Modifications

Suggestions and modifications were reviewed during bi-monthly faculty meetings and faculty CACREP working meetings. Upon review of the program and data collected, faculty recommended the following:

1. Current and former students (via Exit Survey and Program Graduates Survey) noted the desire for more elective courses. We continue to add elective offerings to our schedule, both in the summer and during the academic year (e.g., telehealth, working with adolescents, neurofeedback, mindfulness, etc.). We also arranged our schedule of courses so that students can take opposite-cohort courses as electives (e.g., school students can take addiction and the family system as an elective or addiction students could take counseling children and adolescents as an elective) to increase elective offerings.
2. Recruiting and retaining quality faculty members is one of the highest priorities of the program. We engaged in two national searches in 2021-2022 and hired two full-time tenure-track faculty members (one started in fall 2021 and one will start in fall 2022). We also hired a full-time Clinical Assistant Professor who will start in the fall of 2021. Senior faculty will provide mentorship to junior faculty members through regular check-ins and connection to campus resources (e.g., Center for Teaching and Learning). Faculty teaching courses will also meet once a month for a “brown bag lunch” meeting to talk about teaching related issues and ideas.
3. Our cohort model and our active student organization (CSI) allow students opportunities to build close and supportive relationships with peers. COVID-related restrictions that forced classes to go online and limited group gatherings inhibited the building of these relationships in the last two years. In the year ahead, we plan to reenergize CSI (e.g., bring back the summer community building activity) and reinstitute many of the cohort-building activities that we used to do in prior years (e.g., first year family and friends picnic). We will continue to brainstorm other ways to help students create positive connections with one another, program faculty, and staff.
4. We added a 1-credit required course for second year students to help them prepare for Internship. This course was added in response to student feedback that they would like more structured support in helping locate, evaluate, and secure internship sites. This course can also strengthen students’ perception of advising support, as it will provide in-person, scheduled time to discuss important program related concerns and questions. Faculty teaching this course will also provide an introduction to certification and licensure information. Students cover this information again in third year orientation and in their third-year Seminar course, but hopefully adding an additional time point will help students feel more supported in this area.
5. Faculty reviewed the current assessment and evaluation plan and agreed that it is satisfactory. Of note, however, was the continued low level of employers responding to the employer survey. Faculty will discuss ways to increase response rates prior to the next survey administration such as a raffle for a prize.
6. Faculty reviewed the process of evaluating KPI’s and Program Objectives. Faculty developed and adopted rubrics to evaluate students individually as well as collective on all program KPIs. Faculty agreed that the assessment of MA and Doctoral Program Objectives is satisfactory.
7. Faculty reviewed the process of using the PPADD to monitor students’ professional, personal, and academic development and agreed it is satisfactory.
8. Faculty reviewed enrollment trends. Faculty noted an increased in MA applications compared to last year, reflecting more typical (pre-COVID) application numbers.
9. Faculty reviewed graduation and retention rates. The MA program exceeded the target of graduating 20 students (N = 23). Faculty discussed the current retention plan and agreed to continue current retention activities. Faculty noted there was no attrition among ethically/racially diverse students.
10. We reviewed our program Mission, Strategic Objectives, and Program Goals. Faculty agreed that our mission reflects our program goals and aspirations. We made minor revisions to program objectives and identified areas of focus for the year. We also made minor revisions to program goals.
11. Based on curriculum review and survey data, faculty will consider and/or make the following revisions to the curriculum:

• • Survey data continue to indicate knowledge in Career as a relative area of weakness relative to other core courses. We will continue to monitor scores in the Career section of CPCE. For fall 2022, we selected an instructor for the course who is career counselor with many years of experience and has received high teaching evaluation scores in previous courses taught for our program.
  • Survey data continue to indicate knowledge in Assessment and Measurement and Research and Program Evaluation as relative areas of weakness compared to other core courses. We selected a core faculty member to teach these two courses. The faculty member has a passion for research and assessment, has an active research agenda, and has an advanced certificate in research. We hope having such a highly qualified instructor for these two courses, and having the same instruction teach these courses over time will increase students’ perceived learning.
  • Faculty reviewed the Doctoral program curriculum and outcomes. No changes were made.