Using Assessment to Guide Dynamic Academic Programs
As a small department, Anthropology has been using assessment as a tool to continuously gather empirical data and other measures to answer questions about their students’ learning, experiences, and the paths they take after earning their degree at Boise State. This brief case provides a glimpse into the way that this department, which sits in the heart of the Arts and Sciences, embraces assessment data in its various forms to inform change.
Background
In 2005 assessment expert Barbara Walvoord visited Boise State. John Ziker, then a new professor in the department, Walvoord’s “clear and simple” approach to draw upon what we do every day as faculty and departments to support our assessment questions. The department approaches assessment as a “committee of the whole” to review data and institute curricular and other changes based on assessment findings.
Program Learning Outcomes and Assessment Methods
Anthropology posts its four (4) undergraduate learning outcomes on its website. Each of these learning outcomes is broadly written with related sub-outcomes listed beneath it. Having recently revised these outcomes as a WIDER Persist project, the outcomes are streamlined into groups that are covered through program curricula.
Like the PLOs, the curriculum map is posted on the department website so that students, faculty and advisors can easily refer to them. According to Ziker, advisors and faculty currently use the learning outcomes in conversations with new students and are considering building the curriculum map into advising meetings with upper division students as part of their indirect assessment methods.
The department has been using final portfolios as the centerpiece of its assessment efforts for decades. These binders, submitted by students in their final year, were collected during the fall semester, evaluated in person by the faculty beginning at Thanksgiving, and discussed in a senior interview with each student prior to graduation. They have a departmental rubric which guides the faculty review and helps to standardize the feedback to the students and for the department. The process, until recently, was labor intensive and restrictive since the binders needed to remain in the department and available to multiple faculty simultaneously. The faculty used these portfolios to consider student progress toward program goals and they served as the basis for curricular and programmatic changes considered by the faculty.
In 2015 Anthropology shifted to using Digication e-Portfolios. The online format offered a number of benefits: the faculty could review portfolios on their own schedule and anywhere, the department was able to “collect” e-portfolios in both fall and spring, and the new tool made reading and responding to students much easier and less time consuming. On the other hand, faculty have noted that the number of artifacts that students add to their e-Portfolio has declined. They seemed more inclined to save conference fliers or event handouts and add them to their paper portfolio. It is not clear what has triggered this change and better understanding it might be an area of exploration at some point. As a result of the tendency to include less in the e-Portfolio, the department is looking at ways they can change or increase messaging about building the e-portfolio with their current students.
Another innovative assessment effort conducted by the department that has been beneficial was a study of alumni LinkedIn profiles. Ziker explained that many of the graduates connect to him through LinkedIn. Through that platform, he was able to view alumni job titles and then connect those titles to the Department of Labor employment category codes. In addition, the department has been increasing its efforts to collect “testimonials.” These brief snapshots of Anthropology graduates are featured prominently on the department website to tell the story about exciting paths that graduates have taken. In addition to providing excellent public relations, testimonials and LinkedIn pages contribute a rich source of indirect data for the program. In fact, these sources and others contributed to the decision to launch a new 12 credit hour online certificate in Design Ethnography, a collaborative endeavor between Anthropology and the College of Innovation and Design.
Connecting Outcomes and Curriculum
In 2012, with the introduction of the University Learning Outcomes and the Foundational Studies Program, Anthropology made adjustments to their curriculum to align the coverage of written and oral communication, critical thinking, and teamwork. New Communication in the Disciplines (CID) and Finishing Foundations (FF) courses serve as the home for content and skills to meet these university learning outcomes which have corollaries in the Anthropology outcomes. For example, while the term paper remains a signature assignment in the Finishing Foundations course, the department added peer review in the CID and FF courses to support the development of teamwork strategies.
The CID and FF courses also provide scaffolding for the program to build in student reflection on the PLOs and the way these outcomes prepare students for life after graduation. They also begin priming students to provide feedback to the department through the graduating student survey and alumni surveys. Collecting this end of program indirect assessment is a key to the program assessment plan for the department. Anthropology has emphasized collecting narratives or testimonials from their students to be able to share with future students and to tell the story of what can be done with an anthropology degree. The testimonials, which are linked to the department website, serve several purposes. They provide a narrative about possible paths for new students, they celebrate the accomplishments of alumni, and they inform faculty conversations about curricular changes. These data contribute to the courses and program in general.
Another department activity that both generates assessment data and supports collecting data is the annual majors meeting. This meeting provides an opportunity for the students and faculty to interact annually, for the department to share information about events and activities, and serves as a check in time for students each year.
As the program moves into its 2017 Program Assessment Reporting cycle, it will revisit the outcomes and assessment plans that it has put into place since its last program review in 2011. From there, the faculty will determine its next steps. Questions about the ways that Anthropology continues to collect and evaluate assessment data can be directed to Dr. John Ziker, Chair, Department of Anthropology.