Skip to main content

Current Quality Assurance Review

Download FY2017 Quality Assurance Review Independent Validation Report (PDF)

Internal Audit and Advisory Services Independent Validation of Quality Assurance Self-Assessment

November 23, 2016


  • Betsy Bowers, CIA, CFE, CGFM, CIG, CRMA, CICA
  • Kathy Burgmeier, CICA
  • Mr. Don Soltman, Audit Committee Chair, Idaho State Board of Education Dr. Robert Kustra, President, Boise State University

This Validation of the Self- Assessment of the Boise State Internal Audit Program was performed in accordance with The Institute of Internal Auditors (IIA) Quality Assessment Manual, 2013 Edition. The primary purpose of a Quality Assessment is to determine the internal audit function’s conformance with the International Standards for the Professional Practice of Internal Auditing. There are three possible outcomes of the QA: the internal audit program generally conforms, partially conforms or does not conform to the Standards.

Mr. Don Soltman, Audit Committee Chair, Idaho State Board of Education
Dr. Robert Kustra, President, Boise State University
Mr. Larry Harmon, Executive Director, Internal Auditing and Advisory Services, Boise State University


We were engaged as the validators to conduct an independent Validation of the Self-Assessment Quality Assessment

(QA) of the Boise State University Internal Audit Program as required every five years by the Institute of Internal Auditors International Standards for the Professional Practice of Internal Auditing (IIA Standards). The objectives of the QA were to:

  • Assess conformance with the IIA Standards;
  • Assess the effectiveness and efficiency of the Internal Audit activity in providing services to the Idaho State Board of Education Audit Committee and management of Boise State University; and
  • To identify opportunities for improving the Internal Audit Program at Boise State University.

In acting as independent validators, we are fully independent of Boise State University and have the necessary knowledge and skills to undertake this engagement. The Validation, conducted October 24-26, 2016, consisted primarily of reviewing and testing the self-assessment documentation related to the Boise State Office of Internal Audit and Advisory Services (IAAS) self-assessment report issued September 15, 2016. Additionally, we interviewed other audit team members and several key administrators. These interviews helped us gain a better understanding of the internal control environment within which Boise State University internal auditing operates.

Overall, it is our opinion that the Boise State University IAAS generally conforms to the IIA Standards, the highest rating available. We noted one opportunity that could improve the efficiency and effectiveness of the internal audit program, described in this report. We have reviewed the results of the validation with Mr. Larry Harmon, Executive Director, Boise State IAAS.

Respectfully submitted,

Associate Vice President Internal Auditing & Compliance
University of West Florida
Pensacola, FL
Team Lead

Kathy Burgmeier, CICA Director of Internal Audit
The University of Montana
Missoula, MT

Executive Summary

The internal auditing function has a dual reporting organizational structure, functionally to the Idaho State Board of Education Audit Committee and administratively to Boise State University’s (BSU) president.

To be compliant with international standards promulgated by the Institute of Internal Auditors International Standards for Professional Practice of Internal Auditing (Standards) Boise State University internal audit activity is required every five (5) years to have an assessment to ascertain compliance with these Standards and appraise the quality of operations. The method used was a self-assessment with independent validation.

Other matters that might have been covered in a full independent assessment, such as an in-depth analysis of successful practices, governance, consulting services and use of advanced technology, were excluded from the scope of this independent validation by agreement with Mr. Larry Harmon, Executive Director, IAAS (such exclusions are standard for an independent validation). We have communicated to Mr. Harmon suggestions for minor improvements. We believe these improvements, if implemented, will further add value to the internal audit services already provided.

Boise State University’s IAAS is a respected internal audit operation among its higher education peers. The extent of their communication with the Idaho State Board of Education’s (SBOE) Audit Committee seems appropriate and strong relationships appear to exist between the Executive Director and the Idaho SBOE Audit Committee as well as BSU’s senior administration.


Appendix IV is a maturity model developed by the IIA that is designed for commercial enterprises. It is presented for informational purposes only. There are other models perhaps more suited to the University’s circumstances that could be used as a basis for discussion with senior management and the Idaho SBOE Audit Committee about BSU’s internal auditing program’s current and desired future state.

Conformance with IIA Standards

Conformance Definitions

Generally Conforms means that internal audit has a charter, policies and processes that are judged to meet the spirit and intent of the IIA Standards with some potential opportunities for improvement.

Partially Conforms means deficiencies in practice are noted that are judged to deviate from the spirit and intent of IIA Standards, but these deficiencies did not preclude internal audit from performing its responsibilities in an acceptable manner.

Does Not Conform means deficiencies in practice are judged to be so significant as to seriously impair or preclude internal audit from performing adequately in all or in significant areas of its responsibilities.

Overall Rating for Boise State

Overall, the Boise State University Internal Audit Program was judged to Generally Conforms to IIA Standards, the highest rating available. While minor improvement opportunities remain in various areas, they did not preclude this assessment. We concluded the following individual standards Generally Conform to the IIA Standards.

  • 1000—Purpose, Authority and Responsibility
  • 1100—Independence and Objectivity
  • 1200—Proficiency and Due Professional Care
  • 1300—Quality Assurance and Improvement Program
  • 2000—Managing the Internal Audit Activity
  • 2100—Nature of Work
  • 2200—Engagement Planning
  • 2300—Performing the Engagement
  • 2400—Communicating Results
  • 2500—Monitoring Progress
  • 2600—Resolution of Senior Management’s Acceptance of Risk

While we found Boise State University in conformance with all the Standards, we did identify one Opportunity for Operational Improvement.

Positive Attributes of the Boise State University Internal Audit Program

  • Governance—Boise State University Internal Auditing & Assurance Services (IAAS) Executive Director has dual reporting to the Idaho State Board of Education’s Audit Committee and BSU President. Executive sessions are held between the CAE and SBOE Audit Committee, which evidences independence. Additionally, the Executive Director meets periodically with the President to discuss internal audit operations.
  • Idaho State Board of Education and Senior Management Support—Those interviewed conveyed a high level of support for the BSU internal audit program from the SBOE Audit Committee and BSU senior management. The internal audit program is well respected and seen as collaborative. Management feels comfortable seeking their opinion of problematic situations.
  • Working Paper Documentation— Apart from the final audit report, workpapers represent the main documentary evidence of audit testing, discussions, and observations. While the organization, design, and content of audit workpapers will vary depending on the nature of the engagement, several important considerations are necessary to the creation of effective, high-quality workpapers. Completeness, accuracy and organization are key attributes of working papers. At BSU IIA the audit documentation was very thorough, well written and contained these key attributes. Standardization of working papers was found to be efficient and facilitated expedient work.
  • High-Impact Student Experiences—BSU IAAS is sincere in helping support the university’s mission to provide high-impact student experiences. In 2013 the internal audit internship program was revitalized in partnership with the College of Business allowing one to two students to work in internal auditing in fall and spring semester. Many interviewed cited this as a noteworthy endeavor. Student interns work approximately 15 hours per week for 11 weeks at an hourly rate comparable to a local CPA firm. The students assist with all aspects of a current audit engagement or advisory review. The students sign the IIA Ethics Code and are taught best practices for conducting audit work. To date, there have been ten interns who have worked in internal auditing.

Opportunity for Operational Improvement—Internal Audit and Advisory Services Department

{Boise State University IAAS conforms to the IIA Standards noted below. The item presented is a suggested enhancement to the internal auditing operations.}

Follow-Up: Internal audit reports contain management’s intended corrective actions in response to an audit finding. However, reports lacked two pieces of information: management’s intended implementation date and the responsible auditee. Furthermore, the IAAS internal procedures do not specify when an audit issue should be elevated to a higher level based on the amount of time that the audit recommendation has been outstanding or that management has been nonresponsive.

The lack of a formal agreed-upon implementation date and responsible auditee increases the risk that audit recommendations will not be implemented and complicates the audit follow-up process. We feel this weakens the internal controls at BSU. Delays in timely implementation

of audit recommendations or formally documenting management has accepted the risk of not implementing the recommendation, indicates a red flag regarding internal controls. Best practices in higher education generally state that implementation dates should be set within a 6 month period after the audit report date.

IIA Standards state:

2500 Monitoring “The chief audit executive must establish and maintain a system to monitor the disposition of results communicated to management.”

2500. A1 “The chief audit executive must establish a follow-up process to monitor and ensure that management actions have been effectively implemented or that senior management has accepted the risk of not taking action.”

2600 Communicating the Acceptance of Risks “When the chief audit executive concludes that management has accepted a level of risk that may be unacceptable to the organization, the chief audit executive must discuss the matter with senior management. If the chief audit executive determines that the matter has not been resolved, the chief audit executive must communicate the matter to the board.”


BSU IAAS should include in their audit reports: a) management’s intended implementation date and b) a responsible auditee. This will give a concrete timetable for implementation and help establish a timely follow up process that is designed to update the status of corrective actions implemented in response to IAAS’s findings/recommendations. Protocol should also be established for when management does not provide or follow the implementation date. This tool should be leveraged by IAAS in assessing and reporting on all audit issues (open and closed). In situations where management accepts the risk of not implementing an audit recommendation, this should be documented by management and provided to BSU IAAS.


The BSU IAAS Executive Director has implemented new procedures that require management responses to each observation and recommendation in audit reports to include the responsible employee for implementing the corresponding corrective action, the responsible supervisor for reviewing and approving the corrective action for adequacy and effectiveness, and the date that the corrective action will be implemented. This new procedure is effective for all audit reports issued in FY 2017.

Additionally, the BSU IAAS Executive Director will implement procedures to address situations when management does not provide or follow implementation dates by escalating these issues with the applicable Vice Presidents and the President. These procedures are currently being drafted, will be formally reviewed with the Vice Presidents and President at the February 1, 2017, Administrative Council meeting, and finalized and implemented by February 28, 2017.


Appendix I – Engagement Methodology

Review procedures included:

  • Audit charter and other background/organizational materials regarding BSU and the internal audit program
  • Idaho State Board of Education Audit Committee charter
  • Boise State University Policies and IAAS charter
  • Boise State University Internal Auditing and Advisory Services Operating Manual
  • Prior quality assessment (QA) reports
  • QA advance preparation materials providing background on the internal auditing program and practices
  • Annual audit plan, annual reports, midyear reports, and risk assessment process
  • Selected internal audit project work papers and reports
  • Training histories for staff
  • Audit follow-up practices and reporting of follow-up activities

Interviews included:

  • President of Boise State University
  • Idaho State Board of Education President and former Audit Committee Chairperson
  • Idaho State Board of Education leadership
  • Boise State University officials

Appendix II – List of Stakeholders Interviewed

State Board of Education

  • Ms. Emma Atchley, Idaho SBOE, President and former Audit Committee Chair
  • Mr. Chet Herbst, Idaho SBOE, Chief Financial Officer
  • Mr. Scott Christie, Idaho SBOE, Financial/Performance Audit Manager

Senior BSU Management

  • Dr. Robert Kustra, President
  • Dr. Marty Schimpf, Provost/Vice President for Academic Affairs
  • Ms. Stacy Pearson, Vice President for Finance and Administration
  • Dr. Leslie Webb, Vice President Student Affairs
  • Dr. Mark Rudin, Vice President Research
  • Mr. Kevin Satterlee, Chief Operating Officer, Vice President, and Special Counsel
  • Ms. Randi McDermott, Chief of Staff
  • Mr. Max Davis-Johnson, Associate Vice President Information Technology
  • Mr. Doug Ooley, Information Security Officer
  • Ms. Alicia Estey, Executive Director Institutional Compliance
  • Ms. Jo Ellen DiNucci, Associate Vice President for Finance and Administration
  • Mr. John Kaplan, Executive Director Campus Security & Police
  • Mr. Curt Apsey, Athletic Director
  • Mr. Max Corbet, Associate Athletic Director Mr. Kip McBean, Risk Manager
  • Ms. Karen Henry, Executive Director, Office of Sponsored Programs

Boise State Internal Audit Staff

  • Mr. Larry Harmon, Executive Director
  • Mr. Mark Eisenman, Internal Audit Manager Ms. Danielle Ricco, Internal Auditor
  • Ms. Rene Belleque, Internal Auditor

Appendix III—Institute of Internal Auditing Standards

Quality Assessment Evaluation

  • Overall Evaluation: GENERALLY CONFORMS

For each Standard identified below, Boise State University Internal Auditing Services GENERALLY CONFORMS (highest rating)

Attribute Standards
1000 Purpose, Authority and Responsibility
  • 1010 Recognition of the Definition of Internal Auditing, the Code of Ethics, and the Standards in the Internal Audit Charter
1100 Independence and Objectivity
  • 1110 Organizational Independence
  • 1111 Direct Interaction with the Board
  • 1120 Individual Objectivity
  • 1130 Impairment to Independence of Objectivity
1200 Proficiency and Due Professional Care
  • 1210 Proficiency
  • 1220 Due Professional Care
  • 1230 Continuing Professional Development
1300 Quality Assurance and Improvement Program
  • 1310 Requirement of the Quality Assurance and Improvement Program
  • 1311 Internal Assessments
  • 1312 External Assessments
  • 1320 Reporting on the Quality Assurance and Improvement Program (QAIP)
  • 1321 Use of “Conforms with the International Standards for the Professional Practice of Internal Auditing”
  • 1322 Disclosure of Nonconformance
Performance Standards
2000 Managing the Internal Audit Activity
  • 2010 Planning
  • 2020 Communication and Approval
  • 2030 Resource Management
  • 2040 Policies and Procedures
  • 2050 Coordination
  • 2060 Reporting to Senior Management and the Board
  • 2070 External Service Provider and Organizational Responsibility for Internal Auditing
2100 Nature of Work
  • 2110 Governance
  • 2120 Risk Management
  • 2130 Control
2200 Engagement Planning
  • 2201 Planning Consideration
  • 2210 Engagement Objectives
  • 2220 Engagement Scope
  • 2230 Engagement Resource Allocation
  • 2240 Engagement Work Program
2300 Performing the Engagement
  • 2310 Identifying Information
  • 2320 Analysis and Evaluation
  • 2330 Documenting Information
  • 2340 Engagement Supervision
2400 Communicating Results
  • 2410 Criteria for Communicating
  • 2420 Quality of Communications
  • 2421 Errors and Omissions
  • 2430 Use of “Conducted in Conformance with the International Standards for the Professional Practice of Internal Auditing”
  • 2431 Engagement Disclosure of Nonconformance
  • 2440 Disseminating Results
  • 2450 Overall Opinions
2500 Monitoring Progress
2600 Communicating the Acceptance of Risks
  • The IIA’s Code of Ethics

Appendix IV – The IIA Research Foundation – Internal Audit Capability Model Matrix

Level Services and Roles of IA People Management Professional Practices Performance Management and Accountability Organizational Relationships and Culture Governance Structures
Level 5 – Optimizing IA Recognized as Agent of Change Key Leadership Involvement with Professional Bodies; Workforce Projection Continuous Improvement in Professional Practices; Strategic IA Planning Public Reporting of IA Effectiveness Effective and Relationships Ongoing Independence, Power, and Authority of the IA Activity
Level 4 – Managed Overall Assurance on Governance, Risk Management and Control IA Contributes to Management Development; IA Activity Supports Professional Bodies; Workforce Planning Audit Strategy Leverages Organization’s Management of Risk Integration of Qualitative and Quantitative Performance Measures CAE Advises and Influences Top-level Management Independent Oversight of the IA Activity; CAE Reports to Top-level Authority
Level 3 – Integrated Advisory Services; Performance/Value-for- Money Audits Team Building and Competency; Professionally Qualified Staff; Workforce Coordination Quality Management Framework; Risk-based Audit Plans Performance Measures; Cost Information; IA Management Reports Coordination with Other Review Groups; Integral Component of Management Team Management Oversight of the IA Activity Funding Mechanisms
Level 2 – Infrastructure Compliance Auditing Individual Professional Development; Skilled People Identified and Recruited Professional Practices and Processes Framework; Audit Plan Based on Management/Stakeholder Priorities IA Operating Budget; IA Business Plan Managing within the IA Activity Full Access to the Organization’s Information, Assets and People; Reporting Relationship Established
Level 1 – Initial Ad hoc and unstructured; isolated single audits or reviews of documents and transactions for accuracy and compliance; outputs dependent upon the skills of specific individuals holding the position; no specific professional practices established other than those provided by professional associations; funding approved by management as needed; absence of infrastructure; auditors likely part of a larger organizational unit; no established capabilities; therefore, no specific key process areas

Appendix V – Biography of Reviewers

Betsy Bowers

Betsy Bowers is the Associate Vice President/Chief Audit Executive for Internal Auditing and Compliance at the University of West Florida in Pensacola, Florida. Ms. Bowers is a past national president of the Association of College and University Auditors (ACUA). Ms. Bowers has been at UWF since 1993 and served as the chief audit executive during the entire time. In 2014/15, Ms. Bowers served as the interim vice president for Business, Finance, and Facilities at UWF. Previously, Betsy worked in Tennessee as the Internal Audit Director at Northeast State Technical Community College and Internal Auditor at East Tennessee State University resulting in a total higher education work experience exceeding 30 years. She completed the Institute of Internal Auditor’s (IIA) Quality Assurance training and performed Quality Assurance Reviews at numerous other universities across the nation. Ms. Bowers is a Certified Internal Auditor (CIA), Certified Fraud Examiner (CFE), Certified Government Financial Manager (CGFM), Certified Inspector General (CIG), a Certificate in Risk Management Assurance (CRMA), a Certified Internal Controls Auditor (CICA), and earned her BA in accounting and MBA from East Tennessee State University. She serves on the Board for the Northwest Florida Institute of Internal Auditors, the Board for the Northwest Florida Association of Certified Fraud Examiners (ACFE), and as ACUA Awards Committee chairperson. She has published articles in ACUA’s “College and University Auditor,” the IIA’s “Internal Auditor,” the Association of Local Governmental Accounts (ALGA) “Local Government Auditing Quarterly,” and is a national speaker/presenter. Other professional memberships include the Association of Governmental Accountants, Southern Association of College & University Auditors (SACUBO), Society for Corporate Compliance and Ethics (SCCE), Association of College & University Women, and several civic organizations. Ms. Bowers taught White Collar Crime for UWF in the Justice Studies department and serves as an instructor for the IIA on the topics of new internal auditor and new audit manager.

Kathy Burgmeier

Kathy Burgmeier is the Director of Internal Audit at the University of Montana in Missoula, Montana. Ms. Burgmeier has been at the University of Montana since 1981 working in payroll, student loans, accounts payable, internal audit, budget & planning, and has served as the Director of Internal Audit since 1991. Ms. Burgmeier served the Association of College and University Auditors (ACUA) as a Board Member, Nominations Committee and Midyear Chairperson for several years. She also received ACUA’s Member Excellence Award. Ms. Burgmeier performed Quality Assurance Reviews at several other universities. Ms. Burgmeier is a Certified Internal Control Auditor (CICA) and earned her bachelor’s degree from the University of Montana in Business Administration with an Accounting emphasis. Ms. Burgmeier has been a guest lecturer and presenter about Internal Auditing, Fraud and Internal Controls at the University of Montana, Beta Alpha Psi, Montana Society for Certified Public Accountants, and Pacific Northwest Higher Education Internal Audit Conferences.


The members of the Review Team appreciate the opportunity to be of service to Boise State University and the courtesies and cooperation extended to us during this review.