Performance audit of the North Carolina AIG program

Posted Sunday, April 6, 2008

 E-mail this page    Printer-friendly page

Raleigh, NC - In February, State Auditor Leslie Merritt released an audit of the Department of Public Instruction’s (DPI) Academically or Intellectually Gifted program (AIG). Below is a majority of that report.

PERFORMANCE AUDIT
DEPARTMENT OF PUBLIC INSTRUCTION - EXCEPTIONAL CHILDREN DIVISION
ACADEMICALLY OR INTELLECTUALLY GIFTED PROGRAM
FEBRUARY 2008
OFFICE OF THE STATE AUDITOR
LESLIE W. MERRITT, JR., CPA, CFP
STATE AUDITOR

PURPOSE

This audit report identifies weaknesses in the monitoring and evaluation of the Academically or Intellectually Gifted (AIG) Program and makes recommendations so Department of Public Instruction (DPI) management can take appropriate corrective action.

RESULTS

The AIG Program is not monitored at the State level to ensure that local programs are operated according to documented plans. Also, DPI does not monitor local AIG Program expenditures or determine the quality and effectiveness of local programs. As a result, AIG programs may not be operated as planned and academically or intellectually gifted students may not receive the differentiated educational services they need to excel in the classroom.

We conducted a survey to determine parent satisfaction with local AIG programs. Although the majority of parents who responded to our survey were satisfied with the AIG Program, a significant number believed their child did not receive the services intended by the AIG Program, saw no difference in the type of educational services provided to their child since enrolling in the AIG Program, or recognized no improvement in their child’s academic performance since participating in the AIG Program. A significant number of parents also said they were not provided information about their child’s AIG progress by the school.

State-level monitoring has not been established because the law does not clearly authorize DPI to monitor local AIG Program implementation and spending. Current law only requires Local Education Agencies (LEA) to consider comments and recommendations from the State Board of Education and DPI.

RECOMMENDATIONS

The State Board of Education should establish or clarify authority regarding State-level monitoring of the AIG Program. DPI should develop and implement policies and procedures for State-level monitoring of the AIG Program. DPI should require evidence that differentiated educational services were delivered and AIG funds were used as outlined in local plans. DPI should also establish requirements for demonstrating that appropriate services have been provided before AIG funds are allowed to be used for other purposes.

DPI should establish AIG Program performance standards. The standards should provide the State with an evaluation tool that allows comparison of AIG services across schools and districts statewide. DPI should collect performance data from all Local Education Agencies (LEA) and evaluate the quality and effectiveness of the AIG Program statewide. A comprehensive analysis of data from all 115 LEAs could identify best practices among the individual school systems and result in overall program improvement. Statewide program
evaluation should be performed regularly, and the results should be readily available for public inspection. Parents should be made aware of the standards and provided a method to comment on program results.

AGENCY’S RESPONSE

The Agency’s response is included in Appendix B.

INTRODUCTION

BACKGROUND

In 1996, the Academically or Intellectually Gifted (AIG) Program was separated from its categorization with other special needs children by Article 9B of the North Carolina General Statutes. Article 9B defines AIG students as those that exhibit high academic performance capabilities, but require differentiated educational services beyond those provided by regular educational programs.

The AIG Program is administered through the Exceptional Children Division of the Department of Public Instruction (DPI). The division provides guidance to the 115 Local Education Agencies (LEA) concerning exceptional children’s programs, including the AIG Program. State law requires LEAs to submit local AIG plans to the State Board of Education every three years. DPI provides guidance to the LEAs about the State Board of Education guidelines, reviews local plans, and provides commentary as necessary. An organizational chart showing the lines of authority over the AIG Program is located in Appendix A.

In 2007, 155,221 students were enrolled in North Carolina AIG programs statewide, with a budget of $58.1 million. Current child count data will not be available until April 2008, but the 2008 budget amount rose to $63.3 million. As AIG budget amounts are based on enrollment figures, State spending on the AIG Program will continue to increase as the student population increases.

OBJECTIVE, SCOPE, AND METHODOLOGY

The objective of the audit was to determine whether the Exceptional Children Division of the Department of Public Instruction (DPI) adequately monitors the use of State funding for the Academically or Intellectually Gifted (AIG) Program.

This audit was initiated by the Office of the State Auditor in response to parent concerns that AIG Program funds are being used for other purposes while AIG students are left underserved.

The scope of this audit includes AIG policies and procedures, State laws, and financial and enrollment data maintained by DPI for fiscal years 2003-2008. The scope also includes AIG data maintained by Local Education Agencies (LEA) and information from parents of AIG students enrolled during the 2005-06 school year.

To accomplish our objective, we reviewed State laws, AIG policies and procedures, and State Board of Education guidelines governing administration of the AIG Program. Data were gathered from DPI and LEAs to support AIG funding and expenditure amounts. We also requested LEAs to submit documentation supporting the management of their AIG programs, and DPI personnel and LEA administrators were contacted as necessary through interview, email and telephone.

We conducted a parent satisfaction survey, in which we selected a random sample of 600 students enrolled in AIG programs. We acquired their parents’ names and addresses and mailed a survey to each. Due to the low response rate (24%), the results of our survey may not be representative of all AIG parents. Consequently, we do not project our results to the population. All statements on the results of our survey pertain to the 146 respondents only.

We conducted this performance audit according to generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence that provides a reasonable basis for our findings and conclusions based on our audit objective. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objective.

This report contains the results of the audit including conclusions and recommendations. Specific recommendations related to our audit objective are reported. Because of the test nature and other inherent limitations of an audit, together with the limitations of any system of internal and management controls, this audit would not necessarily disclose all weaknesses in the systems or lack of compliance.

We conducted the fieldwork from May 2007 to January 2008. We conducted this audit under the authority vested in the State Auditor of North Carolina by Section 147-64.6 of North Carolina General Statutes.

LOCAL AIG PROGRAM SERVICE DELIVERY AND SPENDING ARE NOT MONITORED AT THE STATE LEVEL

The Academically or Intellectually Gifted (AIG) Program is not monitored at the State level to ensure that local programs are operated according to documented plans. Also, the Department of Public Instruction (DPI) does not monitor local AIG program expenditures or determine the quality and effectiveness of local programs. As a result, AIG programs may not be operated as planned, and academically or intellectually gifted students may not receive the differentiated educational services they need to excel in the classroom.

Delivery of Planned AIG Services Is Not Monitored at the State Level

The AIG Program is not monitored at the State level to ensure that local programs are operated according to documented plans. Each of the 115 Local Education Agencies (LEA) is required to develop an AIG program plan based on guidelines from the State Board of Education. The guidelines were developed to “serve as suggestions for best practices” and to “provide some statewide consistency in the education of academically or intellectually gifted students.” The local plans must be submitted to the State Board of Education for review and comment every three years. The State Board of Education guidelines require DPI to determine if LEAs are providing planned AIG services. However, once the plans have been reviewed and returned to the LEAs, with comments if applicable, no further monitoring is performed at the State level to ensure local programs are operated in accordance with the final plans.

Consequently, AIG programs may not be operated as planned, and academically or intellectually gifted students may not receive legislatively intended AIG services. AIG programs are necessary to ensure that academically or intellectually gifted students receive differentiated educational services to develop and realize their potential. The State currently does not have monitoring procedures in place to ensure that the appropriate quality and quantity of differentiated educational services are delivered at an appropriate frequency to achieve this goal.

Local AIG Program Expenditures Are Not Monitored at the State Level

Parents have expressed concerns that AIG funds are being used for other purposes while AIG students are left underserved. Parents have asserted that, as schools are faced with tough budget decisions, school administrators are using AIG funds to address other needs viewed as higher priority by the schools or to fill budget gaps. We reviewed documentation for AIG fund transfer requests on file at DPI for the 2006 and 2007 fiscal years. We determined how much of the total AIG allocation was transferred from each LEAs’ AIG fund. The results are listed in Table 1 below.

Table 1 – LEA1 Transfers from AIG Fund

Total Amount Percent
AIG Transferred Transferred
Budget Out (In) Out (In)

2006
LEA 1 $498,833 $404,515 81%
LEA 2 $171,606 $167,825 98%
LEA 3 $63,997 ($4,872) (8%)

2007
LEA 1 $526,554 $384,640 73%
LEA 2 $182,349 $182,187 99%
LEA 4 $339,182 $500 <1%

Source: DPI

For LEA #1 and #2, the stated purpose of the requests was to allow more flexibility in planned AIG spending. Both of these LEAs used AIG funding to pay salaries and benefits to non-AIG certified teachers. LEA #1 provided a statement that the AIG funds were transferred to pay salaries and benefits for “general education teachers who fall in the lower income bracket” that, “will be responsible for providing some of the differentiated services for identified AIG students.” LEA #2 asserts that its transfers qualify under the provisions of its School Improvement Plan and State Board of Education guidelines, adding that AIG funds were transferred “in order to receive more benefits from all funding sources.” By contrast, LEA #3 shows a county that transferred funds into the AIG program, and LEA #4 transferred out a small amount of AIG funds for classroom supplies.

Table 1 and the related explanations illustrate an environment where AIG funds are managed in significantly different ways among LEAs. North Carolina General Statutes allow AIG funds to be used “in accordance with an accepted school improvement plan, for any purpose so long as that school demonstrates it is providing appropriate services to academically or intellectually gifted students in accordance with the local plan.” Certainly, budget flexibility and educational autonomy are vital to an LEA’s ability to provide educational services to its entire student population. But, in the absence of State monitoring of spending and program delivery, LEAs have the ability to transfer AIG funds out of the program based on documented plans only. At the State level, no one determines that the funds were actually spent to provide appropriate services to AIG students, and no one determines if intended results of the AIG program are achieved. LEAs are left to self-monitor, which increases the risk that any mismanagement of State dollars would go undetected.

1 LEA 1 transferred AIG funds both years, as did LEA 2. LEA 3 transferred funds only in 2006, and LEA 4 only in 2007. Four of the 115 LEAs submitted transfer requests to DPI for 2006 – 2007.

We requested documentation from all 115 LEAs in order to gauge availability of information that could be used to monitor the local programs. We received documentation from 114 of the LEAs, but found the information differed significantly in quantity, form and content. Some LEAs provided detailed support for their AIG expenditures, as well as data supporting program results, while others submitted general information or none at all.

DPI Does Not Monitor Local AIG Program Evaluations

DPI does not ensure that the quality and effectiveness of local AIG programs are properly evaluated or that the results are publicly reported. Also, DPI has not established statewide performance standards to evaluate local AIG program quality and effectiveness. North Carolina General Statute 115C-150.7 requires local AIG plans to include “Measurable objectives for the various services that align with core curriculum and a method to evaluate the plan and the services offered. The evaluation shall focus on improved student performance.” DPI reviews the local plans to ensure this component is included, but no one at the State level follows up to ensure that the evaluation is performed or if student performance is improving. Local Education Agencies (LEA) are required to develop performance data, but no one is currently evaluating the quality and effectiveness of the AIG Program statewide. Furthermore, our request for LEA documentation, mentioned earlier, disclosed that at least some LEAs are not developing performance data.

Consequently, AIG Program quality and effectiveness may not be consistent across North Carolina schools. Delivery of AIG services varies significantly among school systems; some school systems have dedicated AIG instructors who conduct AIG classes separate from regular classes, while high schools may simply use advanced placement (AP) courses to deliver AIG services. In the absence of approved standards and statewide monitoring and evaluation, DPI cannot provide assurance that AIG programs across the State are meeting expected, or even minimum, levels of performance. State level monitoring and evaluation would allow DPI to identify school systems that are meeting established performance goals and provide best practices advice to school systems that are underperforming. Implementation of statewide performance standards would also assure parents of AIG students that, no matter which school system their child attends, the State will hold the schools accountable for the quality and results of the AIG programs.

Results of Parent Survey

The level of satisfaction expressed by a program’s participants and stakeholders can serve as an indicator of the quality and effectiveness of the program.

We designed a survey to solicit feedback from parents regarding their child’s respective AIG program. As the students in the AIG Program are minors, we identified the parents as the stakeholder most likely to provide quality feedback regarding the quality and effectiveness of AIG programs statewide. The survey also provided parents with an opportunity to indicate their overall satisfaction with their AIG experience, as well as provide commentary regarding AIG. We only achieved a 24 percent response rate; therefore our results may not be representative of the entire population of AIG parents.

Ample opportunity exists to improve parent satisfaction with the level of services they receive from AIG programs. Overall, 61 percent of responding parents were satisfied with their AIG Program. However, 18 percent of responding parents were dissatisfied with the program, and 21 percent were neither satisfied nor dissatisfied. Additionally, of those parents who responded:

• 51% saw no difference in the educational services received since enrollment,

• 41% saw no improvement in their child’s academic performance, and

• 20% believe their child did not receive the planned AIG services.

Some parents who had moved from one school district to another expressed differing levels of satisfaction with the respective AIG programs. In addition, levels of satisfaction sometimes changed when the student moved from one school level to another (for example, elementary to middle school).

The parent involvement and public information components of the local plans also need improvement. State law requires the local plans to include a public information component. Of parents responding to our survey, 75 percent were not aware of the State law regarding the AIG Program, which outlines the required components of the local plans. And only 55 percent of parents said they were informed about their child’s AIG progress by the school.

Authority to Monitor Local AIG Programs Is Not Clear

A lack of clear authority to monitor the local AIG programs precludes establishing effective State-level monitoring procedures. Article 9B does not include wording that clearly gives the State Board of Education or DPI authority to monitor the local AIG programs.

AIG Program requirements are outlined in Chapter 115C - Article 9B of the North Carolina General Statutes. General Statute 115C-150.6 requires the State Board of Education to “Develop and disseminate guidelines for developing local plans.” North Carolina General Statute 115C-150.7 outlines mandatory components of local AIG plans and requires the LEA to develop and submit a plan “to the State Board of Education for its review and comments.” However, the law only requires the LEA to “consider the State Board’s comments before it implements the plan.” Read literally, the law does not require the LEAs to accept the recommendations of the State Board. Based on this interpretation, DPI infers that it has no authority or responsibility to monitor the implementation of the local plans.

The guidelines published by the State Board, however, state that, “The Department of Public Instruction according to the legislation will monitor the local program plans for academically or intellectually gifted students. The monitoring will determine whether the local plan contains the components required by law and whether the system is providing the services outlined in the plan.” DPI currently monitors for existence of the components within the local plans, but does not monitor whether the system is providing the services outlined in the plan.

Additionally, DPI does not have clear authority to ensure AIG funds are spent appropriately at the local level. State law allows the transfer of AIG funds for other purposes so long as the school demonstrates it is providing appropriate services to AIG students. However, the law does not indicate how and to whom the school should demonstrate that appropriate services have been provided. DPI has not assumed responsibility for making this determination, so what constitutes “providing appropriate services” is left to the discretion of the individual LEA. Parents alleged that schools use the unclear wording of the law as a loophole to divert funding from AIG to other priorities, and DPI conceded this was possible under current policies.

It is noteworthy that State law specifically prohibits transfers from funds allocated for children with disabilities. AIG programs were funded under this law until Article 9B -Academically or Intellectually Gifted Students - was enacted in 1996. Language was also added to Chapter 115C to allow for budget flexibility regarding AIG funding. General Statute 115C-105.25 allows LEAs to use AIG funds for other purposes without approval from DPI or the State Board of Education. Administrators at DPI pointed out that the interpretation of the current laws has resulted in an environment considerably different than when the AIG Program was covered under the laws for children with disabilities. The funds are not guaranteed to be used for AIG programs as they were when covered under the previous law.

Recommendations: The State Board of Education should establish or clarify authority regarding State-level monitoring of the AIG Program. DPI should develop and implement policies and procedures for State-level monitoring of the AIG Program. DPI should also require evidence that planned differentiated educational services were delivered and that AIG funds were used as outlined in local plans. DPI should also establish requirements for demonstrating that appropriate services have been provided before AIG funds are allowed to be used for other purposes.

DPI should establish AIG Program performance standards. The standards should provide the State with an evaluation tool that allows comparison of AIG services across schools and districts statewide. DPI should collect performance data from all LEAs and evaluate the quality and effectiveness of the AIG Program statewide. A comprehensive analysis of data from all 115 LEAs could identify best practices among the individual school systems and result in overall program improvement. Statewide evaluation should be performed regularly, and the results should be readily available for public inspection. Parents should be made aware of the standards and provided a method to comment on program results.

NC Department of Public Instruction
Exceptional Children Division
Program Improvement and Professional Development
Academically/Intellectually Gifted
Performance Audit Response
Purpose of Audit

The North Carolina Academically/Intellectually Gifted (AIG) program under the auspices of the Exceptional Children Division has been under a performance audit through the NC State Auditor’s Office from June, 2007-January, 2008. During that time, NC Department of Public Instruction (DPI) personnel and school district personnel have responded to state auditors’ requests and provided relevant information and documentation. The purpose of the audit was to identify weaknesses in the monitoring and evaluation of the AIG program in order for DPI to take appropriate corrective action. The audit was initiated by the Office of the State Auditor in response to parent concerns that AIG program funds were being used for other purposes while AIG students are left underserved.

Historical Overview

North Carolina has had legislation governing gifted education since 1961. In 1974 legislation identified gifted and handicapped children as children with special needs. In 1977, HB 824 was passed to bring into compliance a system of educational opportunities for all children requiring special education. In 1983, SB 127 changed the program title to “Academically Gifted” to emphasize North Carolina’s commitment to academic programs and legislated that a student’s gifted education program may be described with an Individual Education Plan (IEP) or a Group Education Plan (GEP). The 1993-94 Appropriations Bill, Section 134 (c ) required that the N.C. State Board of Education, “reexamine the State’s laws, rules, and policies concerning the education of academically gifted children.” The legislation required a study of four areas: 1) the need for state criteria for the identification of AG students, 2) the need to establish headcount as part of an annual census, 3) methods to identify and establish performance criteria to evaluate the effectiveness of programs, and 4) methods to ensure that the population of AG children is representative of the population of all students enrolled in North Carolina public schools. As a result of task force recommendations, legislation was passed in 1996, resulting in Article 9B (Article 9B § 115C-150.5 Academically or Intellectually Gifted Students). Article 9B provides a state definition for AIG, and requires the development of three-year AIG local plans with specific components to be approved by local school boards and then to be sent to DPI for review and comment.

Current Legislation and State Authority

Pursuant under Article 9B, as outlined in Chapter 115C-150.6 of the North Carolina General Statutes, are the responsibilities of the N.C. State Board of Education for AIG. The responsibilities include developing and disseminating guidelines for developing local plans for gifted education and providing technical assistance to LEAs in the development, implementation, and evaluation of their local plans. In addition, while Article 9B mandates that “each local board of education develop a local three-year plan designed to identify and establish a procedure for providing appropriate education services to each AIG student,” the statute only requires the Local Education Agency (LEA) to “consider the state board’s comments before it implements the changes.” The law does not require LEAs to accept, implement, or provide evidence of the recommendations provided on behalf of the State Board. Implications of this are that DPI does not have the authority to approve or monitor the effectiveness of local plans for gifted education. In addition, DPI does not have clear authority for the oversight of how LEAs use AIG funds (PRC 034).

Given the parameters of DPI’s current governance allowances within the context of Article 9B the N.C. Department of Public Instruction is providing the following responses to the N.C. State Auditor’s report.

Responses to Findings

The following responses are provided from DPI relative to each specific finding by the NC State Auditor’s Office and respond to the overall finding that “Local AIG program service delivery and spending are not monitored at the state level.”

Finding: Delivery of planned AIG services is not monitored at the state level.

Response: The department (DPI) recognizes the need to provide monitoring, oversight, and guidance to ensure that services for the gifted are implemented in accordance with each local AIG three-year plan. The department has begun to take steps to ensure the quality of services for AIG through the implementation of a U.S. Department of Education Javits grant, Project Bright Idea. This grant through the demonstration sites has had significant results with regard to increasing identification of traditionally under-represented populations in gifted as well as increasing services to gifted students by changing teacher capacity. The department will consider scaling up components of Project Bright Idea state-wide in order to deliver and monitor AIG services.

Finding: Local AIG Program Expenditures are not monitored at the state level.

Response: Program personnel within the Exceptional Children Division will initiate discussion with personnel in Financial and Business Services Division for the purposes of establishing internal systems for AIG program expenditures. In addition, DPI will work with the State Board of Education and/or General Assembly as needed to develop more effective allotment policies, ABC transfer provisions, and legislated oversight of AIG program expenditures.

Finding: DPI does not monitor local AIG program evaluation.

Response: The department recognizes the need to ensure that the quality and effectiveness of local AIG programs are properly evaluated. The state will review the possibility of developing state performance standards as a way to assess AIG program effectiveness and allow for statewide consistency while still supporting local flexibility.

Finding: Parent satisfaction regarding their child’s respective AIG program was mixed.

Response: Even though the response rate to the auditor’s survey was 24% and cannot be generalized, DPI recognizes the need to increase parent involvement, public information, and customer satisfaction with regard to respective AIG programs. Of the respondents 61% were satisfied with their AIG program; 18% were dissatisfied; and another 21% were neither satisfied nor dissatisfied. Literature reviews on the research of parent involvement in the education of students cite beneficial effects of parent involvement on student achievement, attendance, motivation, and behavior. To that end, the state department will work collaboratively with regional AIG liaisons, local program coordinators, state advocacy associations, institutions of higher education, and other stakeholder groups to develop strategies to proactively involve, communicate, and collaborate with parents in order to raise the level of satisfaction with regard to their child’s respective AIG program.

Finding: Authority to monitor local AIG programs is not clear.

Response: Article 9B does not require the LEAs to accept, amend, or implement the recommendations and comments provided by the State Board. There is a lack of statutory or policy authority to monitor local AIG programs. The department will work with the State Board of Education and/or General Assembly in consideration of changing legislation and/or board policies to allow for increased authority with regard to monitoring local AIG programs.

Conclusion

The importance of coherent and comprehensive gifted programs cannot be overstated. The structure that holds gifted programs together is nested in the policies, statutes, and guidelines that states have enacted. Local gifted programs, and subsequently the growth of gifted learners, are heavily influenced by the strength of the initiatives emanating from the state level. Moreover, in the absence of federal legislation, state directives are the cornerstone of gifted programming. Therefore, in an effort to strengthen gifted education in North Carolina, DPI is committed to addressing the necessary changes to ensure the AIG students’ needs are being met.