The evaluation framework allows for continual quality improvement by providing .... To achieve the goal of uniformity in
Setting Standards and an Evaluation Framework for Human Immunodeficiency Virus/Acquired Immunodeficiency Syndrome Surveillance H. Irene Hall and Eve D. Mokotoff for the Advisory Group for Technical Guidance on HIV/AIDS Surveillance rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr
N
ational acquired immunodeficiency syndrome surveillance commenced with the beginning of the human immunodeficiency virus (HIV) epidemic in the United States in 1981, and by 2003 all states had implemented HIV surveillance. This information, used for prevention interventions, and the allocation of resources, must be accurate to determine trends in HIV transmission and the number of persons living with HIV. Standards for data accuracy were developed through a national consensus approach and integrated into a framework for local and national program evaluation. The evaluation framework allows for continual quality improvement by providing information for training and technical assistance efforts. These tools allow comprehensive assessments of whether reported HIV case data are adequate, reliable, and sufficiently accurate for determining the resources needed for HIV prevention and care. KEY WORDS: acquired immunodeficiency syndrome (AIDS),
evaluation, HIV, surveillance
Since the beginning of the human immunodeficiency virus (HIV) epidemic in the United States, all US states, the District of Columbia, and the US territories have reported acquired immunodeficiency syndrome (AIDS) cases to the Centers for Disease Control and Prevention (CDC).1 This information was used to track trends in the HIV epidemic during the early years and, since 1990 with the enactment of the Ryan White Comprehensive AIDS Resources Emergency (CARE) Act, to determine allocation of funding to cities and US states and territories to provide care and support services for individ-
J Public Health Management Practice, 2007, 13(5), 519–523 Copyright C 2007 Wolters Kluwer Health | Lippincott Williams & Wilkins
uals with HIV and AIDS who have low incomes, lack insurance, or are underinsured.2 However, with the advent of highly active antiretroviral therapy (HAART) in 1996, disease progression to AIDS has declined among persons treated with HAART; because of the variability in disease progression, AIDS diagnoses can no longer be used to estimate HIV infection rates. The current HIV epidemic is, therefore, better described by including all HIV-infected persons, whether or not AIDS has developed.3 In 1994, the CDC integrated the national reporting of HIV infection with the reporting of AIDS, and 25 states with confidential, name-based HIV infection reporting started submitting case reports to the CDC. Names and other identifying information are not reported to the CDC. Currently, all states have implemented HIV reporting. States that use code-based identifiers are not reporting HIV cases to the CDC because without names, there is a higher likelihood of case duplication, weakening the accuracy of HIV surveillance. As of June 2006, 45 states and 5 territories report information on all HIV cases to the CDC.
Advisory Group members: Gordon Bunch, MA, County of Los Angeles Department of Health Services, California; Dena Bensen, MPH, Virginia Department of Health, Richmond; James Kent, MS, Public Health—Seattle & King County, Washington, DC; Stanley See, BS, Department of State Health Services, Austin, Texas; and Martha Miller, MPH, and Pamela Gruduah, BBA, Centers for Disease Control and Prevention, Atlanta, Georgia. The findings and conclusions in this report are those of the authors and do not necessarily represent the views of the Centers for Disease Control and Prevention. Corresponding Author: H. Irene Hall, PhD, MS E-47, Centers for Disease Control and Prevention, 1600 Clifton Rd NE, Atlanta, GA 30333 (
[email protected]).
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq H. Irene Hall, PhD, is Epidemiologist, Division of HIV/AIDS Prevention, National Center for HIV, STD, and TB Prevention, Centers for Disease Control and Prevention, Atlanta, Georgia. Eve D. Mokotoff, MPH, is HIV/AIDS Epidemiology Manager, HIV/STD and Bloodborne Infections Surveillance Section, Michigan Department of Community Health, Detroit, and Chair, HIV/AIDS Surveillance Workgroup, Council of State and Territorial Epidemiologists, Atlanta, Georgia.
519
520 ❘ Journal of Public Health Management and Practice Availability of data on cases of HIV infection would allow a better needs-based allocation of CARE Act funds. In 2004, the Institute of Medicine reviewed information to assess whether reported HIV cases are adequate, reliable, and sufficiently accurate for determining CARE Act funding and concluded that existing national HIV reporting was not adequate for determining CARE Act funding. Instead, the institute recommended that activities should be focused on improving the consistency, quality, and comparability of HIV case reporting and on conducting periodic assessments of the completeness and timeliness of HIV infection reporting.2 A limited set of minimum performance standards for HIV reporting was published in 1999,1 but these standards needed to be operationalized and defined in more detail to be measurable. In addition, a comprehensive framework defining best practices and evaluation procedures was not available. We describe the development of evaluation standards and an evaluation framework for HIV/AIDS surveillance by the CDC in collaboration with the Council of State and Territorial Epidemiologists (CSTE). We use the term HIV/AIDS to refer to all HIV cases. The standards and evaluation framework are applied to HIV cases regardless of whether AIDS was diagnosed at the same time; in addition, AIDS reporting may be evaluated using the same approach.
● Development of Evaluation Standards In 2004, the CDC and the CSTE formed workgroups of CDC and state and local HIV/AIDS surveillance staff to revise the technical guidance for HIV/AIDS reporting that had been last updated in 1996. New surveillance methods, such as retaining all documents with the original data (document-based surveillance) by using a relational database structure and electronic reporting, were incorporated, as were standards to allow the routine evaluation of local and national surveillance systems. Each workgroup, composed of experts in a specific subject area, wrote the relevant chapter on that subject, including best practices and standards. In addition, a consultation group composed of CDC and CSTE representatives was formed to oversee the development of the technical guidance. Each chapter was submitted to review by the consultation group, to clearance by the CDC, and to review by all state and local HIV/AIDS surveillance staff. After completion of the review process, the Technical Guidance for HIV/AIDS Surveillance was posted on a password-protected Web site accessible to all HIV/AIDS surveillance programs (a copy of the guidance may be obtained from the corresponding author).
Although each workgroup developed the guidance for their assigned subject area independently, the workgroups were asked to consider the following concepts in drafting the standards: (1) whether a standard pertained to a process or an outcome, (2) the time for which the standard applied, (3) the time at which the standard was to be measured, and (4) whether measurement was to be applied to all or a sample of HIV/AIDS cases.
● Standards for HIV/AIDS Surveillance Data To achieve the goal of uniformity in data quality from the various surveillance systems, the technical guidance includes structural requirements (what a program must, should, or may have to operate an HIV/AIDS surveillance system), process standards (what a program must, should, or may do to achieve the objectives), and outcome standards (measurable objectives). Certain process and outcome standards are required (Table 1); these are activities and measurable objectives that, when met, indicate a fully functioning surveillance system with high-quality data. Other processes or measures described in the technical guidance are recommendations of best practices that will help meet the outcome standards. The outcome standards reflect the activities and measurable objectives that are most commonly listed for any fully functioning surveillance system, and data submitted to the national surveillance program are evaluated by the CDC. A formal certification process by an outside organization like the one used for cancer registries does currently not exist.4,5 For cancer registries, the North American Association of Central Cancer Registries Inc, a professional organization that develops data standards and publishes data from member registries, evaluates data submissions according to their standards, and publishes the names of certified registries (ie, registries that met all data standards). In addition to the completeness and timeliness of case reporting and the complete and error-free reporting of date of diagnosis and basic demographic characteristics, HIV/AIDS surveillance programs must also show high-quality data for risk factor information to determine populations at risk for HIV infection (Table 1). The standard for completeness of reporting of CD4+ T-cell counts may be considered a developmental standard. It reflects a vision for measuring the desired outcome of ensuring that all persons with a new diagnosis of HIV infection have a CD4+ T-cell count within 3 months after diagnosis (a clinical standard). The data standard requires the implementation of laboratory results reporting, but at this time not enough data is available to determine whether this standard is a feasible approach to measuring laboratory reporting and
Setting Standards and an Evaluation Framework for HIV/AIDS Surveillance
❘ 521
TABLE 1 ● Process and outcome standardsa for HIV/AIDS case surveillance in the United States
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq Process standards Death ascertainment Intrastate duplicate review Interstate duplicate review Outcome standards Completeness Timeliness
Edits Risk factor ascertainment
Proportion of death-certificate–only cases Intrastate duplicates Interstate duplicates Data reporting and dissemination CD4 reportingb
Routine record linkage of HIV/AIDS case reports (all cases without minimum death information) and death certificate records (most recent year of deaths available), once a year (minimum) To be conducted monthly To be conducted at least semiannually ≥85% of expected number of cases for a diagnosis year reported by 12 months after the diagnosis year 1. See time component of completeness standard 2. On the basis of the time from the date of diagnosis to the time of report to surveillance program (report date = date document was received at the health department or if not available, date document was entered in the surveillance system), the minimum performance standard is ≥ 66% of cases for a diagnosis year reported within 6 months after diagnosis, assessed at ≥ 85% completeness 12 months after the diagnosis year ≥97% of case records pass all standard data edits. The standard is assessed for the most recent diagnosis year at 12 months after that diagnosis year ≥85% of reported cases or a representative sample for a diagnosis year have an identified HIV risk factor within 12 months after the date of the initial HIV/AIDS case report, measured at 12 months after the diagnosis year
