This Article Abstract Free Full Text (Free PDF) Free Alert me when this article is cited Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Similar articles in ISI Web of Science Alert me to new issues of the journal Add to Saved Citations Download to citation manager Request Permissions Request Reprints Add to My Marked Citations Citing Articles Citing Articles via HighWire Citing Articles via ISI Web of Science (3) Citing Articles via Google Scholar Citing Articles via Scopus Google Scholar Articles by Cawthon, S. W. Search for Related Content Social Bookmarking                   What's this?

Hidden Benefits and Unintended Consequences of No Child Left Behind Policies for Students Who Are Deaf or Hard of Hearing

The University of Texas at Austin

	Abstract

TOP Abstract Closing the Achievement Gap Theoretical Framework No Child Left Behind:... No Child Left Behind:... Why Study Students Who... Research Questions Method Data Analysis Results Hidden Benefits and Unintended... Conclusion APPENDIX Accommodations and... References

 
No Child Left Behind (NCLB) creates a high-stakes environment by holding schools accountable for how all students perform on state assessments, including students with disabilities and students who are English Language Learners. The focus of this article is on the impact of NCLB on students who are deaf or hard of hearing (SDHH). The SDHH have diverse linguistic characteristics and are served in a range of educational settings. The purpose of this article is to explore the hidden benefits and consequences of NCLB policy on SDHH in two areas: assessment and accountability. Drawing on findings from the author’s program of research, the article illustrates areas where policy may differentially affect students depending on their state of residence and educational setting. The discussion ends with a summary of benefits and hidden consequences of NCLB for SDHH.

Key Words: accountability • No Child Left Behind • assessment • students with disabilities

The purpose of this article is to discuss the impact of No Child Left Behind (NCLB) legislation on the educational structures that serve students who are deaf or hard of hearing.¹ As laid out in the introduction to the legislation, the purpose of NCLB is to ensure that "all children will have a fair, equal, and significant opportunity to receive a high-quality education and reach, at a minimum, proficiency on challenging state academic achievement standards and state assessments" (No Child Left Behind, 2001). This article reviews two key components of NCLB: assessment and accountability. Assessment policies include guidelines for how many students participate in standardized and alternate assessments as well as the use of testing accommodations. The accountability framework outlines how states use scores from assessments to make determinations about progress made toward NCLB goals.

	Closing the Achievement Gap

TOP Abstract Closing the Achievement Gap Theoretical Framework No Child Left Behind:... No Child Left Behind:... Why Study Students Who... Research Questions Method Data Analysis Results Hidden Benefits and Unintended... Conclusion APPENDIX Accommodations and... References

 
For the purpose of NCLB accountability legislation, students who are deaf or hard of hearing fall under the group of students with disabilities.² There are substantial gaps between the standardized assessment results for students with disabilities and those without. Education Week recently published "Quality Counts 2004: Count Me In—Special Education in an Era of Standards" (2004), a state-by-state evaluation of participation and proficiency rates for students with disabilities in large-scale assessments.³ At the 4th-grade level, more than half of the states had achievement gaps of 25% to 50% between students with disabilities and students without disabilities (Cawthon, 2004). The gaps in proficiency increased for only students in 8^th and 10^th grades. More recent analyses of changes in student proficiency since the start of NCLB suggest that there may have been some closing of the achievement gaps between student subgroups (Center on Education Policy, 2007b). Yet the data on achievement gap trends for students with disabilities cannot support strong conclusions because of the many changes through the early years of NCLB implementation in how students are tested and how scores are interpreted.

The achievement gap for students who are deaf or hard of hearing appears to be similar to that of students with disabilities as a whole (National Center on Low-Incidence Disabilities, 2006). In early analyses of academic performance on state assessments under NCLB, Cawthon (2004, 2005) presented available information on student achievement at schools for the deaf. For example, as of 2005, a total of 15 states reported standardized assessment results for students attending state-administered schools for the deaf. There was a range of student proficiency rates in math and reading on school report cards. However, overall performance was low; no more than half of students assessed at any school demonstrated grade-level proficiency in math or reading. In some schools, no students met state benchmarks.

The advantage of looking at assessment and accountability components is transparency: NCLB requires states to design policies and data-reporting measures that make it possible to track the progress toward proficiency goals. With NCLB goals of 100% proficiency for all student subgroups within 8 years, there is certainly a long way to go in closing this achievement gap. To what extent will NCLB have an impact on educational outcomes for students with disabilities? It is not clear whether students with disabilities as a subgroup, particularly students who are deaf or hard of hearing, will benefit from all of the strategies implemented under NCLB. Now that reauthorization is in the near future, it is important to identify both the strengths of the assessment and the accountability elements of the program. For students who are deaf or hard of hearing, revisions to the law can strengthen the positive impact already in place and remedy those hidden, but real, negative consequences of NCLB.

	Theoretical Framework

TOP Abstract Closing the Achievement Gap Theoretical Framework No Child Left Behind:... No Child Left Behind:... Why Study Students Who... Research Questions Method Data Analysis Results Hidden Benefits and Unintended... Conclusion APPENDIX Accommodations and... References

 
The NCLB is based on three main conceptual components: standards, assessments, and stakes (Quenemoen, Lehr, Thurlow, & Massanari, 2001; examples of this framework applied are discussed in Ladson-Billings & Tate, 2006; Orfield & Kornhaber, 2001; Peterson & West, 2003). Standards refer to the content knowledge and skills that form the basis for evaluating student proficiency. Standards-based reform began in the 1990s and was the precursor to the larger accountability framework we see today (Loeding & Crittenden, 1993; Porter & Smithson, 2001). Clear standards are essential to an effective accountability framework because it is important to be able to define what schools are asked to teach their students (for an in-depth discussion of state literacy standards, see Thompson, Johnstone, Thurlow, & Clapper, 2004; for a discussion of mathematics content standards, see Palacios, 2005). Assessments are used to measure a school’s effectiveness in teaching students standards-based material (Linn, 2000). Assessments of student achievement can vary in their ability to accurately represent student proficiency in core content areas (Sanders & Horn, 1995). Valid and representative test results are an important part of credible assessments (Phillips, 1994). Stakes are the end result of the accountability process (Moe, 2003). After assessment results are aggregated, policy then designates decision rules about the implications of student assessment scores. If a school is seen to meet expectations of ensuring adequate student proficiency on state standards, as measured by assessments, then the consequences for the school may be a reward or recognition. If expectations are not met, the stakes may be more restrictive or include intervention at the administrative or curricular level in hopes of improving future results.

Taken together, standards, assessments, and stakes make up the Grand Theory of Action of NCLB (Forte-Fast & Hebbler, 2004). Through clear expectations of educational goals, measurement of student progress, and consequences for not meeting these goals, NCLB hopes to motivate schools to improve instruction in core content areas. One of the main assumptions behind NCLB is that states, districts, and schools have the capacity (or potential capacity) to increase student achievement. To maintain accountability for all students, NCLB legislation requires states to disaggregate student performance data by (at minimum) student subgroups, including students with disabilities. The purpose of highlighting student performance for each student subgroup is to encourage schools and districts to improve student outcomes, not just as an average across all students, but specifically for students who have historically been underserved.

	No Child Left Behind: Assessments

TOP Abstract Closing the Achievement Gap Theoretical Framework No Child Left Behind:... No Child Left Behind:... Why Study Students Who... Research Questions Method Data Analysis Results Hidden Benefits and Unintended... Conclusion APPENDIX Accommodations and... References

 
One of the key components of the NCLB legislation is standardized assessment. The purpose of standardized assessments is to have a uniform, efficient, and valid method of measuring schools’ progress in developing student knowledge. Assessments focus on the core content areas of reading, mathematics, and science (new in 2007 to 2008). With an emphasis on annual (now Grades 3 to 8 and one grade in high school) assessment, NCLB represents a more rigorous and extensive implementation of required standardized assessment programs that had been in place in previous years. By tying test participation and results to an accountability mechanism, including funding and school administration elements, NCLB creates a high-stakes assessment environment.

Accommodations and Standardized Assessment
Participating in standardized assessment can be a challenging task for students with disabilities because of barriers inherent in a paper-and-pencil test format. Many students with disabilities now use testing accommodations to participate in statewide measures of student achievement (Lazarus, Thurlow, Lail, Eisenbraun, & Kato, 2006). Accommodations are changes to test administration that are designed to remove factors that penalize students because of their disability (McDonnell, McLaughlin, & Morison, 1997). For example, students with a visual impairment may receive a Braille version of the assessment, thus removing student inability to see the test as a barrier. The goal for accommodations is to decrease the extent to which the test measures the ability to take the test (access skills) and to increase the extent to which the test measures content knowledge and/or target skill (Elliott & Braden, 2000; Phillips, 1994; Shriner & DeStefano, 2003). Because of the high stakes of assessment under NCLB, trustworthy accommodations are critical for schools to accurately document student proficiency in core content areas (Quenemoen et al., 2001).

Although there is no cap on the number of students who can participate with accommodations, there are limits on the kinds of accommodations students can use. Each state sets its own regulations about accommodations use on assessments (Clapper, Morse, Lazarus, Thompson, & Thurlow, 2005; Lazarus et al., 2006; Minnema, Thurlow, Anderson, & Stone, 2005). Three commonly allowed accommodations (without restrictions on the use of student test scores) include extended time, small-groups or individual testing, and the use of interpreters for test directions (Thurlow & Bolt, 2001). Other accommodations that may be allowed, with restrictions on student test scores or only in certain circumstances, include the use of interpreters for test items or having test items read aloud to the student (descriptions of these accommodations examples are given in the appendix). Accommodations that change the presentation of the test item may pose a threat to assessment validity because it can change the cognitive demands of the test item (Koretz & Barton, 2003). These accommodations are thus viewed with greater caution than accommodations that change the setting of assessment (Braden & Elliott, 2003; McKevitt & Elliott, 2003).

Alternate Assessment
There are some students that cannot meaningfully participate in standardized assessments even with appropriate accommodations. Alternate assessments are meant to allow students with the most significant disabilities to participate in statewide assessments (Quenemoen, Rigney, & Thurlow, 2002). Alternate assessments are different in design than standardized assessments and can take on a number of formats (Thompson, Johnstone, Thurlow, & Altman, 2005; see appendix). For example, some states have teachers fill out a checklist of skills that the student has demonstrated during the course of classroom activity. Other states, such as Kentucky or Massachusetts, use a portfolio approach (Turner, Baldwin, Kleinert, & Kearns, 2000; Weiner, 2006). In a portfolio assessment process, teachers compile evidence of student knowledge and skills based on classroom projects and other artifacts. These portfolios may then be scored according to a rubric that yields a score such as "skill is emerging" or "student shows proficiency" (Quenemoen et al., 2002). Depending on the state policy for alternate assessment results, the scores can then be integrated into the overall accountability framework with markers of "proficient" or "not proficient" or as a report of gain in proficiency over time (Hill, 2001; Minnema, Thurlow, Bielinski, & Scott, 2000; Quenemoen et al., 2002; Quenemoen & Thurlow, 2002).

The NCLB places significant restrictions on the proportion of student scores from alternate assessments that can be used toward a district’s calculation of proficiency (Yell, Katsiyannas, & Shiner, 2006). Under the original implementation of NCLB, a maximum of 1% of the students in each district could have their alternate assessment scores counted toward proficiency (these caps apply only to the district and not to individual schools; Yell et al., 2006). In the past year, there have been some proposed modifications that, in certain cases, may allow an additional 2% of a district’s eligible test takers to participate in modified assessments and still be counted toward proficiency rates (National Archives and Records Administration, 2005, 2007). This 2% flexibility option is geared toward students who have persistent academic difficulties (Council for Exceptional Children, 2006). After the district has met the allocation of scores that can be counted from alternate assessments, the remaining scores are counted as not proficient, regardless of the actual assessment results.

	No Child Left Behind: Accountability

TOP Abstract Closing the Achievement Gap Theoretical Framework No Child Left Behind:... No Child Left Behind:... Why Study Students Who... Research Questions Method Data Analysis Results Hidden Benefits and Unintended... Conclusion APPENDIX Accommodations and... References

 
Assessment results are reported on a uniform reporting mechanism for all schools, districts, and states: reports of adequate yearly progress (AYP). Student proficiency data are typically reported on a "school report card" with a rating in whether the school or district has met AYP benchmarks. The AYP frameworks build on each state’s definition of standards for learning, assessments, and benchmarks for proficiency. Although the format is uniform, individual state components vary greatly. (For a discussion of differences in state definitions of proficiency and the benchmarks for annual progress, see Linn, 2003, and Porter, Linn, & Trimble, 2005.)

Group Size for Reporting
An emerging issue in discussions about AYP report cards is the use of a minimum group size for subgroup analysis. States determine the minimum number of students that need to be within each group before their scores are reported and used in AYP calculations (Erpenbach, Forte-Fast, & Potts, 2003). One purpose of establishing a minimum group size is to limit the potential loss of anonymity of student scores, guaranteed under the Family Educational Rights and Privacy Act. Another benefit is that raising the minimum group size allows states to focus on schools where intervention is truly needed because of the large numbers of students struggling to reach proficiency (Olson, 2005). However, there are cases where the minimum group number is set high enough (e.g., n = 50) that it limits disaggregations of students (including those with disabilities) to only very large schools. Early analyses of AYP reports showed that as states have minimum group sizes that exceed the number of students with disabilities enrolled, schools meet AYP guidelines without having to demonstrate effectiveness in bringing students with disabilities to academic proficiency benchmarks. For example, only 8% of schools in California had a students with disabilities subgroup on their school report card (Olson, 2005). Whereas the percentage of schools that include the proficiency scores of students with disabilities will vary by state, these findings point to a disturbing trend of excluding test results from the accountability process.

Confidence Intervals
Along with changes in the minimum subgroup size, states have also begun to include confidence intervals into their AYP calculations (Porter et al., 2005). Confidence intervals are statistical measures used to help account for measurement error when comparing a small group of students with the larger sample (the state as a whole) or a state benchmark for proficiency. For example, a state might use a 95% confidence interval, meaning that the state is 95% confident that the student scores have met the target with a stated range of error. Approximately 40 states have begun to include confidence intervals in AYP calculations to allow for error inherent in calculating statistics (Forte-Fast, Blank, Potts, & Williams, 2002; Porter et al., 2005; Simpson, Gong, & Marion, 2006). There are some very strong reasons for using confidence intervals when making high-stakes decisions about student performance. It would be unfair to ignore any statistical error that might underestimate the proficiency rates of a school’s students when compared to the statewide benchmarks. Confidence intervals allow states to set policies that draw on the strength of statistical models when drawing conclusions about the accuracy of the proficiency rate of the sampled student body.

Students With Disabilities and AYP
There has been some concern voiced that students with disabilities will become a scapegoat for schools that do not meet AYP under new disaggregations of student test scores (Allbritten, Mainzer, & Ziegler, 2004). If there is a significant achievement gap between students with disabilities and those without, it is reasonable to expect that it will be more challenging for students with disabilities to meet ever-increasing benchmarks for proficiency, particularly in the later years of NCLB. To some extent, the first 5 years of NCLB have confirmed these concerns (Olson, 2005; Thurlow & Wiley, 2004). For example, in an analysis of AYP for five states, AYP pass rates for schools with a students with disabilities subgroup are lower than for schools without (Simpson et al., 2006). Schools or districts with more students with disabilities may face greater challenges in meeting AYP than those with fewer students with disabilities.

	Why Study Students Who Are Deaf or Hard of Hearing?

TOP Abstract Closing the Achievement Gap Theoretical Framework No Child Left Behind:... No Child Left Behind:... Why Study Students Who... Research Questions Method Data Analysis Results Hidden Benefits and Unintended... Conclusion APPENDIX Accommodations and... References

 
Unintended consequences and hidden benefits of NCLB have the potential to have long-term effects on children who are deaf or hard of hearing. The deaf and hard of hearing student population is unique in several ways. First, it is a low-incidence group, carrying with it the challenges of having only a few students educated in a single location. There are only an estimated 70,000 students who are deaf or hard of hearing who receive special services in public schools in the United States, less than 1% of the total K through 12 student population (Mitchell, 2004; U.S. Department of Education, 2004). Second, when their primary language is American sign language, many students who are deaf or hard of hearing are part of a cultural and linguistic minority (Lane, 1999). Although classified as a disability for the purpose of NCLB and much of the relevant special education legislation (Americans with Disabilities Act, 1990; Rehabilitation Act, 1973), deafness also has linguistic and cultural aspects that affect how some students experience educational reform. Finally, deaf or hard of hearing students are a diverse group served by several different educational structures (Gallaudet Research Institute, 2005; Ramsey, 1997). For example, hearing losses range from mild to profound, with differences in how each individual perceives sound, particularly speech (Musselman, 2000). Individuals also vary in the onset of hearing loss and whether they have attended early intervention programs for language or speech development (Marschark, 1997).

Education Settings
A central question for parents when raising a deaf or hard of hearing child is where he or she should attend school (Fiedler, 2001; Marschark, 1997). The Individuals with Disabilities Education Act (Public Law 108–446, reauthorized in 2004) outlines equal access that children with disabilities have to a free and appropriate public education. Depending on the resources of their district or state, parents of students who are deaf or hard of hearing have several options as to where their children can attend school. Schools for the deaf are typically stand-alone programs that sometimes include a residential component. District and school programs for the deaf or hard of hearing tend to bring together students on a regional basis, providing either stand-alone education or partial integration into an adjacent regular education setting. Finally, mainstreamed education models focus on providing services, if necessary, to the student within the general education setting. Support services can include interpreters, itinerant teachers of the deaf, or some pull-out services during the school day to reinforce instruction by the regular education teacher.

Although there is often overlap between these three models, such as a school for the deaf providing regional or itinerant services, it is from these three perspectives that I will discuss benefits and consequences of NCLB. Depending on individual state NCLB policies, accountability reform affects the district programs, regular education schools, and schools for the deaf in different ways (Cawthon, 2004). It is upon these unique and varied educational structures that NCLB legislation will reach students who are deaf or hard of hearing. Studying students who are deaf or hard of hearing thus provides policy developers with a useful example of potential long-term effects of NCLB on students with low-incidence disabilities, those who may not use English as a first language, or those who are served by educational settings outside of the mainstream.

	Research Questions

TOP Abstract Closing the Achievement Gap Theoretical Framework No Child Left Behind:... No Child Left Behind:... Why Study Students Who... Research Questions Method Data Analysis Results Hidden Benefits and Unintended... Conclusion APPENDIX Accommodations and... References

 
The purpose of this article is to present 2004 to 2005 data on how NCLB assessment and accountability components affect students who are deaf or hard of hearing. Two questions guide this discussion:

1. How do state NCLB assessment policies affect students who are deaf or hard of hearing?
   
2. How do state NCLB accountability policies differentially affect diverse educational structures that serve students who are deaf or hard of hearing?
   

	Method

TOP Abstract Closing the Achievement Gap Theoretical Framework No Child Left Behind:... No Child Left Behind:... Why Study Students Who... Research Questions Method Data Analysis Results Hidden Benefits and Unintended... Conclusion APPENDIX Accommodations and... References

 
National Survey of Assessments
Findings for the first research question draw on the results of the Second Annual National Survey of Assessments and Accommodations for Students Who Are Deaf or Hard of Hearing (Cawthon & the Online Research Lab, in press). The purpose of the survey was to collect data on the ways students who are deaf or hard of hearing participated in state assessments used for NCLB accountability during the 2004 to 2005 academic year. Data from this survey allow for a descriptive analysis of accommodations use across states with different assessment policies. Participants provided information about the location of the school or program, characteristics of the educational setting, student characteristics, number of students participating in state standardized and alternate assessments, and testing accommodations used, where relevant. The National Survey investigated assessment practices for deaf or hard of hearing students in K through 12 public school settings. The survey format included multiple-choice, Likert-type scale, and open-ended response items. The survey instrument was administered in two ways: (a) online at the project Web site, www.dhh-assess-survey.org (developed using www.surveymonkey.com) and (b) paper versions provided to individuals with stamped, self-addressed envelopes for returned responses. Incentives for participation included entry in a drawing for $25 gift certificates upon completion of the survey.

Multiple sample strategies were required because recruitment can be particularly challenging when the target population is dispersed throughout varied educational settings (Cawthon, 2006). Survey participants were individual teachers recruited initially through the Gallaudet Research Institute’s Annual Survey of Schools and Programs contact list as well as the participant list from the First Annual National Survey. Additional recruitment contacts were made through Web site affiliations, state lists of programs and services, and 687 personal e-mails and postcard invitations by the principal investigator. Nearly all (344, or 88%) of total participants responded online and 48 (12%) responded via hard copy. Participant confidentiality was maintained by coding individual responses with an identification number; no specific names of individuals or schools are reported in the study results.

Participants
A total of 392 participants from mainstreamed educational settings (n = 143), schools for the deaf (n =101), and district or regional programs for the deaf or hard of hearing (n = 148) provided information used in these analyses (either about accommodations, alternate assessments, or both). Respondents were primarily teachers of the deaf (48%) or regular education, special education, or itinerant teachers (20% total). Respondents also included administrators (7%) and those who served in multiple roles (6%). The remaining participants were interpreters, speech-language pathologists, school psychologists, and other professionals who work with students who are deaf or hard of hearing. In this discussion, the terms participants and teachers refer to the respondents as a whole.

Teachers who responded to this survey served just more than 9,300 students nationwide; this is approximately 12% of the estimated 70,000 population in the United States who receive special services under the Individuals with Disabilities Education Act (IDEA; U.S. Department of Education, 2004). The sample was not random; all known schools and programs serving deaf or hard of hearing students were contacted to participate in the study. The representativeness of this sample varied by census region. A total of 10% of the students in the National Survey were from the Northeast Census region, compared with 18% of the Child Count estimates of the deaf or hard of hearing student population (Mitchell, 2004). Students of study participants from the Midwest region made up 22% of the sample, compared with 23% in the Child Count estimates. Because of recruitment procedures, the sample was heavily weighted toward students in the South Census region (43%). This was a higher concentration of students in the South than in the IDEA Child Count (31%). Finally, 24% of the students served by study participants came from the West Census region, compared with 25% in the national estimate. In sum, the study sample was representative of the national population estimates in the West and Midwest Census regions, overestimated population from the South region and underestimated counts from the Northeast Census region.

Accountability Plans and AYP Report Cards
Two data sources were used to address the second research question regarding the effects of accountability policies on different educational structures that serve students who are deaf or hard of hearing. The first source was the Consolidated State Application Accountability Workbooks, first submitted by each state to the U.S. Department of Education in January 2003. These plans were then reviewed by the Department of Education and have been revised over the course of the past several years. Approved state accountability workbooks are found on the Department of Education Web site (http://www.ed.gov/admins/lead/account/stateplans03/index.html). Accountability plans include an articulation of how the state will aggregate and report data for students from schools for the deaf. To provide a consistent picture of assessment and accountability processes, the plans as they applied to testing procedures and AYP decision making in the 2004 to 2005 school year are used in this analysis.

The second source of data was the individual 2004 to 2005 school report cards for schools for the deaf provided on state department of education Web sites. A list of state Web sites can be found at the Education Resources Organization Directory: http://bcol02.ed.gov/Programs/EROD/org_list.cfm?category=ID-SEA. Each state determines its own method of data reporting and display. Most state Web sites had a designated location for NCLB information and accountability data. State special education directors were contacted to verify the location of school report cards and clarify reporting policies. The list of these contacts was found on the National Association of State Directors of Special Education Web site (NASDE, 2006).⁴

	Data Analysis

TOP Abstract Closing the Achievement Gap Theoretical Framework No Child Left Behind:... No Child Left Behind:... Why Study Students Who... Research Questions Method Data Analysis Results Hidden Benefits and Unintended... Conclusion APPENDIX Accommodations and... References

 
National Survey
National survey data were analyzed using the SPSS software package. Analysis began with descriptive reports of the prevalence of student participation in standardized assessments and their use of seven accommodations: extended time, small groups or individual administration, test directions interpreted, test items read aloud, test items interpreted, student signs response to a scribe, and simplified English. Full results of accommodations used can be found at Cawthon & the Online Research Lab (2006). Policy variables were added to the data set to allow for analysis by the type of accommodations policy in each participant’s state (Cawthon, 2007). These designations included whether the accommodation was "Allowed," "Allowed with implications for scoring and/or aggregation," "Allowed in certain circumstances," or "Prohibited" for use on state standardized assessments (Clapper et al., 2005). These policy variables are the main independent variable in the secondary data analysis presented in this article.

Accountability Plans
Accountability plans for each state and the District of Columbia were downloaded and reviewed for their policies on how schools for the deaf were to be included in NCLB accountability frameworks. Specific attention was paid to Principle 1.1: A single statewide accountability system applied to all public schools and local education agencies. I looked at each state’s response to the following requirement:

State has a definition of "public school" and "local educational agency" for AYP accountability purposes. The State Accountability System produces AYP decisions for all public schools, including public schools with variant grade configurations (e.g., K-12), public schools that serve special populations (e.g., alternative public schools, juvenile institutions, state public schools for the blind) and public charter schools. It also holds accountable public schools with no grades assessed (e.g., K-2). (U.S. Department of Education, 2002, p. 7)

State plans were defined as fitting into one of four categories: (a) The school for the deaf receives AYP report card, (b) assessment scores for students at school for the deaf are sent to the sending or referring district, (c) student scores are aggregated at the state level, or (d) schools for the deaf are exempt from score reporting.

School Report Cards
After identifying the states where student scores at schools for the deaf were to be aggregated in a school report card, I then went to the NCLB report card link at each state department of education Web site. Data collection included student proficiency rates on assessments in core content areas, minimum n for reporting scores on the report card, state benchmarks for meeting AYP, and current AYP status. When data were not reported, I contacted the state department of education staff member responsible for NCLB data reporting to confirm whether the information was reported in another format. These data were summarized for all states that had both a publicly funded school for the deaf and an accountability plan designating the school for the deaf as a recipient of an NCLB report card.

	Results

TOP Abstract Closing the Achievement Gap Theoretical Framework No Child Left Behind:... No Child Left Behind:... Why Study Students Who... Research Questions Method Data Analysis Results Hidden Benefits and Unintended... Conclusion APPENDIX Accommodations and... References

 
Assessment and Students Who Are Deaf or Hard of Hearing
Standardized assessment and accommodations
Because state NCLB report cards do not provide information about use of accommodations or participation in alternate assessments, it is difficult to track the assessments practices that lead to student scores. The first goal of this analysis is to better understand how state NCLB policies might affect assessment practices for students who are deaf or hard of hearing. Participation data from 2004 to 2005 statewide assessments are shown in Figure 1 (Cawthon & the Online Research Lab, in press). In this national sample of 7,646 students, the majority participated in standardized assessments with accommodations (70%). (This figure is smaller than the overall sample for the study because not all participants provided student assessment participation information.) An additional 13% of students represented in this study participated in alternate assessments. The 10% that did not participate includes students who were not enrolled for the entire school year or who were in grades that were not assessed in 2004 to 2005.

View larger version (19K): [in this window] [in a new window]   Figure 1 Distribution of deaf or hard of hearing students participating in 2004 to 2005 state assessments. Figure 1represents the proportion of students (N = 7,646) who participated in standardized assessments, with or without accommodations, alternate assessments, or who did not participate in the statewide assessments. Data are from the Second National Survey of Assessments and Accommodations for Students Who Are Deaf or Hard of Hearing (Cawthon & the Online Research Lab, in press).

Interaction of assessment policy and practice
The kind of accommodation(s) used during testing, together with the state policy for score reporting, affects how student scores are integrated into the accountability system. States such as New York, Texas, and California have higher populations than states such as New Hampshire and Kansas; state policies in more populous sates thus have a larger impact on students who are deaf or hard of hearing than those with smaller populations. Table 1 gives a summary of the estimated distribution of students by state policy for key accommodations used with students who are deaf or hard of hearing for 2004 to 2005 state standardized assessments.

Depending on where the student lived, policies for score use may result in different ways that students using the accommodations were integrated into the accountability framework. This is demonstrated in Table 1 by the distribution of students in states with policies for student-signed response and test items interpreted. For student-signed responses, just more than a third (37%) of students lived in states where state policies allow for students to sign their responses to test items without implications for scoring. However, about half (53%) lived in states where there was no clear policy for this accommodation in the Clapper et al. (2005) summary. Lack of policy information was not as significant for test items interpreted (only 12% of students lived in states without a policy). Instead, about a third of students lived in states where there were no implications for scoring (30%); another third in states where test items interpreted was allowed in certain circumstances but with no implications for scoring (38%), and the remaining students (19%) in states where there were restrictions both in the tests for which this accommodation is allowed and in how scores were aggregated within the school’s accountability framework. This is a surprisingly small proportion of students in states that allowed test items interpreted, particularly when seen in comparison to the distribution of students in states that allow test items to be read aloud.

To illustrate the importance of the state policy context, consider data for students from the state of California. In the Second Annual Survey, 40 participants from California (serving 890 students) provided information about their assessment practices with deaf or hard of hearing students. California policy for the 2004 to 2005 assessment year allowed students to use extended time, small groups, test directions interpreted, and student signs response without restriction on the state assessments. Changes to the test items, such as having test items read aloud or interpreted, could only be used in certain circumstances and had implications for including the score in aggregates of student performance. There was no policy information available for the use of simplified English. California had a relatively lenient accommodations policy, whereas other states have more restrictions on assessments taken with accommodations. States such as California served a relatively large proportion of the study sample; policies in the larger states weigh more heavily in these overall summaries of how accommodations policy and practice interact.

Table 1 provided an overview of the potential impact of state policies based on the distribution of students by state of residence. Table 2 shows available data on assessment practices with students who are deaf or hard of hearing, aggregated by state policy categories (Clapper et al., 2005). Figures in Table 2 represent the proportion of teachers who had at least one student participate in standardized assessments using each accommodation. Percentages across each row total to 100%, or all teachers who provided information for that accommodation. Cells with "—" indicate accommodations where no teacher reported information about student participation in states with those policies.

On the other end of the table, we see which accommodations were not used by teachers with students who are deaf or hard of hearing. Student-signed responses (83%) and simplified English (91%) stand out as accommodations that are not common practice. As shown in Table 1, these accommodations were also those that had a large percentage of students who lived in states without policy information on their use in accountability frameworks.

For most of the accommodations in Table 2, students either largely received accommodations that allow for their score to be included or the accommodations are not prevalent enough to cause concern. However, read-aloud test items and interpreted test items showed a more complex picture of the relationship policy between practice. For read aloud, only 3% of teachers using the accommodation lived in states where it is allowed without implications. Approximately half (49%) lived in states where it is allowed in certain circumstances, usually for tests that do not target reading skills. An additional 16% lived in states that limit both the tests during which read aloud can be used and how the scores are aggregated. Student scores with interpreted test items also have restrictions, but in contrast with read aloud, 40% of teachers that report using this accommodation lived in states with implications for scoring.

How does the interaction between policy and practice play out in the California example? California state policy explicitly allows students to use extended time, small groups, interpreter for test directions, and student sign response without restriction. Students who use these accommodations should have their scores fully included in the NCLB accountability process. We turn our focus to the remaining three accommodations, each with restrictions on how scores can be used: test items read aloud, test items interpreted, and the use of a simplified English format. For test items read aloud, 36% of participants had a student use this accommodation for math, reading, or both subjects. A similar proportion, 38%, used test items interpreted for at least one tested subject. Only 8% of teachers in California used a simplified English format of an assessment with their students who are deaf or hard of hearing. For students who participated using either test items read loud or interpreted (students from more than a third of California survey participants), it is likely that their test scores were not aggregated or automatically received a "zero" or "below basic" proficiency rating.

In sum, assessment practices with students who are deaf or hard of hearing are complex, but concerns about the validity of test scores and inclusion in AYP accountability appears to be limited to read-aloud and interpreted test item accommodations. The application of these findings is not limited to students who are deaf or hard of hearing. Students with disabilities, as a whole, use accommodations to participate in statewide NCLB assessments, including those that result in restrictions in how scores are included in AYP decisions (Thompson et al., 2005). Students who are English Language Learners (ELLs), especially those with disabilities, may face similar difficulties in having their scores meaningfully represented in the accountability frameworks (Albus & Thurlow, 2005; Anderson, Minnema, Thurlow, & Hall-Lande, 2005). English language proficiency and literacy are essential to a student’s ability to access information on a standardized test. Accommodations, such as having the test items read aloud or interpreted, may help a student gain access to the test content. Yet assessment policies for ELL students with disabilities are still in the early stages of development (Anderson et al., 2005). Increasing clarity in this area should have a direct benefit for teachers and administrators who serve students who are developing proficiency in English literacy, including those who are deaf or hard of hearing.

Alternate assessments
From the perspective of NCLB, students who are deaf or hard of hearing typically do not qualify for alternate assessments unless they have multiple disabilities that make it difficult to provide appropriate accommodations for a standardized assessment. National Survey results provide some information on the use of alternate assessments with students who are deaf or hard of hearing. Teachers at schools for the deaf (n = 74) had a mean of 16% (SD = 26%) participate in alternate assessments; district or regional programs (n = 98) had a mean of 23% (SD = 29%) participate in alternate assessments; and mainstreamed programs (n = 72) had a mean of 14% (SD = 25%) of students participate in alternate assessments. There were no significant differences in these figures by educational setting, F (2, 244) = 2.768, p = .065. These results indicate that there is a potential for districts to have more than the allocated percentage of students (1% to 3%) who can participate in alternate assessments and have their scores counted toward a district’s AYP benchmarks. With greater enrollments of students who are deaf or hard of hearing, schools for the deaf (that serve as their own district) and district programs have the greatest potential to reach the cap on the proportion of students who can participate in an alternate assessment. Yet the true significance of these findings can only be known with more information about the total number of students at each school site, the use of alternate assessments with other students, and how these figures aggregate at the district level.

Summary
Assessments are one of the key components of accountability frameworks established by No Child Left Behind legislation. Students who are deaf or hard of hearing, on the whole, do participate in some form of state assessment. The majority participate in standardized assessments, many with accommodations. However, a small but still sizable (under NCLB criteria) proportion of students participate in alternate assessments. How the accommodations use and alternate assessment scores of students affect the AYP ratings of the different educational settings that serve students who are deaf or hard of hearing is an important area for future study.

Accountability
Schools for the deaf and AYP
The amount of information available about schools for the deaf and AYP is potentially representative of students in these settings but suffers from attrition in the route from policy to practice (Figure 2). There are 45 states (including the District of Columbia) with publicly funded schools for the deaf. In just under two thirds of the states (n = 29), schools for the deaf received their own school report card and AYP status for student scores in the 2004 to 2005 school year. A third of these states (n =13) reported student data at the sending or referring school or district. Two states exempted schools for the deaf from state testing and one state, Tennessee, aggregated data for students at schools for the deaf to the state level for accountability purposes.

View larger version (20K): [in this window] [in a new window]   Figure 2 Schools for the deaf and adequate yearly progress designations. Figure 2 represents the drop in the number of states with No Child Left Behind (NCLB) assessment data available for students attending schools for the deaf. Data are from a review of state policies and report cards for the 2004 to 2005 academic year.

Challenges in data reporting
Technical issues that affect AYP calculations for small schools are salient in the discussion of report cards for schools for the deaf. Schools that serve only a small population face special challenges in NCLB calculations on AYP report cards (Erpenbach et al., 2003). Enrollments at state schools for the deaf range from 35 students (North Dakota School for the Deaf) to more than 700 students (Utah School for the Deaf and Blind; "List of Schools and Programs for the Deaf and Hard of Hearing," 2001). These are K through 12 enrollments that often combine both the deaf and the blind student populations. Some AYP report cards disaggregated student scores by each assessment grade, resulting in far smaller numbers of students for each AYP criterion.

If a school for the deaf with an AYP report card was in a state with a very low minimum number of students (e.g., Maryland, with a minimum n = 5), it was more likely to report individual grade proficiency rates than if the school is in a state with a high minimum group size (as with California, n = 100). A school will automatically meet AYP if there are not enough students to meet minimum n for data reporting. Although many schools for the deaf enroll more than the minimum for data reporting, it is often the case that there were not enough students in the assessed grade (e.g., 8^th or 10^th grade) to meet these requirements. Of the 15 states with report cards for their schools for the deaf for 2004 to 2005, five (Arizona, Florida, Hawaii, Rhode Island, and Washington) did not report data because there was an insufficient sample of student scores for the assessed grade. These findings confirm research that suggests that a rise in minimum n results in exclusion of students with disabilities from the accountability framework (Simpson et al., 2006). If states raise their minimum group sizes, we will likely know less about how NCLB is working to improve educational outcomes for students who attend schools for the deaf.

A second challenge for AYP report cards at schools for the deaf is in how small student populations interact with the use of confidence intervals. When combined with small group sizes, confidence intervals can result in misleading interpretations of student scores. The smaller the group size, the wider the range of scores that statistically meet the criteria for passing the AYP benchmark. Large confidence intervals allow for a wide range of average scores to be counted as proficient, ranges that can hide what are actually low rates of student proficiency. There are even cases where a 0% proficiency, because it is within the confidence interval, is counted as meeting AYP. In 2004 to 2005, of the nine schools for the deaf that met AYP targets for mathematics, four did so through the use of confidence intervals. For reading, two of the five schools that met AYP targets did so through confidence intervals. The relationship between proficiency rates and confidence intervals will be an important factor to watch in future years.

Summary
NCLB offers transparency in how schools are working toward closing the achievement gap. The NCLB reporting mechanisms, however, are limited in how much information they can provide about the effectiveness of accountability reform on students who are deaf or hard of hearing. Schools for the deaf are the only educational setting that receives report cards specifically for these students, and only a handful are able to give full reports because of state policies in how AYP is calculated. Those that have been able to meet AYP benchmarks will face significant challenges to continue this trend. The rate of change projected for students to reach proficiency outpaces any growth in student achievement seen even in recent years on the National Assessment of Educational Progress (The Education Trust, 2004). States started slowly with the rising achievement benchmarks but will need to increase them substantially to comply with the NCLB deadline for 100% proficiency. As with many schools and districts, schools for the deaf that have made AYP targets in the past may be less likely to do so in the future.

	Hidden Benefits and Unintended Consequences

TOP Abstract Closing the Achievement Gap Theoretical Framework No Child Left Behind:... No Child Left Behind:... Why Study Students Who... Research Questions Method Data Analysis Results Hidden Benefits and Unintended... Conclusion APPENDIX Accommodations and... References

 
Hidden Benefits of NCLB
The overall goal of NCLB is to improve student achievement for all students. There are several aspects of NCLB legislation that may benefit students who are deaf or hard of hearing and, by extension, other students with disabilities or English Language Learners: An emphasis on student outcomes instead of placement are more transparent reporting mechanisms and better research on testing accommodations.

Student outcomes
Schools have historically struggled to bring students who are deaf or hard of hearing to academic proficiency, particularly in literacy and content areas that rely on strong reading and writing skills. The focus on student outcomes, instead of student access, reflects a shift from the previous way of measuring equity for students with disabilities. Under IDEA, students were ensured equal access to education, but there was little guarantee of equal outcomes. The assumption was that if students were exposed to general education curriculum and taught in as free and appropriate a setting as possible, students with disabilities would have the greatest chance of achieving academic goals. Based on student achievement results on state assessments, this has not occurred for students with disabilities or for students who are deaf or hard of hearing. The NCLB therefore provides the opportunity to hold the education system accountable for student learning, not just questions of student placement. Under NCLB, programs from all teaching and communication philosophies will be held to the same standards for student outcomes. Although this may not lead to competition between education models, per se, it does put the public attention on what programs can do to improve academic proficiency in students who have previously been underserved.

Reporting frameworks
The second benefit of NCLB is the potential for consistent and transparent data-reporting frameworks. Schools and districts within each state are held to the same standards for all students, including those with disabilities. States must provide an easily accessible report of how their schools and districts meet the benchmarks for accountability purposes. All in all, a good start. Although there are many limitations to the reporting framework, NCLB has put promising data collection elements into place. For students with low-incidence disabilities, disaggregated proficiency results by disability, even if only at the state level, would bring much needed clarity to how effective NCLB measures are at raising student achievement.

Accommodations research
Under NCLB, high-stakes testing is used to make decisions with far-reaching consequences for schools, districts, and states. When standardized assessments are used as the primary source of information about student progress, it is essential that the scores are valid and accurately represent student knowledge. Accommodations rooted in language and communication are most important for students who are deaf or hard of hearing. Tables 1 and 2 highlighted two accommodations where validity research is needed to help inform policy: test items read aloud and test items interpreted. Without conclusive evidence on the effect of accommodations on test scores, policies are likely to remain variable from state to state.

The high stakes of state standardized assessment should result in more rigorous and much needed research on the validity of accommodations that change the presentation of the test items. Several researchers have begun comprehensive validity programs of study for students with disabilities as a whole (Elliott, Kratochwill, & McKevitt, 2001; Elliott, McKevitt, & Kettler, 2002; Koretz & Hamilton, 2001; Tindal & Fuchs, 2000). These studies include a look at the effects of read aloud on both reading and mathematics assessments. However, results thus far are limited in that they do not study the validity of using these accommodations with students who are deaf or hard of hearing. In addition to emerging research on the use of test items read aloud or interpreted, other language accommodations include having students sign responses that are then recoded by a scribe. Each of these accommodations allows students who are deaf or hard of hearing to participate in assessments in their primary language, which is sometimes also the language of instruction. Research on language, communication, and American sign language accommodations within test administration will play an important role in improving the validity of accommodations used with students who are deaf or hard of hearing.

Unintended Consequences
In addition to the potential benefits of the NCLB accountability framework, there are some significant challenges to ensuring that students who are deaf or hard of hearing have full access to the resources provided under this legislation. An unintended consequence of NCLB is in how unevenly its components are applied to students with diverse backgrounds or who are served by a range of educational settings. Areas that need further attention include integration of standardized scores into AYP frameworks and consideration of how alternate assessments can be used with students who are deaf or hard of hearing.

Accommodations and integrating test scores
States are generally cautious about how scores of assessments with accommodations are used in determining adequate yearly progress. Although many students participate in standardized assessments, they do so using accommodations that result in restrictions in how the scores are reported or aggregated for AYP purposes. A visual representation of the impact of policy on student scores with these accommodations is shown in Figure 3. This figure gives a breakdown of participants whose students used accommodations, aggregated by the state policy for its score (Figure 3 does not include students who did not use the accommodation). For example, of participants whose students use test items interpreted, only a third do so in a state where that accommodation is allowed without restrictions or with implications for scoring. The situation is worse for test items read aloud, where less than 10% of participants with students using this accommodation were in states where it was allowed without restriction. If students used the accommodation on restricted tests, such as a reading assessment, the scores would not be included with other scores. When test scores are automatically assigned a "not proficient" value, one loses information about what knowledge or skill the student demonstrated on the assessment. There is a concern therefore that students who are deaf or hard of hearing may be disproportionately underrepresented in state accountability frameworks. Furthermore, schools and districts with high numbers of students who are deaf or hard of hearing, such as those with regional programs, may face greater challenges in meeting AYP because of the number of students participating using restricted accommodations.

View larger version (12K): [in this window] [in a new window]   Figure 3 Percentage of participants with students using accommodations, by type of state policy that allows or restricts assessment scores. Data represent the percentage of participants who report having students use each accommodation in the 2004 to 2005 standardized assessments, aggregated by the state policy for that accommodation in their state of residence.

Differential effect on educational structures
School report card policies are unevenly applied to educational settings that serve this target population. Because students who are deaf or hard of hearing are taught in both stand-alone and mainstreamed classrooms, educational setting is a significant factor in how high-stakes decisions affect this target population. The group of students least affected by NCLB are those in regular education settings. Mainstreamed schools are accountable for students with disabilities, yet students who are deaf or hard of hearing are likely to make up only a very small proportion of the overall students with disabilities population (the closest level of aggregation on school report cards). Unless the school already hosts a large population of students with disabilities, AYP reports for mainstreamed schools are not likely to change based on the progress made by one or two students who are deaf or hard of hearing attending that school.

The second setting where students are taught is district programs that serve students from schools in the region. Districts are accountable for student proficiency, but district AYP report cards offer little information about the effectiveness of programs for special populations, specifically. District programs are not like stand-alone schools for the deaf because they do not receive a single report card that holds the program accountable for student achievement. Instead, student scores are sent back to the referring or sending school, dispersing the student scores across the district. This said, district programs may affect district AYP designations to the extent that students in their programs make up a large proportion of the students with disabilities within the district. For example, depending on the number of other students scoring below proficiency, scores from students who are deaf or hard of hearing may alter whether the district meets state proficiency benchmarks. There are also similar concerns about the inclusion of student test scores that were obtained using accommodations such as test items read aloud or interpreted. Concerns about the use of alternate assessments, particularly when the prevalence exceeds the districts’ 3% cap on whether proficiency scores can be counted, are most salient for districts with large programs for students with disabilities. As with accountability for students in mainstreamed settings, accountability for district programs for deaf or hard of hearing students is related to the overall status of AYP reports for students with disabilities as a whole.

Most of the assessment and accommodation information available for students who are deaf or hard of hearing is for those who attend schools for the deaf. In contrast with mainstreamed settings and regional programs, schools for the deaf are the only educational setting that receives its own report cards and AYP designation. Whether a school for the deaf itself is accountable for student test scores depends on the state policy for special education schools. For states that aggregate student scores at the hosting school, there is the potential for a transparent and consistent mechanism for knowing how students fare on state assessments. At schools for the deaf, issues of minimum subgroup size and confidence intervals will continue to play a role in what we know about student proficiency for students with disabilities. Yet overall, this represents transparent accountability for only a small proportion of students who are deaf or hard of hearing.

	Conclusion

TOP Abstract Closing the Achievement Gap Theoretical Framework No Child Left Behind:... No Child Left Behind:... Why Study Students Who... Research Questions Method Data Analysis Results Hidden Benefits and Unintended... Conclusion APPENDIX Accommodations and... References

 
One challenge of a large-scale accountability system is to hold all schools to the same standards while leveraging the mechanisms that will improve outcomes for a diverse student population. By disaggregating assessment scores by subgroups of students and integrating them into how schools are evaluated, NCLB broadens accountability from the school level (overall student performance) to the subgroup level. As a result, the bar for satisfactory progress has been raised and shows that there is significant work to be done.

There are places where the assumptions of the NCLB Grand Theory of Action are met and some where there is little connection between policy and the lived reality in schools. There are two key lessons we can take from this discussion. First, NCLB accountability depends on a clear assessment process—one with alignment between state policies and teacher practice. One finding from this analysis is teachers use a range of accommodations that reflect the linguistic diversity of their students. State policies are, at times, at odds with local-level decisions. The implication is not simply that teachers should change how they include students in assessments or that policies should be revised to meet what happens in classrooms. Instead, we need a thorough investigation of why these accommodations are used with students who are deaf or hard of hearing, what is needed to ensure access to test content, and what impact these changes have on the validity of the test scores. The result of this effort should be an informed policy framework that takes both research and practice into account.

The second requirement for a successful accountability framework is a consistent application of high-stakes policies. The Center on Education Policy noted in its own analysis that shifting policies and measurement practices result in great difficulty in tracking the progress of students with disabilities who are proficient from year to year (Center on Education Policy, 2007b). Because both assessment and AYP guidelines vary from state to state, it is difficult to know how to interpret the relative level of student achievement in schools and districts. For students from low-incidence populations, cross-state analyses are necessary because of the small number of students who may participate in each assessed grade. Yet inconsistent accountability guidelines make comparisons of student outcomes impossible. For example, students who participate in assessments with controversial accommodations may find their scores included in school AYP decisions in one state but not in another. This leads to different decisions about the quality of the school with potentially significant consequences. Reducing variability in assessment and accountability policy would improve accuracy in measuring how NCLB increases student learning.

Findings in this article also illustrate the broader challenges of using standardized assessments as the basis for high-stakes AYP decision making. State standardized assessments were not designed to measure achievement in students without grade-level English literacy proficiency and academic preparation. Testing accommodations are useful in facilitating participation in assessments, but they are limited in their capacity to "even the playing field" for students with disabilities without invalidating test scores. Current pilot programs for using measures of student gains in knowledge and skills, or growth models, are potential ways to demonstrate how students are making progress toward proficiency benchmarks instead of absolute achievement levels (U.S. Department of Education, 2007). States will need to be deliberate in how they adjust growth models to apply to students without English proficiency, with disabilities, or who may have other persistent academic difficulties. For example, growth models can consider the additional time these students may need to reach proficiency, extending the time schools have to bring all students to proficiency. Similarly, models can tie benchmarks to the student’s level of English proficiency and opportunity to learn content area materials, creating a more accurate picture of what schools are accountable for in a given school year.

Even under recent modifications to the guidelines, students who are deaf or hard of hearing do not typically qualify for alternate assessments, limiting the way in which their scores can be included into accountability decisions. The same issue applies to students who may not have a severe cognitive disability but do not have the written English proficiency to meaningfully participate in a standardized assessment. There is a hole here that needs to be filled in a way that allows teachers to make assessment decisions that best match the linguistic and academic skills of their students. The Commission on No Child Left Behind, although not endorsing formative assessments or portfolios of student achievement, does concur with the position that current assessment tools for students with disabilities or who are English Language Learners are insufficient (Commission on No Child Left Behind, 2007). Perhaps the role of alternate assessments in NCLB thus needs to be revisited, either by broadening whose scores are eligible or by providing additional options for students who cannot meaningfully participate in standardized assessments because of language barriers. Without these adjustments, students who are deaf or hard of hearing and other English Language Learners may be unintentionally excluded from the NCLB.

Analysis in this article demonstrated that there are some significant barriers to demonstrating accountability for all student groups in AYP decisions, especially students who are a low-incidence group or who attend special education schools. Only a small number of student assessment scores, those at schools for the deaf that receive a report card, are transparent in the current AYP structure. In 2004 to 2005, more than half of the states where there was a policy in place to report scores at schools for the deaf had done so. Students in mainstreamed and district program settings are folded into the larger "students with disabilities" context, holding the schools accountable in a general sense but making it difficult for the community to monitor academic proficiency of students who are deaf or hard of hearing, specifically. The NCLB accountability should not be limited to students who are a part of a large subgroup or who attend their local neighborhood school. State-level or district-level aggregation of student achievement for low-incidence groups is one recommended way to increase the transparency of NCLB effectiveness for students from low-incidence populations.

The success of NCLB will not be measured in how it raises scores for the majority of students but in how well students who have previously struggled to meet academic goals achieve new levels of success. The research community now has the opportunity to study the effectiveness of accountability strategies on improving student achievement for student subgroups, not just students as a whole. Yet NCLB lacks a mechanism for understanding how the education structure changes and improves student outcomes. Measures of instructional content and its impact on students, particularly those from diverse subgroups, are largely missing from the NCLB Grand Theory of Action. In essence, we see what comes out of the "black box" of learning but understand little about what happens within it. For example, do all students have the opportunity to learn standards-based content that is on state assessments? How do we know whether student gains are because of changes in instruction or other factors? The implicit assumption that NCLB will change the quality of instruction needs to be explicit for the community to be able to target areas for improvement. Future reauthorizations of NCLB (or additional reforms) thus will need to go beyond measuring student achievement and focus on actually increasing our capacity to meet the educational needs of students with disabilities, English Language Learners, and students with diverse backgrounds.

	APPENDIX Accommodations and Alternate Assessment Formats

TOP Abstract Closing the Achievement Gap Theoretical Framework No Child Left Behind:... No Child Left Behind:... Why Study Students Who... Research Questions Method Data Analysis Results Hidden Benefits and Unintended... Conclusion APPENDIX Accommodations and... References

 

	Footnotes

 
STEPHANIE W. CAWTHON has a background in educational psychology and human development and focuses on the impact of educational policy on students with disabilities. Department of Educational Psychology, SZB 254, 1 University Station D5800, The University of Texas at Austin, Austin, TX, 78712-0383; stephanie.cawthon@mail .utexas.edu.

The author’s research background is in educational psychology and human development, looking specifically at issues of language and academic achievement in students who are deaf or hard of hearing. The author has conducted a series of studies investigating the affect of educational reform at both the local level (in classrooms) and the macro level (across the United States). The author hopes this research shows how large-scale reforms affect students with low-incidence disabilities and those with diverse linguistic backgrounds. From this vantage point, the author hopes to help develop research-based practices and educational policies that are responsive to the needs of these students.

This work was supported in part by the 2006 Faculty Excellence Fund at Walden University. The author wishes to thank the many participants of the National Survey of Assessments and Accommodations for Students Who Are Deaf or Hard of Hearing, her students in the Online Research Lab at Walden University, and all of those who have supported this work. Portions of the findings here were presented at the 2005 Conference of Educational Administrators of Schools and Programs for the Deaf and the 2006 meeting of the American Educational Research Association. The author also gives many thanks to the editors and reviewers for their assistance in shaping this article for publication.

¹ The definition of deaf and hard of hearing varies by hearing threshold and cultural identity. Researchers in this field take care to describe the characteristics of the deaf or hard of hearing population participating in each study. Deaf or hard of hearing may include cultural deaf, sign language users, those with cochlear implants, those who wear hearing aids, and those who use a range of communication styles in a variety of settings. Because there is no clear distinction between who identifies with deaf and deaf or hard of hearing groups, this article will refer to the group as a whole.

² Designation of deafness as a disability does not go uncontested by those who advocate for a linguistic and cultural minority group understanding of deaf persons (Lane, 1999).

³ These data include participation in, accommodations with, and proficiency scores for statewide, large-scale assessments. Many formulations are similar, though not always exactly, to the assessment procedures used to determine adequate yearly progress.

⁴ This approach was used in analyses of student proficiency in other school years (Cawthon, 2004, in press).

Received for publication September 11, 2006. Revision received May 12, 2007. Accepted for publication July 3, 2007.

	References

TOP Abstract Closing the Achievement Gap Theoretical Framework No Child Left Behind:... No Child Left Behind:... Why Study Students Who... Research Questions Method Data Analysis Results Hidden Benefits and Unintended... Conclusion APPENDIX Accommodations and... References

 

• Albus, D, & Thurlow, M. (2005). Beyond subgroup reporting: English language learners with disabilities in 2002–2003 online state assessment reports (ELLs with Disabilities Report No. 10), University of Minnesota, National Center on Educational Outcomes: Minneapolis. Retrieved January 23, 2007, from http://education.umn.edu/NCEO/OnlinePubs/ELLsDisReport10.html.
• Allbritten, D, Mainzer, R, & Ziegler, D. (2004). Will students with disabilities be scapegoats for school failures? Teaching Exceptional Children, 36(3), 74-75
• Americans with Disabilities Act of 1990, 42 U.S.C. 12101 et seq. (1990).
• Anderson, ME, Minnema, JE, Thurlow, ML, & Hall-Lande, J. (2005). Confronting the unique challenges of including English language learners with disabilities in statewide assessments (ERIC Document Reproduction Service No. ED 486 230). Minneapolis: University of Minnesota, National Center on Educational Outcomes
• Braden, JP, & Elliott, SN. (2003). Accommodations on the Stanford-Binet Intelligence Scales. (5). Itasca, IL: Riverside
• Cawthon, S. (2004). Schools for the Deaf and the No Child Left Behind Act. American Annals of the Deaf, 149(4), 314-323[CrossRef][Web of Science][Medline] [Order article via Infotrieve]
• Cawthon, S. (2005, April 30–May 1). No Child Left Behind and deaf education. Paper presented at the Annual Conference of Educational Administrators of Schools and Programs for the Deaf: Atlanta, GA
• Cawthon, S. (2006). Pebbles in the mainstream: How do we find them? American Annals of the Deaf, 151(2), 105-113[CrossRef][Web of Science]
• Cawthon, S. (2007, April 8–13). Findings from the first two years of the National Survey of Assessments and Accommodations for Students Who Are Deaf or Hard of Hearing. Paper presented at the Annual Meeting of the American Educational Research Association: Chicago, IL
• Cawthon, S. Schools for the Deaf and NCLB: Integration into the accountability framework. In Johnson, RC, Karchmer, M, & Mitchell, R (Ed.). Assessing the academic achievement of deaf students in an age of accountability. Washington, DC: Gallaudet University Press. in press.
• Cawthon, S, & the, Online Research Lab. (2006). Findings from the National Survey on Accommodations and Alternate Assessments for Students Who Are Deaf or Hard of Hearing. Journal of Deaf Studies and Deaf Education, 11(3), 337-359[Abstract/Free Full Text]
• Cawthon, S. the Online Research Lab. Findings from the Second Annual National Survey of Assessments and Accommodations for Students Who Are Deaf or Hard of Hearing. Journal of Deaf Studies and Deaf Education. in press. in press. in press.
• Center on Education Policy. (2007a, May). NCLB’s accountability provisions for students with disabilities: Center on Education Policy roundtable discussion. Washington, DC: Author
• Center on Education Policy. (2007b, June). Answering the question that matters most: Has student achievement increased since No Child Left Behind? Washington, DC: Author
• Clapper, AT, Morse, AB, Lazarus, SS, Thompson, SJ, & Thurlow, ML. (2005). 2003 State policies on assessment participation and accommodations for students with disabilities (Synthesis Report No. 56), University of Minnesota, National Center on Educational Outcomes: Minneapolis. Retrieved March 22, 2006, from http://education.umn.edu/NCEO/OnlinePubs/Synthesis56.html.
• Commission on No Child Left Behind. (2007, June 14). Statement by commission on FEA report on assessment and accountability in NCLB. Retrieved July 2, 2007, from www.nclbcommission.org.
• Council for Exceptional Children. (2006). New flexibility in testing students with disabilities a positive step. Retrieved September 8, 2006, from http://www.cec.sped.org//AM/Template.cfm?Section=Home.
• The Education Trust. (2004). Education watch: Achievement, attainment and opportunity from elementary school through college. Washington, DC: Author
• Elliott, SN, & Braden, J. (2000). Educational assessment and accountability for all students: Facilitating the meaningful participation of students with disabilities in district and statewide assessment programs. Madison: Wisconsin State Department of Public Instruction
• Elliott, SN, Kratochwill, TR, & McKevitt, B. (2001). Experimental analysis of the effects of testing accommodations on the scores of students with and without disabilities. Journal of School Psychology, 39, 3-24[CrossRef][Web of Science]
• Elliott, SN, McKevitt, B, & Kettler, R. (2002). Testing accommodations research and decision-making: The case of "good" scores being highly valued but difficult to achieve for all students. Measurement and Evaluation in Counseling and Development, 35, 153-166[Web of Science]
• Erpenbach, W, Forte-Fast, E, & Potts, A. (2003, July). Statewide educational accountability under NCLB. Washington, DC: Council of Chief State School Officers
• Family Educational Rights and Privacy Act, ([date]). 20 U.S.C. Sec 1232g; 34 CFR Part 99no date for this act (?).
• Fiedler, B. (2001). Considering placement and educational approaches for students who are deaf and hard of hearing. Teaching Exceptional Children, 34(2), 54-60
• Forte-Fast, E, Blank, R, Potts, A, & Williams, A. (2002, December). A guide to effective accountability reporting. Washington, DC: Council of Chief State School Officers
• Forte-Fast, E, & Hebbler, S. (2004, February). A framework for examining validity in statewide accountability systems. Washington, DC: Council of Chief State School Officers
• Gallaudet Research Institute. (2005, January). Regional and national summary report of data from the 2003-2004 Annual Survey of Deaf and Hard of Hearing Children and Youth. Washington, DC: Gallaudet University, Gallaudet Research Institute
• Hill, R. (2001). The impact of including special education students in accountability systems, National Center for the Improvement of Educational Assessment, Retrieved January 1, 2007, from http://www.nciea.org/publications/CCSSOSpecialEd_Hill01.pdf.
• Horvath, LS, Kampfer-Bohach, S, & Kearns, JF. (2005). The use of accommodations among students with deaf-blindness in large-scale assessment systems. Journal of Disability Policy Studies, 16(3), 177-187[Abstract/Free Full Text]
• Koretz, D, & Barton, K. (2003). Assessing students with disabilities: Issues and evidence. Los Angeles: Center for the Study of Evaluation
• Koretz, D, & Hamilton, L. (2001). The performance of students with disabilities on New York’s Revised Regents Comprehensive Examination in English (Technical Report No. 540). Los Angeles, CA: Center for the Study of Evaluation
• (2006). In Ladson-Billings, G, & Tate, W (Ed.). Education research in the public interest: Social justice, action, and policy. New York: Teacher’s College Press
• Lane, H. (1999). The mask of benevolence. (2). San Diego, CA: Dawn Sign Press
• Lazarus, SS, Thurlow, ML, Lail, KE, Eisenbraun, KD, & Kato, K. (2006). 2005 state policies on assessment participation and accommodations for students with disabilities (Synthesis Report No. 64). Minneapolis: University of Minnesota, National Center on Educational Outcomes
• Linn, RL. (2000). Assessment and accountability. Educational Researcher, 29(2), 4-16[Abstract/Free Full Text]
• Linn, RL. (2003). Accountability: Responsibility and reasonable expectations. Educational Researcher, 32(7), 3-13[Abstract/Free Full Text]
• List of schools and programs for the deaf and hard of hearing. (2001). American Annals of the Deaf, 146, 133-161
• Loeding, B, & Crittenden, J. (1993). Inclusion of children and youth who are hearing impaired and deaf in outcomes assessment. In Ysseldyke, J, & Thurlow, M (Ed.). Views on inclusion and testing accommodations for students with disabilities (Synthesis Report No. 7). Minneapolis: National Center on Educational Outcomes
• Marschark, M. (1997). Raising and educating a deaf child. New York: Oxford University Press
• McDonnell, L, McLaughlin, M, & Morison, P. (1997). Educating one and all: Students with disabilities and standards-based reform. Washington, DC: National Academy Press
• McKevitt, BC, & Elliott, SN. (2003). Effects of perceived consequences of using read-aloud and teacher-recommended testing accommodations on a reading achievement test. School Psychology Review, 32, 583-600[Web of Science]
• Minnema, J, Thurlow, M, Anderson, M, & Stone, K. (2005). English language learners with disabilities and large-scale assessments: What the literature can tell us (ELLs with Disabilities Report No. 6). Minneapolis: University of Minnesota, National Center on Educational Outcomes
• Minnema, J, Thurlow, M, Bielinski, J, & Scott, J. (2000). Past and present understandings of out-of-level testing: A research synthesis (Out-of-Level Testing Project Report No. 1). Minneapolis: University of Minnesota, National Center on Educational Outcomes
• Mitchell, R. (2004). National profile of deaf and hard of hearing students in special education from weighted survey results. American Annals of the Deaf, 149(4), 336-349[CrossRef][Web of Science][Medline] [Order article via Infotrieve]
• Moe, T. (2003). Politics, control, and the future of school accountability. In Peterson, P, & West, M (Ed.). No Child Left Behind? The politics and practice of school accountability (pp.336-349). Washington, DC: Brookings Institution
• Musselman, C. (2000). How do children who can’t hear learn to read an alphabetic script? A review of the literature on reading and deafness. Journal of Deaf Studies and Deaf Education, 5(1), 9-31[Abstract/Free Full Text]
• Nagle, K, Yunker, C, & Malmgren, K. (2006). Students with disabilities and accountability reform. Journal of Disability Policy Studies, 17(1), 28-39[Abstract/Free Full Text]
• National Archives and Records Administration. (2005, December 15). Department of Education. Federal Register, 70(240), 74629
• National Archives and Records Administration. (2007, April 9). Department of Education. Federal Register, 72(67), 17748-17781
• National Center on Low-Incidence Disabilities. (2006). Statewide assessment results for students with low-incidence disabilities. Retrieved January 20, 2007, from http://nclid.unco.edu/outcomes/.
• No Child Left Behind Act of 2001, 20 U.S.C. 6301 (2002).
• Olson, L. (2005). AYP rules miss many in special education. Education Week, 25(4), 1-25, 24
• Orfield, G, & Kornhaber, M. (2001). Raising standards or raising barriers? Inequality and high-stakes testing in public education. New York: Century Foundation Press
• Palacios, L. (2005). Critical issue: Mathematics education in the era of NCLB—Principles and standards. Naperville, IL: North Central Regional Educational Laboratory
• (2003). In Peterson, P, & West, M (Ed.). No Child Left Behind? The politics and practice of school accountability. Washington, DC: Brookings Institution
• Phillips, SE. (1994). High-stakes testing accommodations: Validity versus disabled rights. Applied Measurement in Education, 7(2), 93-120[CrossRef]
• Porter, A, Linn, R, & Trimble, S. (2005, April). The effects of state decisions about NCLB adequate yearly progress targets. Paper presented at the Annual Meeting of the National Council on Measurement in Education: Montreal, Canada
• Porter, A, & Smithson, J. (2001). Are content standards being implemented in the classroom? A methodology and some tentative answers. In Fuhrman, SH (Ed.). From the capitol to the classroom: Standards-based reform in the states—One hundredth yearbook of the yearbook of the National Society of the Study of Education, Part II (pp.60-80). Chicago: University of Chicago Press
• (2004, January). Quality Counts 2004: Count me in—Special education in an era of standards. Bethesda, MD: Editorial Projects in Education
• Quenemoen, RF, Lehr, CA, Thurlow, ML, & Massanari, CB. (2001). Students with disabilities in standards-based assessment and accountability systems: Emerging issues, strategies, and recommendations (Synthesis Report No. 37). Minneapolis: University of Minnesota, National Center on Educational Outcomes
• Quenemoen, R, Rigney, S, & Thurlow, M. (2002). Use of alternate assessment results in reporting and accountability systems: Conditions for use based on research and practice (Synthesis Report No. 43). Minneapolis: University of Minnesota, National Center on Educational Outcomes
• Quenemoen, R, & Thurlow, M. (2002). Including alternate assessment results in accountability decisions (Policy Directions No. 13). Minneapolis: University of Minnesota, National Center on Educational Outcomes
• Ramsey, C. (1997). Deaf children in public schools. Washington, DC: Gallaudet University Press
• Rehabilitation Act of 1973, 29 U.S.C. 794 (1973).
• Roeber, E. (2002). Setting standards on alternate assessments (Synthesis Report No. 42). Minneapolis: University of Minnesota, National Center on Educational Outcomes. Retrieved March 22, 2006, from http://education.umn.edu/NCEO/OnlinePubs/Synthesis42.html.
• Sanders, W, & Horn, S. (1995). Educational assessment reassessed: The usefulness of standardized and alternative measures of student achievement as indicators for the assessment of educational outcomes. Educational Policy Analysis Archives, 3(6). Retrieved January 20, 2007, from http://epaa.asu.edu/epaa/v3n6.html.
• Shriner, J, & DeStefano, L. (2003). Participation and accommodation in state assessment: The role of individualized education programs. Exceptional Children, 69(2), 147-161[Web of Science]
• Simpson, MA, Gong, B, & Marion, S. (2006). Effect of minimum cell sizes and confidence interval sizes for special education subgroups on school-level AYP determinations (Synthesis Report No. 61). Minneapolis: University of Minnesota, National Center on Educational Outcomes. Retrieved January 23, 2007, from http://education.umn.edu/NCEO/OnlinePubs/Synthesis61.html.
• Thompson, SJ, Johnstone, CJ, Thurlow, ML, & Altman, JR. (2005). 2005 state special education outcomes: Steps forward in a decade of change. Minneapolis: University of Minnesota, National Center on Educational Outcomes. Retrieved September 8, 2006, from http://education.umn.edu/nceo/OnlinePubs/2005.StateReport.htm.
• Thompson, SJ, Johnstone, CJ, Thurlow, ML, & Clapper, AT. (2004). State literacy standards, practice, and testing: Exploring accessibility (Technical Report No. 38). Minneapolis: University of Minnesota, National Center on Educational Outcomes. Retrieved January 23, 2007, from http://education.umn.edu/NCEO/OnlinePubs/Technical38.htm.
• Thurlow, M, & Bolt, S. (2001). Empirical support for accommodations most often allowed in state policy (Synthesis Report No. 41), University of Minnesota, National Center on Educational Outcomes: Minneapolis. Retrieved January 3, 2006, from http://education.umn.edu/NCEO/OnlinePubs/Synthesis41.html.
• Thurlow, ML, & Wiley, HI. (2004). Almost there in public reporting of assessment results for students with disabilities (Technical Report No. 39), University of Minnesota, National Center on Educational Outcomes: Minneapolis. Retrieved January 23, 2007, from http://education.umn.edu/NCEO/OnlinePubs/Technical39.htm.
• Tindal, G, & Fuchs, L. (2000). A summary of research on test changes: An empirical basis for defining accommodations. Lexington, KY: Mid-South Regional Resource Center
• Turner, M, Baldwin, L, Kleinert, H, & Kearns, J. (2000). The relation of a statewide alternate assessment for students with severe disabilities to other measures of instructional effectiveness (ERIC Document Reproduction Service No. EJ611320). Journal of Special Education, 34(2), 69-76[Abstract/Free Full Text]
• US Department of Education. (2002). Consolidated state application accountability workbooks. Washington, DC: Author
• US Department of Education. (2004). Twenty-sixth annual report` to Congress on the implementation of the Individuals with Disabilities Education Act. Washington, DC: Author
• US Department of Education. (2007). Growth models: Ensuring grade level proficiency for all students by 2014. Washington, DC: Author. Retrieved July 18, 2007, from http://www.ed.gov/admins/lead/account/growthmodel/proficiency.html.
• Weiner, D. (2006). Alternate assessments measured against grade-level achievement standards: The Massachusetts "Competency Portfolio" (Synthesis Report No. 59), University of Minnesota, National Center on Educational Outcomes: Minneapolis. Retrieved January 20, 2007, from http://education.umn.edu/NCEO/OnlinePubs/Synthesis59.html.
• Yell, M, Katsiyannas, A, & Shiner, J. (2006). The No Child Left Behind Act, Adequate Yearly Progress, and students with disabilities. Council for Exceptional Children, 38(4), 32-39

American Educational Research Journal, Vol. 44, No. 3, 460-492 (2007)
DOI: 10.3102/0002831207306760


CiteULike    Connotea    Del.icio.us    Digg    Reddit    Technorati    What's this?

This article has been cited by other articles:

				J. Lollis and C. LaSasso The Appropriateness of the NC State-Mandated Reading Competency Test for Deaf Students as a Criterion for High School Graduation J. Deaf Stud. Deaf Educ., January 1, 2009; 14(1): 76 - 98. [Abstract] [Full Text] [PDF]

				S. J. McCarthey The Impact of No Child Left Behind on Teachers' Writing Instruction Written Communication, October 1, 2008; 25(4): 462 - 505. [Abstract] [PDF]

This Article Abstract Free Full Text (Free PDF) Free Alert me when this article is cited Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Similar articles in ISI Web of Science Alert me to new issues of the journal Add to Saved Citations Download to citation manager Request Permissions Request Reprints Add to My Marked Citations Citing Articles Citing Articles via HighWire Citing Articles via ISI Web of Science (3) Citing Articles via Google Scholar Citing Articles via Scopus Google Scholar Articles by Cawthon, S. W. Search for Related Content Social Bookmarking                   What's this?