Improving Testing Through English Language Learners: A Comprehensive Approach to Designing, Building, Implementing & Interpreting Better Academic Assessments Book

Preface     xi
Purpose and Overview     1
Changing Demographics in a Testing Culture: Why This Issue Matters     13
Demographics and School Policy     14
Demographics Snapshot     14
An Educational Policy Snapshot     17
ELLs: A Heterogeneous Population     19
Culture and Learning     20
Culture and Assessment     25
The Educational Assessment of English Language Learners     29
Tests of English Language Proficiency     30
Large-scale Academic Achievement Assessments     33
The Past as Preparation: Measurement, Public Policy, and Implications for Access     37
A Brief History of Large-scale Achievement Testing     37
Development of Measurement Theory Models     42
Reliability to Multidimensional Item Response Theory     42
A Changing View of Test Validity     45
Fifty Years of Federal Policy Regarding Equity and the Use of Achievement Testing     50
ESEA: Full Inclusion, Disaggregated Reporting, and Beyond     50
Other Federal Influences     53
Building Assessments to be More Accessible for ELLs     55
Conceptual Underpinnings     55
Developing Accessible Tests     59
Estimating Levels of Access     60
Getting Started: Issues of Participation, Alignment, and Validating Access with Test Specifications     65
Participation in Test Development     66
Expert Participation in Test Development     66
Student Participation in Small Sample Pilots and Large Sample Field Tests     67
Test Specifications and Content Validity     68
Components of Test Specifications     68
Documenting Content Validity for English Language Learners: The Access Specifications Package     70
Alignment Considerations     73
Overview     73
Alignment and English Language Learners     76
Providing the Foundation of Principled Test Construction: Maintaining the Integrity of the Item Targets     81
Overview     82
Overview of Access and Item Development     84
The Centrality of Item Targets: Clearly Defining Intended Knowledge and Skills     88
Development and Completion of Access-based Item Templates     90
Access-based Item Development     101
Contextual Factors     105
Culturally broad experiences     105
Making Expectations Clear and Explicit     108
Prior Learning Expectations     110
Structural Factors     113
Simple Language Structures     114
Vocabulary     118
Effective Visuals     121
Item Format     128
Text Amount     132
Tools     134
Impact of Home Language     135
A Note about Rubrics     136
Tools, Test Forms, and Reviews     141
Tools     141
Text Supports     143
Manipulatives     145
Content-based Resources     147
Activities     148
Form Considerations     149
Choosing Items     150
Plain Language Forms     152
Computer-based Testing     153
L1 Form Considerations     154
Using Portfolios     157
Reviews     160
Internal Reviews     160
Fairness Reviews     161
Putting It All Together     167
In What Language Should English Language Learners Be Tested?     169
Defining ELLs     171
Variation Among ELLs     171
Patterns of Language Proficiency     173
Additional Sources of Linguistic Differences     176
Limitations of Official Definitions of "ELL"      177
Section Summary     179
Classifying ELLs     180
Schooling Background     180
Defining and Assessing Language Proficiency     181
Literacy and Academic Language as Part of Language Proficiency     182
A Modern View of Language Proficiency: Communicative Competence     183
Limitations of Mandated English Language Development Tests     185
Section Summary     187
Testing ELLs     188
Limitations of Testing ELLs in English     189
Limitations of Testing ELLs in L1     191
Major Challenges in the Selection of Language for Testing ELLs     192
Section Summary     194
Future Directions     194
Local Specificity of Testing Models     195
Optimum Number of Items as Testing Accommodation     196
Other Relevant Accommodations and Pretest Support     201
Promising Administration Accommodations     203
Extra Time     203
Small Group     204
Oral Administration     205
Language Liaison     206
Promising Response Accommodations     208
Written Home Language or Code-switching     209
Oral in English, the Home Language, or Code-switching     209
Demonstrated or Modeled Response     210
Pretest Support     211
Classroom Support     212
Family Assessment Night     214
Proper Assignment of Accommodations to Individual Students     217
Key Assignment Considerations     218
Student Factors     219
Accommodations     225
Capacity     227
Defining the Task     228
Policy-based Approaches     229
Research-based Approaches     233
Operational and Prototypical Systems     240
Guidance Models     241
Research-based Standardized Data Collection and Assignment Systems     244
Implications     251
Scoring Considerations for English Language Learners     255
Issues Which Impact Scoring     256
Linguistic Considerations Influenced by the Home Language     257
Cultural Considerations     262
Language Acquisition Issues     268
Access-based Scorer Training     271
Scoring Process Considerations     274
Selected Technical Considerations     279
Developing an Interpretable Research Agenda for Documenting Validity      281
Validation Design Model     282
Research Design Considerations     286
Selected Quantitative Approaches     288
Modeling Relationships of Variables: Selected Methods     289
Modeling Data from Continuous and Categorical Variables     290
Modeling Frequency Data from Categorical Variables     295
Comparison of Groups and Treatments: Selected Methods     297
Comparison of Test Score Level Data     297
Differential Item Functioning     298
Other Methods for Analyzing Items     304
Comparison of Model Structures     305
Qualitative Approaches     306
Tryouts and Pilots     307
Cognitive Labs     308
Comparability Considerations     310
Defining Comparability     310
Analyzing Construct Equivalence     313
Analyzing Score Equivalence     316
References     319
Index     345