A Tale of Two Tests: The Role of Topic and General Academic Knowledge in Traditional versus Contemporary Scenario-Based Reading

Y

59

Institute of Education Sciences (ED)
Office of Naval Research (ONR) (DOD)

R305A150176
R305A180144
N000141712300

Comparative Analysis; High School Students; Inferences; Reading Tests; Reading Comprehension; Vignettes; Test Format; Task Analysis; Knowledge Level; Correlation; Readability; Readability Formulas; Team Sports; Multiple Choice Tests; History; Academic Language; Course Content

California; Idaho; Oklahoma

10.1016/j.learninstruc.2021.101462

We compared high school students' performance in a traditional comprehension assessment requiring them to identify key information and draw inferences from single texts, and a scenario-based assessment (SBA) requiring them to integrate, evaluate and apply information across multiple sources. Both assessments focused on a non-academic topic. Performance on the two assessments were moderately correlated (r=0.57), but the SBA was more difficult (Study 1; n=342). The two assessments similarly depended on basic reading skills but diverged in the relation to academic knowledge and (non-academic) topic knowledge (Study 2; n= 1,107). Academic knowledge was highly predictive of traditional comprehension, but less so for SBA. Topic knowledge was more predictive of SBA than traditional comprehension. Thus, the two assessments tap into similar constructs related to comprehension; however, the level of topic knowledge is more important for performance on scenario-based, multiple-source reading tasks, whereas academic knowledge is more important for traditional reading comprehension tasks. [This paper was published in "Learning and Instruction" v73 Article 101462 2021.]

As Provided

Yes

2021

ED611752