The Necessary Move to Online Testing

With schools moving to remote learning in the last few weeks, more and more education activities previously thought to only be effectively delivered in-person have moved online. On Friday, College Board announced the first-ever at-home administration of the AP exams. Critics have questioned the validity of these new exams, though students overwhelmingly wanted the opportunity to demonstrate mastery of the material they’d spent the year learning. With the limitations put in place by necessary social distancing and stay-at-home measures, these at-home tests likely provide the best possible option in the next few months.

Accelerating an Already Growing Trend

The online AP exams won’t be the first online tests College Board or ACT have offered—they’ve provided online (though not remote) administrations of the SAT and ACT for the last several years. The test organizations have increasingly used this option in the last decade to increase security for international testers as well as provide another testing option for district and state-funded mandatory, school-day tests. ACT recently announced the first online testing option for national test date testers, signing up individually on Saturdays, as an option (at available testing sites). The move raises some big questions: Who benefits from online testing options? Are the scores comparable to pencil and paper testing? How can students prepare?

Who has Access?

Taking the ACT and SAT won’t typically be the first-time students will encounter technology in an educational setting. Indeed, the use of ed tech has increased significantly in recent years, as devices and high-quality tools have become available in more and more schools. Ultimately, students with increased familiarity and comfort working through computer-based assessments benefit from the online testing option. Access to those tools in schools is not distributed equally. Predictably, higher-income students have more access to educational technology tools and devices in their schools. New Schools’ recent survey found that while about 8 in 10 students have access to either devices on shared classroom carts or in classroom libraries (and around a third of students also have 1-to-1 devices available in their schools), there was variability in student groups. Students from low-income households, black students, students attending urban school districts, and students in the south were the least likely to report having their own device to use in a school setting. The decreased familiarity and experience for high-need students raises questions of equity in using online testing in a high-stakes setting like college entrance exams.

The remote, at-home testing option may exacerbate issues of access and equity. Around 10% of Americans don’t use the internet at all, with higher quantities in low-income and rural areas. College Board is working to offer options to those students for AP tests, though the extent and success of those options is not yet clear.

Does Test Format Matter?

After considering equity and access, it’s worth digging into the validity of these online exams. The AP tests this year will take a very different format than they have in the past (only 45 minutes and entirely free-response items). The ACT and SAT have offered identical versions of their tests in pencil and paper and online formats. Significant research has been done to see if students do perform similarly on the two formats, even when controlling for demographic factors. Differences in testing experience may lead to differences in performance for many students.

The Common Core PARCC test was given to five million testers nationally in a single school year. One in five was given the pencil and paper version—and, on average, they scored higher than the four in five taking the computer-based version of the test. In Illinois, 43% of testers taking the pencil and paper version scored proficient or above on the English Language Arts portion of the test. Only 36% of students taking the computer-based version performed similarly. In Baltimore, MD, researchers analyzed student performance on the two versions of the test while controlling for demographic and academic performance—and found that the pencil and paper testers scored 14 points higher than the computer-based testers. There’s certainly compelling evidence that these differences in test construct result in differences in performance—but does this translate to the college entrance exams?

Computer-Based Entrance Exams Early Research

Both ACT and SAT have provided preliminary reports on the comparability of computer-based and pencil and paper tests. The tests should be relatively similar—both the computer-based versions are linear tests with the same passages and items provided in the same order. College Board performed 3 studies on this question, randomly assigning students to take one version of the test. They found that while students performed similarly on the Writing and Language and Math sections, but that students performed higher on the computer-based version of the Reading test. The difference was equivalent to 5-10 points on the SAT scale, primarily originating from command of evidence items. These items require students to find the evidence in a long passage to support a claim from a prior question. It’s not terribly surprising, then, that the ability to scroll through the passage and highlight text on the screen improves student performance.

After initially adjusting the timing of the online ACT (and finding that it did not impact scores), ACT determined that the online version of the test should exactly mirror the pencil and paper version in terms of timing and question count. However, ACT found similar results in performance —that students performed better on the online version of their reading test as well. Both testing companies have moved to a policy of equating—adjusting the scale for the online version of the test to account for these differences. This means students may need to get more questions right on the computer-based version to get the same score as the print version, should equating remain necessary.

Preparing for Computer-Based Testing

With a variety of questions still out there on the comparability and equity of computer-based testing, students, families, and teachers may instead focus on how to best support students and prepare them for these tests should they be available online this spring and in the future. Familiarity with the tools available—such as using text highlighters and computer calculators—can help prepare students for their test-day experiences. In addition, consideration of how to interact with the passages and items using strategies they may have previously practiced with pencil and paper is essential.

Ultimately, the best way to prepare for these exams (and to prepare for college) is to continue rigorous education in the coming months. With a wealth of tools and e-learning opportunities, educators at Academic Approach will be working to engage our students in the process of becoming college-ready critical thinkers and readers both on and beyond the tests.