Various exams and cover letters lose their power to distinguish ability
Rethink selection systems instead of tightening supervision

[The View] What Should Exams Ask Now? View original image

I recently asked students in my department, "Is it fair to earn an A+ by using artificial intelligence (AI) skillfully?" Most said no. They were not saying that people should not use AI. They meant that grades should show what a student knows and can do, and answers written by AI on a student's behalf do not show that. The students already understood precisely what economists call the essence of a "signal."


As the College Scholastic Ability Test (CSAT) approaches, AI glasses are drawing attention. According to the signaling theory of Nobel laureate Michael Spence, one condition must be met for test scores to serve as a signal of ability: the cost of achieving the same score must differ between people with and without ability. AI, however, erases that cost difference. People without the ability can easily imitate those who have it by using AI, eliminating the ability of test scores to distinguish between them.


This phenomenon has already been observed in the labor market. Researchers at Yale University analyzed what happened after an AI tool for writing cover letters was introduced on an online job platform. Before its introduction, the higher the quality of a cover letter, the more often employers contacted the applicant. After the tool was introduced, that correlation fell by more than half. Once anyone could produce polished writing, it became harder to gauge a job seeker's abilities from a cover letter alone. Employers soon began looking for other signals.


In fact, selection systems have always broken down and been rebuilt in this way. Because it was difficult to distinguish students based solely on school grades, which varied from school to school, a nationwide standardized test was introduced. As criticism grew that the test measured only memorization, it was redesigned as the CSAT, which draws on the U.S. SAT and tests critical thinking. The United States followed a similar path. As calculators became widespread and calculation skills no longer served as a signal of mathematical ability, the SAT began allowing calculators in 1994 and shifted its questions toward reasoning. Rather than blocking new technology, the testing system was redesigned.


Universities in the United States are facing a similar problem. As AI has weakened the connection between the quality of essays and students' abilities, some prestigious universities have eliminated essays and begun requiring standardized tests administered under supervision again. But even that last line of defense, supervised testing, is at risk of becoming ineffective in the face of hardware like AI glasses. This year's CSAT is a prime example. AI glasses have not become a major source of controversy in the United States, perhaps because admissions criteria there are spread across grades, essays, and tests. In the CSAT, where everything hinges on a single day's exam, the incentive to cheat is correspondingly greater.


Ultimately, the long-term answer lies in redesigning how students are assessed. To give an example from my own teaching, I ask students to submit the full record of their conversations with AI along with their reports. I then create different personalized questions for each student based on the report and conversation record. A student who has thought independently, even with AI's help, can answer these questions easily, while someone who left everything to AI will find them difficult. This allows me to determine whether students truly understand and can apply what they learned in class.


At first glance, this approach may seem to require much more time and money, but that is not necessarily the case. I also use AI to help create personalized questions. A professor at New York University in the United States administered oral exams using an AI voice agent, at a cost of just 42 cents per student. Individual assessments that had seemed prohibitively expensive have become almost free thanks to AI. In other words, AI has not only lowered the cost of cheating the existing signals, but also the cost of verifying individual abilities.


Of course, this approach cannot simply be applied to this year's CSAT right away. Selection systems will eventually change to suit the AI era, but the generation passing through this transition will bear the full cost. As someone who was educated in Korea and took the university entrance exam, I deeply empathize with the anxiety and frustration felt by students preparing for this year's CSAT, which is just around the corner. We need to make choices now to minimize the burden on the generation going through this transition. Should we give exam proctors better magnifying glasses? Should we impose harsher penalties for cheating? Or should we start by asking again what can distinguish human ability in the age of AI?



Seo Boyoung, Professor at Indiana State University in the United States


This content was produced with the assistance of AI translation services.

© The Asia Business Daily. All rights reserved. Unauthorized AI training and use prohibited.

Today’s Briefing