Using AI to Generate English Passages
Ambiguous Accountability and Security Risks

[Inside Chodong]CSAT Difficulty: AI Cannot Be the Solution View original image

"Adjusting the difficulty level of the College Scholastic Ability Test (CSAT) is 'the realm of the divine.'


Was this an attempt to ease the burden of ensuring an 'appropriate level of difficulty'? Lee Geunho, acting president of the Korea Institute for Curriculum and Evaluation (KICE), which oversees CSAT question development, made this remark last month during a briefing to the National Research Council for Economics, Humanities and Social Sciences under the Prime Minister's Office, when asked, "Is it really that difficult to control the level of difficulty?" He faced significant criticism for his response. While his comment was likely intended to highlight how challenging the task is, the fact that 8 out of the 12 previous heads of KICE have resigned mid-term due to issues related to CSAT questions makes the phrase "the realm of the divine" sound like an attempt to evade responsibility. This is why many argue that "unreasonable mistakes should not be dismissed as an act of God." What the education authorities should do is not invoke divine intervention, but rather analyze the causes of failure and improve the system.


This year's CSAT English section was flawed throughout the entire process, from the vetting of question writers to question creation and review. After the controversy over so-called "killer questions," KICE claimed to ensure fairness by randomly selecting question writers and reviewers. However, this failed to sufficiently reflect their experience in question creation and writing, resulting in a lack of expertise. Only 33% of the English question writers were teachers, lower than the 45% in other subject areas, meaning that the group most familiar with students' actual academic abilities was underrepresented. As a result, while only one Korean language question and four math questions were changed, 19 out of 45 English questions were replaced. Given that questions were being revised right up until the test, it is unlikely that the difficulty level was properly checked.


The Ministry of Education has proposed follow-up measures such as strengthening the vetting of question writers' expertise, increasing the proportion of teachers to 50%, and establishing an 'Educational Assessment Question Support Center.' Among these, the most notable is the plan to use artificial intelligence (AI) to generate English passages. By predicting difficulty levels using vast amounts of data, the hope is to avoid the wild swings between extremely difficult and overly easy CSATs. This could also ease the burden on question writers.


However, AI is not a panacea. Technology is only a supplementary tool and brings with it new risks. The most prominent of these is 'AI hallucination,' where the system fabricates plausible but false information. Even if AI produces grammatically perfect sentences, it may still generate passages that are factually incorrect. There is also the possibility of machine-generated passages that do not follow the logic of human thought. This means that the review process may need to be even more rigorous than it is now.


Security is also a major concern. Every year, the CSAT question development team creates questions in seclusion for over a month, completely cut off from the outside world. It must be carefully assessed whether such airtight security can be maintained when AI generates English passages, and whether simply building separate servers will suffice. As technology evolves, so do hacking methods, and unexpected incidents can occur at any time. It is also necessary to check whether there is enough time before the pilot implementation of the 2028 mock assessments (in June and September).


Nevertheless, what concerns test-takers most is the frequent policy changes. In college entrance exam communities, many posts ask, "How much longer will those born in 2009 have to be used as guinea pigs?" This is because, following the shift to an integrated CSAT, yet another variable is being introduced.



While handing over the difficulty level to a higher power and expecting solutions from AI, the question of responsibility becomes increasingly blurred. If a problem arises with AI-generated passages, who will be held accountable? Will it be KICE under the Prime Minister's Office, or the Ministry of Education, which holds ultimate responsibility for the nation's education? Before invoking higher powers or AI as arbiters of difficulty, the chain of accountability must be made clear. After all, the CSAT is a test designed and overseen by people. We cannot ask AI to resign mid-term, can we?


This content was produced with the assistance of AI translation services.

© The Asia Business Daily. All rights reserved. Unauthorized AI training and use prohibited.

Today’s Briefing