Dokpamo: Upstage, SKT, and LG Advance to Final Three-Way Contest...Third and Final Evaluation Awaits (Comprehensive)

Motif Scores High on AAII Benchmark but Eliminated in Second Evaluation
"NVIDIA B200 GPU Support Increased to 1,000 Units for Three Teams"
"Third Phase Evaluation to Proceed as Planned... Final Two Teams to Be Selected"
"Support Method to B

In the second stage evaluation of the Ministry of Science and ICT's independent artificial intelligence (AI) foundation model project, known as "Dokpamo," Upstage, SK Telecom, and LG AI Research (in alphabetical order) were selected as the final three models to advance to the next round. Motif Technologies (Motif), which joined the evaluation process later than the others, caused a stir by taking the top overall spot on the Artificial Analysis Intelligence Index (AAII)—a renowned AI performance assessment index—but was nonetheless eliminated. The government plans to select the final two teams through the third stage evaluation scheduled for the end of this year.

On August 18, the Ministry of Science and ICT held a briefing at the Government Complex Seoul and announced that three teams—Upstage, SK Telecom, and LG AI Research—would advance to the next stage out of four participating companies in the second stage evaluation of the Dokpamo project. According to the Ministry, "Motif was eliminated by a narrow margin as a result of combining the scores from the benchmark evaluation (worth 40 points), expert evaluation (worth 35 points), and user evaluation (worth 25 points)."

Ryu Jemyung, Vice Minister of the Ministry of Science and ICT, is announcing the results of the second evaluation of the independent AI foundation model project at the Government Seoul Office in Jongno-gu, Seoul on August 18, 2026. Photo by Cho Yongjoon

According to the expert committee's assessment, Upstage received high marks for connecting its development achievements to platforms such as Daum and Timely, making the results tangible to the public, and for collaborating with FuriosaAI on neural processing units (NPUs), mitigating dependency on foreign hardware. SK Telecom achieved top-tier performance in mathematical reasoning and Korean language, and was highly evaluated for usability and practicality as its model is already deployed in large-scale commercial services. LG AI Research was recognized for differentiating its agentic AI and for establishing global collaboration strategies, including partnerships with international organizations, to increase global impact. While Motif was eliminated in this evaluation, it nonetheless demonstrated world-class performance on par with global big tech companies in advanced benchmark categories such as agent modeling and coding.

The government plans to expand support for high-performance graphics processing units (GPUs, NVIDIA B200) to the three teams advancing to the next stage. Support, which previously included 768 GPUs, will be increased to approximately 1,000 units.

Motif Eliminated Despite High AAII Score... "No Significant Gaps in Evaluation Criteria"

Looking at the detailed results of this second evaluation, for the benchmark evaluation (worth 40 points), the average score across all four teams was 22.5 points, with only a 4.0-point gap between the highest and lowest scores. None of the four teams exceeded 30 points in the benchmark evaluation. The expert evaluation (worth 35 points) yielded an overall average of 28.8 points and a 2.4-point gap between first and last place, while the user evaluation (worth 25 points) saw an average of 17.6 points, with a gap of 5.0 points between first and last place.

The Ministry of Science and ICT did not disclose the ranking of the scores for each evaluation category in this second stage. Vice Minister Ryu Jemyung explained, "The differences among the four companies for each evaluation item were so small that announcing a first-place winner would provide little benefit, and we took into comprehensive account the impact this might have on other companies." He added, "For the general public evaluation, in which over 1,400 people applied and 185 ultimately participated, its influence was not decisive in determining the teams to advance."

Although Motif made headlines by scoring the highest among all participating companies on a global benchmarking agency's assessment during the second stage of the Dokpamo evaluation, it ultimately failed to proceed. Motif’s “Motif3” scored 47 points on the AAII, surpassing SOLAR OPEN 2 (37 points, Upstage), A.DotX-K2 (35 points, SK Telecom), and K-XAONE 2.0 (31 points, LG AI Research). However, it was noted that the benchmark scores did not always reflect real-world model performance. It is interpreted that the expert (35 points) and user (25 points) evaluations, which together made up 60% of the total score, played a decisive role in the outcome.

The limitations of the AAII benchmark also had an impact. AAII is a composite index that converts several independent benchmarks—for domains such as knowledge, reasoning, and coding—into a single score, which means total scores can fluctuate significantly depending on the combination of benchmarks used. According to the Ministry, the National Information Society Agency (NIA) benchmark provided a more comprehensive assessment by evaluating Korean language, context suitability, safety, and reliability for models developed in Korea, helping to better assess their competitiveness.

The Ministry also dismissed industry concerns over "benchmarking" manipulation. Some in the industry had expressed worries that one of the participating companies in the Dokpamo project may have heightened their AAII scores by artificially boosting only the weakest items. Vice Minister Ryu Jemyung stated, "After consulting with the agency responsible for the assessment, there was no evidence of systematic memorization or overfitting for evaluation purposes, and the response we received indicated that there was no significant issue."

The government noted that, although Motif was eliminated from this evaluation, it nonetheless achieved levels on par with global big tech companies in challenging benchmark domains such as agent modeling and coding. Vice Minister Ryu Jemyung said, "Motif, despite being a small-scale R&D organization, achieved remarkable results," adding, "It appears Motif's elimination in the second stage evaluation was primarily due to a relatively lower score in usability compared to the other teams."

Two Teams Will Be Selected at the Third Evaluation... Support Approach to Be Fully Reconsidered

Ryu Jemyung, Deputy Minister of Science and ICT, is announcing the results of the second evaluation of the independent AI foundation model project on August 18 at the Government Seoul Building in Jongno-gu, Seoul. 2026.8.18 Photo by Jo Yongjun

After the briefing, Vice Minister Ryu Jemyung told reporters, "We deeply recognize the need to develop AI models at the highest global standards, and we are currently discussing with relevant ministries a new support system that transcends existing methods. We plan to announce more concrete details in the future."

The government is currently reviewing the development of "frontier-level" AI models. It is also looking into the integration and restructuring of related projects, including the existing Dokpamo initiative and the general-purpose AI (AGI) project.

However, the selection process for the final two teams through the third evaluation, which will be held between the end of this year and early next year, will proceed as scheduled. The government plans to reconsider its support mechanism for the teams after that. Vice Minister Ryu Jemyung said, "The third-stage evaluation and the selection of two teams will go ahead as planned," but emphasized, "Whether to continue support for these two teams in their current form in 2027 is under a comprehensive review."

This content was produced with the assistance of AI translation services.