AI Becomes a "Token-Eating Monster": The Thrifty Will Win [Tech Talk]

Token Costs Soar as AI Becomes Mainstream
Exponential Token Consumption in Coding Tasks
Success Depends on Mastery of Tokenomics

Editor's NoteFrom AI, semiconductors, and telecommunications to biotech, we break down the essential yet often unfamiliar technologies that are shaping our daily lives in an easy-to-understand way.

Major artificial intelligence (AI) companies such as Google, OpenAI, and Anthropic set their service prices based on "tokens." Depending on the subscription plan, users are allotted a certain number of tokens for a specified period. However, as AI is now being used for general office work, coding, and various other tasks, the volume of tokens consumed has surged. It has become increasingly common for the number of tokens that previously lasted a month to be depleted in just a few hours. As the costs required for cutting-edge AI development rise exponentially, there are predictions that, in the future, how efficiently one uses tokens will become a key measure of competitiveness.

The Minimum Standard for AI to Understand Our Language: Tokens

Artificial intelligence (AI) processes human text by breaking it down into token units. Because of this, the cost of AI is also calculated based on the number of tokens. Pixabay

A token is the smallest unit that large language model (LLM) AIs use to process natural language data. Just as we break up words according to the combination of alphabet letters or Hangul characters, AI also recognizes and interprets human language in token units. Generally, one English word or one to two Korean characters correspond to a single token.

Since token units are the fundamental data processing standard for AIs, AI companies also base their pricing on tokens. More specifically, the price is determined by the number of tokens an AI outputs when responding to a user. The AI chatbot subscription plans from Google, OpenAI, and Anthropic all allocate tokens. For instance, if you look closely at OpenAI’s pricing details, you’ll find a breakdown of how many dollars you pay for every one million tokens output by the chatbot.

The Cost of AI Conversations Determined by Tokens

ChatGPT logo. Photo by AP Yonhap News

A few years ago, when AI was just a novel gadget or a foreign language translator, there was little need to worry about tokens. The issue has emerged now that AI is actually used for all kinds of complex work such as office tasks, legal services, or coding. Even asking an AI to develop a simple program can result in the creation of tens of thousands, or even hundreds of thousands, of lines of code at once, causing token usage to skyrocket. In particular, programmers frequently interact with AIs to refine their code, checking whether it works as intended and troubleshooting errors, a process that can easily consume millions or tens of millions of tokens.

In fact, on Reddit, the largest online community in the West, developers are constantly posting questions about how to reduce token consumption on ChatGPT, Claude, and Gemini. Even companies that actively use AI find token costs burdensome. According to the U.S. tech media outlet Wired, executives at major IT companies such as Meta, Uber, and Salesforce have recently expressed concerns about skyrocketing token costs. Last month, Microsoft canceled its engineers’ Claude subscriptions and encouraged the use of its own products like Copilot for the same reason: the burden of token expenses.

Tokenomics Will Determine Future Corporate Competitiveness

NVIDIA's AI Supercomputer Center. NVIDIA

The burden of tokens is expected to keep increasing. As AI becomes more sophisticated, it will penetrate even more business fields, naturally increasing the number of tokens processed and output. For this reason, some observers suggest that "using AI well" will soon mean "accomplishing work with as few tokens as possible." A new term, "Tokenomics" (token economy), has emerged to describe this focus on maximizing token efficiency in AI usage.

According to a report on "AI Tokenomics" released last month by the global market research firm Gartner, "Token-based billing is changing the cost structure that used to be based on traditional IT subscription services," warning that "if developers’ token usage is not made visible, it will be impossible to track budget overruns and value for cost." In short, to save tokens, companies need tools that allow them to monitor at a glance how many tokens their developers are consuming for their work.

Gartner emphasized, "With the spread of AI, token consumption is bound to increase, and companies are now at a stage where they must manage not only the performance of AI but also the efficiency of token usage."

This content was produced with the assistance of AI translation services.