The authors of the paper “Attention Is All You Need” pose for a photo at NVIDIA’s “Transforming AI” session at GTC 2024. Third from the right is Jakob Uszkoreit, and fifth from the right is NVIDIA CEO Jensen Huang. Photo: NVIDIA

The authors of the paper “Attention Is All You Need” pose for a photo at NVIDIA’s “Transforming AI” session at GTC 2024. Third from the right is Jakob Uszkoreit, and fifth from the right is NVIDIA CEO Jensen Huang. Photo: NVIDIA

View original image

“Attention Is All You Need.” This was the title of a paper presented at the 2017 Conference on Neural Information Processing Systems by a team of eight Google researchers, including Jakob Uszkoreit. The title was a parody of the Beatles song “All You Need Is Love.”


The paper focused on the Transformer architecture, centered on self-attention. Self-attention calculates how the words in a sentence relate to one another and, when processing each word, incorporates information from other words according to their importance. A Transformer is a neural network built around self-attention. Recurrent neural networks, which had been widely used, processed words in sequence, passing information from earlier inputs on to the next step. As a result, they struggled to retain and use earlier information as text grew longer. By contrast, Transformers use self-attention to directly calculate relationships between words far apart in a text and process multiple words in parallel. This architecture was well suited to large-scale training. Combined with the powerful parallel-processing capabilities of graphics processing units (GPUs) such as those made by Nvidia, Transformers accelerated the development of large language models (LLMs) such as GPT, which in turn paved the way for ChatGPT.


Born in 1982, Uszkoreit studied computer science and mathematics at the Technical University of Berlin. His father, Hans Uszkoreit, studied computational linguistics, the field that enables computers to process language. Uszkoreit originally wanted to work in a computer-related field connected to biology, but after joining Google in 2008, he became interested in language processing, his father's field, according to “Nvidia CEO Jensen Huang: The Thinking Machine.”


While considering how to use artificial intelligence (AI) to teach literacy, I suddenly wondered what Uszkoreit was doing. He left Google in 2021 and co-founded Inceptive. As the company's CEO, he is now leading the development of RNA-based drugs using AI. He has returned to his first interest: life.


Language and RNA have something in common. A sentence is a structure in which words are arranged in a particular order, while RNA is a structure made up of sequences of four types of bases: adenine (A), uracil (U), guanine (G), and cytosine (C). Just as the meaning of a word is influenced by its relationships with other words, RNA's function is influenced by the sequence of its bases and the interactions between them. AI can help identify, among the many base sequences capable of producing the same protein, candidates likely to improve RNA stability and protein production efficiency while appropriately regulating immune responses.


Inceptive is applying this approach to the development of RNA therapeutics design technology. In June 2026, it signed a co-development agreement with RNA therapeutics company Alnylam Pharmaceuticals under which it could receive up to $2 billion, depending on future results.


Many AI researchers have entered drug development. But Uszkoreit's transformation is drawing particular interest because he was one of the key figures behind the development of Transformers. I hope to hear about his achievements before I find myself wondering about them again.



Woojin Baek, Economic Columnist


This content was produced with the assistance of AI translation services.

© The Asia Business Daily. All rights reserved. Unauthorized AI training and use prohibited.

Today’s Briefing