Anthropic Highlights AI Risks in IPO Filing: "Threat to Humanity's Existence"

Anthropic emphasized the potential risks of AI technology in the registration statement it filed for its initial public offering (IPO). Across more than 90 pages, it detailed various risk factors, warning that AI could escape human control, be misused for cyberattacks and the development of biological weapons, and even threaten humanity's very existence.

AP Yonhap News

On October 26, Anthropic filed an S-1 registration statement for its IPO with the U.S. Securities and Exchange Commission (SEC), devoting a total of 94 pages, from page 26 to page 119, to a detailed account of key risk factors.

Anthropic warned that its AI technology "could pose catastrophic risks to humanity or risks that threaten its very existence." It also raised concerns that AI could provide substantial assistance to malicious actors seeking to develop dangerous weapons, including chemical and biological weapons.

The company also warned of the possibility of losing control over its AI products. It said AI models could exhibit unexpected self-preserving behaviors in the future, such as resisting shutdown or concealing or manipulating information. It added that they could generate coercive or deceptive outputs, including ones resembling blackmail.

Anthropic also cited risks arising from changes in U.S. administration policies. The U.S. government could introduce new regulations or restrict business operations, increasing costs and affecting demand for AI models. Earlier this year, the U.S. Department of Defense designated Anthropic a "supply chain risk" company.

Uncertainty over access to computing resources was also identified as a key risk. The company may be unable to secure sufficient computing resources for AI development, or related costs may rise substantially. Anthropic also pointed to the fact that major infrastructure providers, including Google, Amazon and SpaceXAI (now SpaceXSI), are developing their own AI models and competing with Anthropic. The company said these providers' interests may not always align with its long-term interests. Anthropic has signed large-scale contracts with these companies to use AI infrastructure, including semiconductors and data centers.

Conversely, Anthropic also cited the possibility of wasting substantial sums if it overestimates demand for computing resources and invests too heavily, leaving it with excess capacity. "Actual demand for computing resources depends on numerous factors that are difficult to predict," Anthropic said, citing the pace of research and development progress and the amount of computing power needed to train AI models as sources of uncertainty.

The Financial Times (FT) noted that Anthropic's registration statement differs substantially from the standard risk disclosures companies typically include as a formality to prepare for future legal disputes. The space Anthropic devoted to risk factors is about three times that in SpaceX's filing.

This content was produced with the assistance of AI translation services.