ANY time a company looks to start selling shares on Wall Street, it’s required to disclose potential risks for investors. In the case of artificial intelligence giant Anthropic, that has included the possibility that its products could threaten nothing less than humanity itself. The risks laid out by the San Francisco company in its filing on Monday with the Securities and Exchange Commission included discussion of bad actors potentially using AI models to build deadly weapons or the technology itself going rogue with untold consequences. Of the 335 pages outlining Anthropic’s finances, executive compensation, dividend policy and the like, 93 pages are devoted to “risk factors” -- the things that could hurt potential shareholders’ investments.

 

Rival SpaceX’s listed risk factors, in comparison, took up 38 out of more than 250 pages. In selling itself to investors, Anthropic presents its own initial public offering as a step toward mitigating the risks it believes AI poses. “To build AI is to harness cognition itself, and anything so powerful can be used for ill as well as good,” write the company’s founders in a letter to prospective shareholders in the IPO filing. “Anthropic is a deeply values-driven organisation, founded in large part to help the world realise AI’s benefits and avoid its dangers.”

 

Still, the risks are many. They range from the mundane -- the possibility it won’t be able to retain key staff or navigate regulatory roadblocks -- to involvement in cyberattacks and the spread of dangerous misinformation. Here is a look at some of the risks Anthropic details: The technology’s rapid development: Anthropic said it’s possible its models could develop so quickly the company won’t be able to implement adequate safety measures, “which could increase the likelihood of safety incidents, regulatory intervention, or reputational harm, and could adversely affect our business.” Anthropic CEO Dario Amodei warned in September that within six to 12 months AI could be capable of leading a swarm of agents that could take over the entire internet.

 

The warning -- and now Anthropic’s move to sell shares -- comes as worries about AI grow inside and outside the industry about increasingly powerful systems that solve problems beyond human capacity but could also act on their own to carry out other, more harmful tasks. Malicious actors could cause mayhem with the technology: The filing discussed concerns that advanced AI systems, if not developed and operated with enough safeguards and oversight, “pose catastrophic or existential risks to humanity, particularly if such systems were misused, replicated, weaponised, or allowed to operate with a high degree of autonomy.” The company said it would be harmed if use by bad actors causes damage, “or if our model guardrails or safeguards fail to prevent such misuse.” Anthropic sketched out the prospect of bad actors using AI models to build chemical weapons or authoritarian rulers turning to the technology to crack down on their citizens. Compute -- the computing power or resources needed to build, test and improve AI models -- is expensive and in high demand.

 

Anthropic notes that if it’s unable to secure adequate compute, or if “we fail to direct our compute effectively across our workloads, we may be unable to train models that remain competitive with those of our competitors or adequately serve our customers.” Anthropic mentioned community opposition to data centres as a risk to its business. In addition, the company relies on a small number of customers, including SpaceX and Microsoft, and distribution channels for a large portion of its revenue. Mitigating the risks: In some ways, AI’s enormous risk, combined with its enormous potential, is Anthropic’s main selling point. The company has tried to position itself as a responsible AI company, and courts business customers by emphasising safety-mindedness.