My breath taken, I looked up from my desk to gaze at the red-orange blanket of fallen leaves just outside my window. I needed the pause, or maybe the distraction, from what I was reading.
The thick document before me looked like any other standard business proposal: a prospectus for an initial public offering with boilerplate language about dividends, compensation and Class A stock. But the more I read, the more it morphed into experimental literature, equal parts business plan, philosophical treatise and sci-fi thriller.
This will be an I.P.O. unlike any other, its language both matter-of-fact and unnerving, its ultimate subject our future. Not futures, as in the stock market, but the future, as in what it might look like and whether we have one.
The prospectus comes from Anthropic, the artificial intelligence concern immersed in the technologies rapidly transforming the human experience. It is poised to offer shares to the public for the first time, and this document, recently filed with the Securities and Exchange Commission, lays out its blueprint for potential investors to decide whether to bet on the future of civilization.
I cannot give you odds on this existential gamble. My prior awareness of Anthropic extended little beyond clicking on a button with hundreds of thousands of other writers to join a massive class-action copyright lawsuit. As someone who knows next to nothing about investments, I read the Anthropic prospectus solely out of my vested interest as a human being.
In seeking the considerable capital needed for chips, data centers and electricity, the company — with an estimated valuation well north of $1 trillion — begins its pitch by dangling the possibility that artificial intelligence could “solve problems that have resisted generations of human effort.”
It asks us to imagine its A.I. models as “a country of geniuses in a data center,” working in concert to improve our quality of life. Helping to cure disease. Connecting the underserved to medical expertise. Increasing economic productivity. Improving government services. (In California, it says, Anthropic’s chatbot, Claude, is already “helping to shorten the lines at the DMV.”)
You’d have to be a robot not to … Let me rephrase: You’d have to be a toaster oven not to want an end to both cancer and the long lines at the DMV.
Anthropic is a public benefit corporation, dedicated to balancing social responsibility with profit, and its seven co-founders have championed a measured approach to developing what the prospectus calls “the most transformative technology in human history.” It advocates for developers to compete over safety and security, “rather than rushing to release increasingly powerful models without adequate safeguards.”
In discussing a particularly advanced form of A.I. that Anthropic expects to match or surpass human capability and become widely available, the document says: “If Powerful A.I. is directed at solving humanity’s hardest problems and the risks are managed responsibly, the potential is extraordinary.”
If.
If we work together to ensure the safe and responsible growth of machine intelligence, the prospectus seems to be saying, what a wonderful world this will be (notwithstanding an economic disruption “greater in magnitude” than any other technical advancement in history). If not, then economic upheaval may be the least of our worries.
Keep in mind, dear potential investor: “To build A.I. is to harness cognition itself, and anything so powerful can be used for ill as well as good.”
The foreboding I began to feel intensified the more I read. Throughout the document’s 300-plus pages are hair-on-fire sentences delivered in the flat language of business. Sentences such as:
“A.I. can be misused by those who wish to do harm, for example to conduct cyberattacks or to design biological weapons.”
And:
“Authoritarian governments may use A.I. to create high-tech surveillance states, deepen repression, and make themselves much more difficult to dislodge.”
Even more unsettling is the acknowledged risk of artificial intelligence becoming so autonomous that humans might lose control: “Our A.I. models may behave in unintended, misaligned, or autonomous ways, including by taking harmful actions that no user or third party directed.”
It is one thing to recall similar cautions from the astrophysicist Stephen Hawking in a 2014 interview; quite another to read these warnings amid discussions of income tax implications.
At times, the prospectus evokes a muffled Mayday call struggling to convey calm. Consider this mouthful of a paragraph:
“Concerns have been raised both within our organization and by other researchers, regulators, policymakers, and industry participants that sufficiently advanced A.I. systems could, if not developed, deployed, and operated with appropriate safeguards, oversight, and security measures, pose catastrophic or existential risks to humanity, particularly if such systems were misused, replicated, weaponized, or allowed to operate with a high degree of autonomy.”
Catastrophic or existential risks to humanity?
To Anthropic’s credit, it dedicates more than a quarter of its prospectus — 93 pages — to “risk factors,” a rare, perhaps unprecedented, transparency about the potential downsides of a business proposal. Imagine, for example, if Facebook’s initial public offering in 2012 had, in addition to boasting of a more open and connected world, addressed the mental health impact on children.
Anthropic does not understate the magnitude of the risks it lays out. Perhaps there was no way to mention only in passing the potential for the end to life as we know it.
Last year the company’s chief executive officer, Dario Amodei, put the odds of A.I. leading to the end of civilization at between 10 and 25 percent. He has since downplayed the doomsday scenario, arguing in part that the odds drop if governments and the tech industry commit to a more deliberate, safety-first approach to A.I. development.
Beyond its conjuring of a new and better world, the Anthropic prospectus is, in the end, an investment proposal in which Dr. Amodei and his co-founders each have a stake worth an estimated $18 billion. Such outsize numbers only intensify the sense that a chosen few humans — fallible, like all humans — will decide what comes next, leaving the rest of us to hold our collective breath.
I came away from reading the Anthropic prospectus with no desire to get in on the ground floor of a never-before-seen opportunity, no matter how transparent the pitch. I didn’t want in on this cosmic bet.
Instead, I got up from the desk and took my dog for a long walk through another Technicolor autumn day, so familiar and, suddenly, so precious.
