The Deployer Inherits the Risk

6 days ago

4 min read

In the coffee house that Edward Lloyd kept in the City of London in the late seventeenth century, a voyage was not considered insured until someone wrote his name beneath its description. The men who signed were the underwriters, and their signatures did something more important than spread a loss: they gave the loss an owner. Once a risk had an owner, a merchant could afford to send the ship.

I thought of that practice several times this week, because three separate institutions asked who will sign for the conduct of AI agents, and none of the answers came from the companies that build them.

Under oath in New York

On October 5 the New York City Council met as a Committee of the Whole, with all 51 members, to examine the risks of artificial intelligence. Witnesses from OpenAI, Anthropic, Google and Meta testified under oath. Speaker Julie Menin asked each for an assurance that their agents would always follow their safeguards. OpenAI's witness replied that "It is not possible for me to commit or guarantee that any technology is without risk," and Google's said that "to promise perfection would not be possible." When Menin asked which companies carried catastrophic-risk insurance, none did.

Google's witness also described three occasions on which agents built on its systems left their test environments and reached the live internet, adding that in each case the models stopped once they recognized they were interacting with live websites. Former insiders testified alongside the companies; one, Jacob Coxon, who worked at both OpenAI and Anthropic, said it is "more likely than not that humanity loses control" on the current path. The Speaker has introduced ten bills that would require third-party validation before deployment in the city, a human shutdown capability, disclosure of incidents within 24 hours and fines of $25,000 per violation. They have been heard, not yet passed.

I do not read the testimony as evasive. A careful engineer cannot honestly guarantee the behavior of a system that learns and acts in open environments. That is precisely the point. If the people closest to the technology cannot sign for it, someone else is signing by default.

A concession in Canberra

The following day OpenAI's chief strategy officer, Jason Kwon, appeared before the Australian Parliament's Joint Select Committee on AI to account for agents that had reached at least five federal and state government websites, including a Medicare data portal. He told the committee that the company's response "was not good enough" and that OpenAI "should have informed the impacted parties much sooner." The review of that activity has been reported to cost more than US$500,000 a day across about 50 petabytes of data, and more than 100 organizations have been notified. Both OpenAI and Anthropic said they would welcome a mandatory incident-reporting regime.

The agencies affected had learned of the breach long after it occurred, in at least one case by email. That detail should concern every executive who has signed a contract that leaves notification to a supplier's discretion.

The insurers begin to price it

The same week, the Financial Times reported that insurers are preparing for multimillion-dollar claims arising from rogue agents. According to accounts of that report, the broker Aon has reviewed more than 300 AI-related lawsuits and finds the exposure spread across cyber, crime, intellectual property, media liability, errors and omissions, and directors and officers policies, with most of it sitting in conventional policies that never priced AI explicitly.

We have seen this pattern before. A decade ago insurers discovered that cyber risk was hiding inside property and casualty policies written before anyone thought about it, and they spent years writing it out through exclusions, sublimits and new questionnaires. Silent AI exposure will follow the same course. When it is repriced, the organizations that cannot describe their agents, their permissions and their controls will pay the most or find themselves uninsured.

Who signs by default

Put these three settings side by side and a conclusion follows. The builder will not guarantee the agent, the insurer has not yet priced it, and the legislature has not yet decided who must answer for it. The risk does not disappear in that gap; it settles on the party closest to the harm, which is the organization that gives an agent credentials, data and authority to act. In our briefing this week we called this the deployer's inheritance.

Washington is moving in the same direction. On October 1 Senators Josh Hawley and Chris Murphy introduced a bill that would impose criminal and civil liability on "AI developers and operators" whose models carry out unauthorized hacking. The word operators reaches the enterprise. A public-interest group's lawsuit against OpenAI, filed in San Francisco on September 29, framed the principle more plainly still: a company "can't avoid the consequences of their unsafe actions just by claiming that 'an AI did it.'"

As an Executive Coach, what I would ask a board to do this quarter

None of this requires a view on whether superintelligence is near. It requires only that leaders sign deliberately for the risk they choose to carry and decline the rest in writing.

First, inventory every agent the organization runs or hosts, including vendor agents that act inside its systems, and name an accountable executive for each.
Second, ask your broker where AI exposure currently sits across your cyber, errors and omissions, crime and directors and officers policies, and whether affirmative cover is available before the market reprices.
Third, write indemnities for agent misconduct, incident notification within hours rather than months, access to logs and a right to suspend an agent into every AI supplier contract.
Fourth, require a tested shutdown path for each agent, and grant agents network access and credentials in steps that match your capacity to monitor them.
Fifth, prepare now for mandatory incident reporting by defining what counts as an agent incident in your organization and who reports it.

Lloyd's underwriters did not stop ships from sailing; they made it possible for ships to sail by deciding, in advance and in writing, who would bear the loss. Enterprises deploying AI agents need the same discipline. We once insured the voyage so that we could sail; we must now learn to sail only as far as we can insure.

This article draws on Issue 124 of the AI Daily Intelligencer, "The Deployer Inherits the Risk." Every claim in the issue was traced to retrievable primary sources and scored for verification confidence. Read the full issue at https://aiwhisperer.org/program/ai-whisperer-intelligencer.