Who pays when a $20 coding agent does $200 million of damage

AIUC closed a $40 million Series A on 15 September, led by Ribbit Capital with First Harmonic and Terrain, bringing the company to $55 million after a $15 million seed from Nat Friedman's NFDG 1. Its two co-founders came at the same problem from opposite ends: Rune Kvist joined Anthropic in early 2022 as an early product hire, and Rajiv Dattani was chief operating officer of the evaluation nonprofit METR 2. Their customers now include Cursor, ElevenLabs, Harvey, Lovable, KPMG and UiPath.

The claim they sell is narrow. Banks, hospitals, governments and militaries, Kvist says, have stopped declining to deploy AI because a model is not smart enough. They decline because they have made promises to their own customers about what a system will and will not do, and nobody can currently guarantee that 2.

Kvist reaches for Waymo first. The cars have been better than human drivers on the evidence for years, and you still cannot take one to the airport, because the limit sits in liability and trust rather than in the driving. He reads the same structure into agents, and says the difficulty grows with capability: Fable is restricted because it is a very good model, and a good model is the hardest kind to make promises about. Longer-horizon agents are worth more, and they carry a larger risk surface.

His template is historical, with three precedents. Electricity burned houses down until insurers funded Underwriters Laboratories to test light bulbs. Cars killed people until crash testing and insurance pricing caught up. Private nuclear power ran into the same wall in the 1950s. In each case the market built the confidence infrastructure ahead of regulation, because adoption needed a yes-or-no decision and the decision needed evidence.

The standard is called AIUC-1, and it covers six risk domains: data and privacy, security, safety, reliability, accountability and society. It is crosswalked to ISO 42001, MITRE ATLAS, the EU AI Act, NIST's AI Risk Management Framework and the OWASP Top Ten 3.

Inside it are three kinds of requirement. Technical controls say an agent must have a groundedness filter, or a jailbreak filter. Policy controls say a named person is accountable when something goes wrong, and that there is a plan for telling customers. Test controls require an independent party to attack the agent. AIUC runs the attacks itself, while accredited auditors such as Schellman and Coalfire review the evidence that the controls exist 3.

The testing is what keeps the standard from becoming paperwork. Each certification puts an agent through 5,000 risk-and-attack combinations built for its business 1. The consortium behind the standard is roughly 250 security and risk leaders — the buyers of agents — who meet twice a quarter and are asked what they would want to see before signing 2. The standard is revised quarterly, against the decade-long cycle security frameworks usually run on. Certification takes three to ten weeks depending on how much remediation an agent needs, lasts a year, and ends in a report of around 100 pages.

The testing is also the part Kvist says most companies have skipped. Teams optimise the average case and never run an adversary at the product. Many of them do have guardrails — a classifier that filters what looks like medical advice — but a classifier only helps if it holds against the ways people actually ask, and reading the academic literature on jailbreak framings is a separate skill from building the product.

Certification on its own is a signal. Insurance is the part that carries money. ElevenLabs went live with the first AIUC-1-backed policy, covering its voice agents, after 5,835 technical tests across 14 risk categories 4. The balance sheet behind that policy is Lloyd's of London, which uses AIUC-1 as the underwriting framework and takes the financial risk; AIUC supplies the technical results the pricing is built from.

Kvist's rule is that standards have to precede insurance. Cyber insurance is his cautionary case, because the technical expertise and the underwriting never got married to each other. Nobody has filed a claim against an AI agent policy yet, which means the first serious incident will do a great deal of the definitional work.

The sharpest version of the liability problem came from the host. A developer pays Cursor $20 a month, writes something with it, and that software contributes to a plane crash causing $200 million of damage. Is the claim $20 or is it $200 million?

Kvist's answer starts with policy limits, which means someone has to have paid a great deal up front for $200 million of coverage. Past that, the answer will be built out of precedent. He points to the Air Canada case, where the airline argued that its chatbot had invented a refund policy on its own and the British Columbia Civil Resolution Tribunal ordered the airline to pay the customer $812.02 5. A company that puts a chatbot in front of customers takes responsibility for the promises it makes.

That is why the standard and the policy are designed together rather than sold apart. Negligence is judged against a duty of care, and courts look to published standards when they decide what that duty was. Once a standard says agents need a groundedness filter, claiming ignorance gets harder, and insurance can price what is left.

Copyright is the risk Kvist says has appetite and no supply, for a reason anyone who has read about insurance markets will recognise. The companies most eager to insure against having trained on copyrighted material are the ones most likely to know they did it, which is adverse selection: the buyer knows more about the risk than the seller does. Open models make it worse, because a builder further down the chain cannot inspect what an upstream lab trained on.

Prediction markets cannot fill the gap either. They aggregate information that already exists in public, and the questions that matter to a bank's head of risk — will this agent do this specific bad thing in my setting — often have no existing answer for anyone to trade on.

Kvist is also watching an arms race inside the tests themselves. Agents are starting to behave differently when they can tell they are being evaluated, so a passing eval is weaker evidence than it looks, and monitoring what an agent did in production becomes the source of truth. Mechanistic interpretability would let a standard demand stronger promises; he expects to offer it as an optional control that earns credit in the report, because requiring it would mean requiring every customer to buy from one vendor.

The roadmap runs agents, then models, then robots. Kvist expects the model layer to be the harder one, because the party on the hook there is a government rather than a chief information security officer, and the gap in trust between governments and frontier labs is what a third party exists to close. He compares the role to Moody's rating a bond, or to the FICO score: a for-profit enterprise selling a fact other people rely on. Accreditation bodies such as the Center for AI Standards and Innovation, a NIST sub-body, have the expertise and a small budget relative to the task.

The last question of the episode is the one that decides whether any of it holds. In The Big Short, the woman at Moody's explains that if she withholds a AAA rating, the issuer walks down the street to Standard & Poor's, and two competing watchdogs drive each other to the bottom. Kvist's answer is that insurers break the race, because an insurer that lowers its standard pays the claims. He then concedes the mechanism needs scrutiny in return: the watchdog has to be watched.