By: Griffin Wahl
The case for independently assuring AI systems has never been stronger. And yet, demand for external AI assurance remains well below where it needs to be. That gap between the value assurance offers and the rate at which organizations actually seek it is the subject of the report from the Partnership on AI (PAI), Demand and Incentives for External AI Assurance. Armilla's Head of Distribution, Phil Dawson, participated in PAI's 2025 AI Assurance Workshop, which informed the report.
The report is part of PAI's broader Strengthening the AI Assurance Ecosystem series and one of the most practically grounded analyses of the AI governance challenge I've seen. It names the barriers, identifies who needs to act, and explores where insurance fits into the picture. As a company building AI liability insurance for enterprise deployers and AI solution vendors, much of what it describes maps directly to what we see in the market.
Here's what stood out.
The report identifies three drivers of demand for AI assurance: commercial incentive, regulation, and liability. In theory, all three should be pushing organizations toward external evaluation of the AI systems they build and deploy. In practice, none of them are doing the job well enough yet.
Why? Five underlying factors:
1. Limited awareness of AI risk. Many organizations deploying AI, particularly outside heavily regulated sectors like finance and healthcare, don't have a clear picture of what can go wrong or how significant the consequences could be. Reputational risk in particular is systematically underestimated at the enterprise level.
2. Low awareness of assurance as a solution. Even organizations that understand their AI risk exposure often don't know that independent assurance services exist or that they can help. This isn't a supply problem so much as a visibility and education problem.
3. Regulatory fragmentation. With AI regulation evolving unevenly across jurisdictions, many organizations are in a wait-and-see mode, reluctant to invest in assurance before the regulatory landscape clarifies. The EU AI Act will create meaningful pressure for some categories of high-risk systems, but it won't reach everything.
4. Security and IP concerns. Developers worry about giving external assurers access to models, weights, and documentation. Building justified trust in assurers, including confidence in how they handle sensitive information, is part of the solution.
5. In-house expertise. Model developers have internal red-teaming and evaluation teams. This is valuable, but external, independent assurance carries a different kind of credibility for downstream users, regulators, and the public.
Each of these barriers are addressable. None of them go away on their own.
Several policy levers worth highlighting.
Regulation is the most direct. Mandatory conformity assessments for high-risk AI systems, as required under the EU AI Act, will create hard demand. But regulation alone won't reach the full breadth of AI deployment, and it can only work where agreed standards exist against which systems can actually be assessed.
Procurement frameworks are underutilized. Governments that require independent assurance as a condition of AI procurement, or that treat it as a positive factor in evaluation, send a powerful market signal. The report recommends that these frameworks be updated now, with a view toward making assurance a precondition for eligibility as the market matures.
Transparency mechanisms such as use case registers, incident reporting systems, and standardized documentation raise the visibility of AI failures and risk, which in turn increases demand for tools to prevent them. The OECD's AIM monitor and the EU AI Act's incident reporting requirements are early examples of this in practice.
Legal safe harbors are a more contested option. California Senate Bill 813 proposed exempting developers from tort liability if their models were reviewed and approved by independent panels. The report acknowledges the appeal, noting it creates a genuine incentive to seek external assessment, but raises reasonable concerns about whether a blanket tort exemption appropriately distributes risk between developers and the public, especially given where evaluation science currently stands.
This is, understandably, where our attention sharpened.
The report identifies insurance as a potential "virtuous circle," or "incentive flywheel," alongside assurance. The logic is compelling: insurers pricing AI risk accurately will offer lower premiums to organizations that have sought independent assurance, creating a direct financial incentive to do so. Over time, assurance and insurance reinforce each other, building trust across the ecosystem.
We believe this is the right long-term model. It's the direction Armilla is building toward.
The report is honest about where things stand today. Accurately pricing AI risk requires robust mechanisms to measure it, and those mechanisms are still maturing. Most insurance coverage is currently "silent" on AI, meaning policies don't explicitly address AI at all. As insurers continue adding AI exclusionary and inclusionary language to policies, it will accelerate the development of explicit AI-specific coverage and, with it, more sophisticated risk-pricing that rewards good governance practices.
The report calls on insurers to condition coverage or adjust premiums based on organizational risk management practices, including the engagement of independent assurance. And it highlights the value of insurers participating in standards development, evaluation research, and sharing insights from their own risk assessment work.
These are things we think about constantly at Armilla. AI liability insurance that doesn't engage with the underlying risk, that doesn't distinguish between organizations with strong governance practices and those without, isn't doing its job. I couldn’t agree more. The power of insurance as a governance lever depends entirely on how well it can measure and price what it's covering.
No single actor can solve this alone. Recommendations span policymakers, insurers, AI deployers, investors, and industry bodies, and for good reason. The assurance ecosystem is a coordination problem as much as a technical one.
A few priorities stand out as particularly urgent:
The full PAI report is worth reading. It's a good account of where the market stands and a practical roadmap for closing the gap between the value of AI assurance and the rate at which it's actually being used. You can access it here: Demand and Incentives for External AI Assurance
At Armilla, we build AI liability insurance for enterprises deploying AI, and performance warranties for vendors building and selling AI solutions. We believe that insurance, done well, is one of the most powerful tools available to align incentives in the AI ecosystem, rewarding organizations that govern their AI thoughtfully and providing meaningful protection when things go wrong.
If you're thinking about AI risk, governance, or insurance coverage for your organization, we'd like to talk.
Explore AI Insurance → | Get in Touch →
If you’re a broker looking to learn more, or in AI governance and looking to partner, reach out to Armilla’s Head of Distribution, Phil Dawson, or our Director of Underwriting, Andrew Correll.
Interested in chatting about AI, marketing, or design? Send me a message on LinkedIn.
Disclaimer: The insurance products described on this website are offered through surplus lines insurers. Surplus lines insurance is not subject to all the insurance laws and regulations of your state, and such insurers are generally not licensed by or subject to the supervision of the insurance department of your state. Coverage may not be available in all jurisdictions and is offered only through properly licensed surplus lines brokers.
Nothing on this website constitutes an offer to sell or a solicitation to buy insurance. No coverage will be bound or effective unless and until confirmed in writing by an authorized representative and all underwriting requirements have been satisfied. Any descriptions of coverage are for general informational purposes only and do not amend, modify, or supplement any actual policy or contract of insurance.
Please consult with a licensed surplus lines broker to determine availability and eligibility in your state.