Armilla Review - AI Bias, Regulation, Safety, and Innovation

This week's review covers aspects of AI development and deployment, including identifying biases in AI audit systems, exploring new opportunities in AI insurance, balancing innovation with effective regulation, and advancing AI capabilities with tools and safety measures. Stay informed on the latest in AI governance, evaluation challenges, and groundbreaking industry practices.
June 5, 2024
5 min read

Potential Bias in IRS AI Audit System Identified

A Government Accountability Office (GAO) report revealed potential unintended biases in the IRS's AI-based audit selection system, the Dependent Database (DDB), which hasn't undergone a comprehensive review. The report highlights that outdated rules and filters in the DDB, alongside demographic biases in risk scores, might contribute to racial disparities in audit selection. Despite not collecting race and ethnicity data, the IRS's own research indicates racial disparities in audits. The GAO recommended the IRS adhere to an AI accountability framework and improve its monitoring activities to better identify high-risk noncompliant returns, which the IRS has agreed to implement.


AI Insurance: A New Frontier for Risk Management and Business Opportunity

The increasing adoption of AI technology introduces various risks that the insurance industry can turn into a significant business opportunity. A recent report by the World Economic Forum highlighted AI as the top technology risk for many organizations. Deloitte projects that by 2032, the AI insurance market could reach $4.7 billion annually, driven by the need to mitigate risks associated with AI failures. As AI becomes more integrated into daily life, insurers are beginning to develop policies to protect businesses from potential AI-related losses, similar to the evolution of cyber insurance. Early movers in this space, such as Munich Re and Armilla AI, are setting the stage for a burgeoning market by creating specialized AI insurance products and frameworks.

Source: Deloitte

Effective AI Regulation: Balancing Innovation with Vigilance

Regulating AI is challenging due to its rapid evolution and neutral nature, demanding regulations that focus on outcomes and require constant vigilance. The EU has developed a tiered framework categorizing AI risks, emphasizing the importance of traceability, testability, and liability in AI regulation. These principles ensure transparency, discourage cheating, and manage the risks associated with AI usage. Effective regulation must evolve continually to address new risks and maintain public trust without stifling innovation, similar to the approach taken with other rapidly advancing technologies like the internet.

Source: World Economic Forum

Struggles to Regulate AI's Role in Key Decisions Stall in US States

Efforts to regulate AI's influence on hiring, housing, and medical decisions are encountering significant challenges in US state legislatures. Of seven proposed bills addressing AI discrimination, only Colorado's has passed, albeit with reservations from the governor. These bills aim to require companies to assess AI's potential for bias and inform consumers when AI is used in decision-making. Despite broad acknowledgment of the need to address algorithmic discrimination, the complexity of AI technology, concerns over innovation, and fears of exposing trade secrets are hindering legislative progress. Critics suggest stronger anti-discrimination laws and independent testing organizations as potential solutions.

Source: AP News

Anthropic's AI Strategy: Balancing Innovation with Safety and Responsibility

Anthropic, an AI company co-founded by Dario Amodei, prioritizes safety over rapid deployment, delaying the release of its powerful chatbot, Claude, to focus on internal safety testing. This decision contrasts with competitors like OpenAI, highlighting Anthropic's commitment to preventing a race to build potentially dangerous AI systems. Despite the financial and competitive costs, Anthropic aims to set industry standards for AI safety, advocating for responsible scaling and regulatory frameworks. The company's approach seeks to align AI development with societal benefits, maintaining a balance between innovation and caution in a rapidly evolving field.

Source: Time

Claude 3 Models Now Capable of Advanced Tool Use for Enhanced AI Functionality

The Claude 3 model family now supports tool use, allowing it to interact with external tools and APIs for more dynamic and accurate responses. This feature, available on the Anthropic Messages API, Amazon Bedrock, and Google Cloud's Vertex AI, enables Claude to perform tasks such as extracting structured data, converting requests into API calls, and automating tasks. Companies like StudyFetch, Intuned, and Hebbia are leveraging these capabilities to improve educational platforms, browser automation, and complex workflows, respectively, demonstrating significant enhancements in efficiency and user engagement.

Source: Anthropic

Challenges and Opportunities in Evaluating Generative AI Applications

Andrew Ng highlights the challenges in evaluating generative AI applications, particularly those generating free-form text. Standardized tests for general-purpose models exist, but specific applications lack efficient evaluation methods. Creating labeled test sets is expensive, and using advanced language models to evaluate outputs can be noisy and costly. Ng emphasizes the need for better techniques to improve evaluation processes, suggesting that advancements in automated evaluation methods will be crucial for progress in the field.

Source: X

Resource for the EU AI Act Now Available

The EU AI Act, formally approved by the European Council on May 21, 2024, will soon enter into force, setting a global precedent for AI regulation with its human-centric and product-safety approach. The IAPP EU AI Act Topic Page offers a one-stop-shop for resources, including official texts, compliance matrices, stakeholder maps, and implementation steps, providing continuous updates and guidance on this pivotal legislation.

Source: International Association of Privacy Professionals