TESTING EMERGING MARKET

AI Safety Testing & Certification

A new safety testing, red teaming and certification services market is forming around the EU AI Act — as high-risk AI systems face mandatory conformity assessment and GPAI models with systemic risk face explicit adversarial testing obligations.

Key Deadline: 2 December 2027 (high-risk AI systems, Annex III)

Executive Summary

Regulation (EU) 2024/1689, the EU AI Act, is the world's first comprehensive AI governance law. It classifies AI systems by risk level and imposes binding obligations on providers and deployers of high-risk systems. From 2 December 2027, high-risk AI systems in sensitive areas — biometrics, employment, education, critical infrastructure — must complete conformity assessment, carry CE marking, and maintain full lifecycle risk management documentation.

The AI Omnibus (Regulation (EU) 2026/1744), which entered into force on 27 July 2026, extended the original high-risk deadline by 16 months but did not reduce the obligations. Fines remain severe: up to €35 million or 7% of global turnover for prohibited practices, and €15 million or 3% for high-risk requirement violations.

Alongside the AI Act, a commercial market for AI safety testing and red teaming services is emerging. Specialist platforms (Giskard, Holistic AI, Mindgard), major consultancies (EY, Deloitte, Accenture) and cybersecurity firms are building AI safety testing practices. Third-party market research reports estimate the global AI red teaming services market in the low single-digit billions, though estimates vary and are not independently verified.

This Topic Hub tracks whether an independent AI safety testing and certification services market is forming, who participates, what evidence supports the thesis, and what remains uncertain.

EII Judgment

What Is Real

The EU AI Act is binding law. GPAI obligations have applied since August 2025. The Commission's enforcement powers over GPAI are active since August 2026. The AI Omnibus has been adopted. ISO/IEC 42001 (AI management systems) is published. The AI red teaming market already has real revenue and real providers.

What Is Forming

AI safety testing platforms, red teaming-as-a-service, conformity assessment bodies preparing for notification, AI governance consulting, and model evaluation tooling are all clustering around regulatory requirements. Major cybersecurity firms are acquiring AI-native safety startups.

What Is Not Yet Proven

Whether notified body capacity will be sufficient for the volume of conformity assessments. Whether AI safety testing becomes a recurring compliance requirement or a one-time gate. Whether the market will consolidate around a few large platforms or remain fragmented. Whether non-EU jurisdictions will adopt equivalent requirements.

What Matters Now

High-risk system providers have roughly 14 months until the Annex III deadline. Conformity assessment requires documented risk management, adversarial testing evidence, data governance records and human oversight mechanisms — all of which take months to build. The preparation window is narrowing.

Research Assessment

83EII /100
78Human Value /100
60Crowding /100

EII Score reflects binding regulation with confirmed enforcement timeline and active market formation. Human Value Score reflects growing demand from AI developers and deployers but slightly lower direct public impact than commodity compliance topics. Crowding Score reflects significant and growing service provider activity — the AI safety testing market is more crowded than CBAM or EUDR at a similar stage.

Why This Market Is Forming Now

  • Regulation is enforceable. The EU AI Act is not a voluntary framework. It carries fines of up to €35 million or 7% of global turnover. The Commission has had enforcement powers over GPAI since August 2026. High-risk obligations follow in December 2027.
  • Conformity assessment is mandatory. High-risk AI systems must undergo conformity assessment before being placed on the EU market. Some require third-party notified body assessment. All require CE marking, technical documentation and an EU declaration of conformity.
  • Testing obligations are binding. High-risk AI systems face mandatory risk management, accuracy, robustness, cybersecurity and testing requirements under Article 15. For GPAI models with systemic risk, Article 55 additionally requires documented adversarial testing (red teaming). NIST AI RMF recommends pre-deployment red teaming. These are compliance evidence requirements, not suggestions — but the scope of adversarial testing obligations differs between high-risk systems and GPAI models.
  • Commercial service activity is real. Specialist testing platforms, major consultancies and cybersecurity firms are actively building AI safety testing practices. Third-party market research reports estimate the global AI red teaming market in the low single-digit billions of dollars, though estimates vary and are not independently verified by EII. The core market-formation signal is binding regulation, compliance deadlines and real service provider activity — not market size estimates.
  • Standards infrastructure is forming. ISO/IEC 42001 (AI management systems) is published and certifiable. ISO/IEC 42005 (AI system impact assessment) provides structured processes. These standards create the audit framework that safety testing services must map to.

Regulatory Timeline

Aug 2024
EU AI Act enters into force

Regulation (EU) 2024/1689 published. World's first comprehensive AI governance law. Establishes risk-based classification and binding obligations.

Source: EUR-Lex
Feb 2025
Prohibited practices and AI literacy apply

Ban on unacceptable AI practices (social scoring, subconscious manipulation). AI literacy obligations for providers and deployers.

Source: Art. 4-5, AI Act
Aug 2025
GPAI obligations apply

General-purpose AI model providers must conduct model evaluations, systematic risk assessments, and document adversarial testing. Transparency obligations for chatbots and deepfakes.

Source: Art. 51-56, AI Act
Aug 2026
Commission GPAI enforcement powers active

AI Office can enforce GPAI obligations. Fines for GPAI violations: up to €15M or 3% of turnover.

Source: AI Act enforcement provisions
Jul 2026 OMNIBUS
AI Omnibus Regulation (EU) 2026/1744

Extends high-risk system deadlines by 12-16 months. Simplifies notified body application procedures. Narrows safety component definition. Does not reduce core obligations.

Source: Regulation (EU) 2026/1744
2 Dec 2027
High-risk AI systems (Annex III) apply

Conformity assessment, CE marking, risk management, data governance, human oversight and transparency obligations for standalone high-risk AI systems in sensitive areas.

Source: Art. 6-15, AI Act as amended by AI Omnibus
2 Aug 2028
High-risk AI embedded in regulated products

AI systems integrated as safety components in machinery, medical devices, toys and other regulated products. Obligations fold into existing CE marking processes.

Source: AI Omnibus transitional provisions

Who Is Affected

AI System Providers

DIRECT LEGAL RESPONSIBILITY

Developers of high-risk AI systems must complete conformity assessment, maintain technical documentation, carry out testing for accuracy and robustness, and affix CE marking before placing systems on the EU market.

GPAI Model Providers

ACTIVE OBLIGATIONS NOW

Providers of general-purpose AI models (OpenAI, Google, Anthropic, Meta) must already conduct model evaluations, systematic risk assessments and document adversarial testing. Enforcement powers active since August 2026.

AI Deployers in Regulated Sectors

COMPLIANCE OBLIGATION

Banks using AI for credit scoring, hospitals using AI for diagnosis, employers using AI for recruitment screening — all must ensure their AI systems meet high-risk requirements including human oversight and transparency.

Product Manufacturers

INTEGRATED OBLIGATION

Manufacturers adding AI as a safety component in machinery, medical devices, vehicles or toys must integrate AI Act requirements into existing CE marking processes from August 2028.

Notified Bodies & Testing Labs

ACCREDITATION ROLE

Conformity assessment bodies preparing for notification under the AI Act. Must be designated by member states to perform third-party assessments. Capacity building is underway but not yet at scale.

What AI Safety Testing Requires

AI safety testing is not a single certification exam. It is a set of testing, evaluation and documentation activities that must be performed across the AI system lifecycle.

Adversarial Testing (Red Teaming)

Systematic attempts to provoke failures, biases, harmful outputs and security vulnerabilities from AI models. Includes prompt injection, jailbreak attempts, data extraction attacks and edge case exploration. Article 55 explicitly requires documented adversarial testing for GPAI models with systemic risk. High-risk AI systems under Articles 9–15 face testing, robustness and cybersecurity requirements that may include adversarial testing as part of conformity evidence, but the explicit adversarial testing mandate in Article 55 applies specifically to GPAI models.

Risk Management System

A continuous lifecycle process for identifying, assessing and mitigating AI risks. Must cover design, development, deployment and post-market monitoring. Not a one-time assessment but an ongoing operating capability.

Conformity Assessment

Pre-market evaluation demonstrating that the AI system meets all applicable requirements. May be internal (for some systems) or require third-party notified body assessment. Results in CE marking and EU declaration of conformity.

Post-Market Monitoring

Ongoing surveillance of AI system performance after deployment. Incident reporting, drift detection, model update validation and continuous risk reassessment. Required throughout the system's operational lifetime.

AI Safety Testing Service Map

From regulation to emerging safety testing and certification market.

REGULATION & STANDARDS
EU AI Act
Regulation 2024/1689
AI Omnibus
Regulation 2026/1744
ISO/IEC 42001
AI Management
NIST AI RMF
Risk Framework
↓
TESTING REQUIREMENTS
Adversarial
red teaming
Robustness &
accuracy testing
Bias & fairness
assessment
Data governance
validation
↓
SERVICE LAYERS
Platforms
Red teaming &
evaluation SaaS
Consulting
AI governance &
compliance
Assessment
Notified bodies
& testing labs
Cybersecurity
AI-specific
security testing
↓
BUYERS
AI model
providers
High-risk AI
system deployers
Product
manufacturers
Regulated
industry firms

Emerging Service Ecosystem

AI Safety Testing Platforms

Giskard (Barcelona) — open-source AI testing platform with EU AI Act mapping. Holistic AI — AI risk management and testing. Mindgard — AI security testing platform. Kosmoy — self-hosted evaluation with regulatory mapping. Promptfoo — open-source red teaming with EU AI Act plugins.

Major Consulting & Advisory

EY, Deloitte, Accenture, PwC — all building AI governance and compliance practices. NTT DATA, NRI SecureTechnologies — AI security assessment in Asia-Pacific. Services include readiness assessments, gap analysis, risk management system design and conformity assessment preparation.

Cybersecurity Firms (AI Extension)

CrowdStrike, Check Point (acquired Lakera), HiddenLayer, Trail of Bits, Bishop Fox — extending traditional security testing into AI-specific adversarial assessment. M&A activity accelerating: large security groups acquiring AI-native startups.

Standards & Certification Bodies

ISO/IEC 42001 — certifiable AI management system standard. BSI, TÜV, SGS — preparing AI Act conformity assessment capabilities. Existing notified bodies under MDR/IVDR streamlining applications for AI Act notification.

Evidence Classification

What Is Verified

  • EU AI Act (Regulation 2024/1689) is binding law
  • GPAI obligations active since 2 August 2025
  • AI Omnibus (Regulation 2026/1744) adopted July 2026
  • High-risk deadline: 2 December 2027 (Annex III)
  • Fines: up to €35M or 7% global turnover
  • ISO/IEC 42001 (2023) published and certifiable
  • AI safety testing services market has real commercial activity (third-party market size estimates vary and are not independently verified)

What Is Inferred

  • AI safety testing will become a recurring compliance requirement, not just pre-market
  • Notified body capacity will be a bottleneck for conformity assessments
  • Market consolidation likely as large cybersecurity firms acquire AI-native startups
  • Non-EU jurisdictions will adopt equivalent requirements, creating global demand

What Is Unknown

  • How many notified bodies will be designated and by when
  • Whether AI safety testing becomes commoditised or remains specialised
  • How member states will differ in enforcement approach
  • Whether open-source AI models face different compliance pathways

What Could Break the Thesis

  • Further delays. The AI Omnibus already extended deadlines by 12-16 months. Political pressure could trigger additional delays, reducing urgency for compliance investment.
  • Enforcement inconsistency. Member states must designate competent authorities and notify conformity assessment bodies. If capacity varies widely, the level playing field may not materialise.
  • Open-source exemption gap. If open-source AI models receive broad exemptions, a significant portion of AI development may operate outside the compliance framework, reducing the addressable market.
  • Testing commoditisation. If automated scanning tools and open-source red teaming frameworks become sufficient for basic compliance, the premium testing and consulting market may be smaller than expected.
  • Jurisdictional fragmentation. If different jurisdictions adopt conflicting AI governance frameworks, service providers may face complexity costs that limit market growth.

What Companies Should Prepare

  1. Classify your AI systems — determine which fall under prohibited practices, high-risk (Annex III), limited risk or minimal risk categories
  2. Map conformity assessment routes — determine whether your systems require internal control or third-party notified body assessment
  3. Build risk management systems — establish lifecycle risk management processes covering design, development, deployment and post-market monitoring
  4. Conduct adversarial testing — run red teaming exercises covering prompt injection, jailbreak, data extraction, bias and harmful output scenarios
  5. Prepare technical documentation — compile the evidence required for conformity assessment including data governance, human oversight and transparency measures
  6. Implement post-market monitoring — establish incident reporting, drift detection and continuous risk reassessment processes
  7. Monitor notified body availability — check whether member states have designated conformity assessment bodies in your AI system's domain
  8. Map to ISO/IEC 42001 — align your AI management system with the certifiable standard, which increasingly serves as the baseline for procurement requirements

Sources

Tier A — Primary Regulatory Sources

  • EUR-Lex: Regulation (EU) 2024/1689 of the European Parliament and of the Council of 13 June 2024 laying down harmonised rules on artificial intelligence (Artificial Intelligence Act). OJ L 2024/1689.
  • EUR-Lex: Regulation (EU) 2026/1744 — AI Omnibus. Amending the AI Act as regards timelines and simplification measures. Entered into force 27 July 2026.

Tier B — Official / Institutional Sources

  • European Commission AI Office: EU AI Act implementation guidance, GPAI Code of Practice, high-risk classification guidance.
  • ISO/IEC 42001:2023 — Artificial intelligence — Management system standard. Published and certifiable.
  • NIST AI Risk Management Framework (AI RMF 1.0) — US framework for AI risk management, referenced by EU AI Act testing requirements.

Tier C — Commercial / Industry Sources

  • Giskard: Open-source AI testing platform with EU AI Act compliance mapping. Barcelona. giskard.ai
  • Holistic AI: AI risk management, testing and compliance platform. holisticai.com
  • QYResearch: Global AI Red Teaming Services Market Report 2026-2032. Market size estimates and growth projections.
  • Promptfoo: Open-source LLM red teaming tool with EU AI Act testing plugins. promptfoo.dev

Tier D — Industry Media (not official sources)

  • Multiple law firm publications on AI Act implementation timelines, conformity assessment pathways and AI Omnibus implications.
  • Gartner Market Guide for AI security and Guardian Agents (2025).

Published: 2026-10-06 · Last updated: 2026-10-06 · Status: TESTING

This is an independent research publication by Emerging Industries Intelligence. It is not legal advice.