How to Test Agentic AI Systems in Banking Before DORA Makes It Mandatory

Why Classical QE Breaks on Agentic Systems

The fundamental premise of classical quality engineering is that a known input produces a predictable output, and deviation from that expectation is a defect. That premise held for decades of deterministic software. It does not hold for agentic AI systems, and applying it without modification is not just inefficient — it produces dangerously false assurance.

An agentic AI system is architecturally different from a model endpoint or a rules-based workflow. It operates through a planning loop: it receives a goal, decomposes it into sub-tasks, selects and invokes tools or sub-agents, interprets intermediate results, and revises its plan before taking the next action. The path from input to outcome is not fixed. It is shaped by the model's internal state, the outputs of external tool calls, the context window it holds at each step, and — in multi-agent configurations — the decisions made by peer agents it cannot fully control. Two identical prompts can legitimately follow different execution paths and still both be correct. A test suite that flags divergence as a failure will generate unacceptable noise. One that does not check behavioural constraints at each step will miss real risk.

The specific failure modes that break classical QE on agentic systems fall into several categories. First, action sequence variability: the agent may accomplish a goal through a different sequence of tool calls on each run, making trace-level assertions brittle. Second, emergent cross-agent behaviour: in multi-agent pipelines, no single agent's test coverage captures the risk that emerges from their interaction — a handoff between a summarisation agent and a decisioning agent can introduce context loss that neither agent's unit tests would surface. Third, memory and state drift: agents that maintain conversational or episodic memory accumulate state across sessions, meaning a test run that passes on a clean context can fail on a production context that has accumulated several hours of prior interactions. Fourth, tool call side effects: when an agent can write to a database, trigger a payment, or send a communication, the side effects of a test run are themselves a risk that classical test isolation assumptions do not account for.

Non-determinism is not a bug in agentic AI — it is the design intent. Every QE approach that treats it as an edge case is already wrong.

What DORA and EU AI Act Article 9 Demand from AI Validation Today

Two regulatory frameworks now create explicit, enforceable obligations that directly implicate agentic AI validation in banking and insurance. Understanding their intersection is essential for any QE lead who needs to defend a go-live decision to a CISO or internal audit team.

The Digital Operational Resilience Act, applicable across the EU financial sector from January 2025, establishes ICT risk management requirements that go beyond traditional IT controls. The Regulatory Technical Standards developed under DORA — specifically Article 25, which addresses testing of ICT systems — require that financial entities subject their ICT systems and tools to regular, risk-proportionate testing. For systems that support critical or important functions, this includes threat-led penetration testing and testing that reflects the realistic operational conditions of those systems. An agentic AI pipeline that handles customer credit decisions, fraud referrals, or claims processing is almost certainly supporting a critical function. A static regression suite run once before go-live does not satisfy the intent of Article 25, and any competent supervisory reviewer will read it that way.

The EU AI Act classifies AI systems used in financial services — particularly those affecting creditworthiness, insurance pricing, or access to essential services — as high-risk under Annex III. Article 9 of the Act requires that high-risk AI systems be subject to a risk management system that is a continuous, iterative process throughout the system's lifecycle. It mandates identification of risks arising from the system's intended purpose, its reasonably foreseeable misuse, and its interaction with other systems. It explicitly requires testing against predefined metrics and thresholds, and that testing be performed with data representative of the operational environment. An agentic pipeline that routes customer queries, synthesises documents, or assists underwriters is squarely within scope. A test approach that validates the model in isolation — without testing the tool-use layer, the multi-agent orchestration, or the memory state — does not satisfy Article 9's requirement for lifecycle-continuous risk management.

ISO 42001 Clause 8.4 adds a third layer: it requires that organisations establish processes to manage AI system performance over time, including monitoring of operational data and criteria for when re-evaluation is triggered. Together, these three frameworks demand something classical QE was never designed to provide: adaptive, continuous validation that is sensitive to behavioural drift, cross-system interaction risk, and the operational context of autonomous decision-making.

The Three Capability Shifts

Moving from classical QE to agentic QE is not primarily a tooling change. It is a capability change across three dimensions, each of which requires deliberate investment.

The first shift is from scripted to adaptive test generation. Classical test suites are authored by humans who anticipate what the system should do. Agentic systems explore a behavioural space that humans cannot fully anticipate in advance. Adaptive test generation uses the agent's own planning traces as a signal — identifying the branches the agent actually takes in staging and production, then generating test cases that probe the boundaries of those branches. It also uses adversarial generation: constructing inputs that are designed to push the agent toward constraint violations, policy breaches, or unexpected tool invocation patterns. This is not optional sophistication. It is the only way to achieve meaningful coverage of a system whose output space is not enumerable in advance.

The second shift is from siloed component testing to cross-agent observability. A multi-agent pipeline has emergent risk at the boundary between agents — at the point where one agent's output becomes another agent's input. Observability at that boundary requires structured tracing that captures not just the final output but every intermediate state: which tools were called, in what order, with what parameters, and what each tool returned. This trace must be the primary artefact of test evaluation, not the end-state output alone. Without cross-agent observability, a QE team is effectively testing a shadow of the system, not the system itself.

The third shift is from binary pass/fail verdicts to continuous risk scoring. In a deterministic system, pass/fail is a meaningful signal. In an agentic system, the question is not whether the agent produced the correct output — it is whether the agent's behaviour over time is drifting toward policy boundaries, whether its confidence calibration is degrading, whether the frequency of certain tool calls is shifting in ways that indicate model drift or prompt injection pressure. Risk scoring replaces the single-point assertion with a time-series signal that can be monitored, trended, and thresholded — producing an alert before a compliance event, not after one.

Step-by-Step Walkthrough of an Agentic QE Pipeline

A mature agentic QE pipeline operates across four phases: instrumentation, evaluation, governance integration, and continuous monitoring.

Instrumentation begins before the first test is written. Every agent in the pipeline must emit structured traces — including tool call metadata, planning chain outputs, memory read/write events, and confidence scores where available. These traces are the raw material of evaluation. Without them, the QE pipeline is operating on partial information. Instrumentation must also extend to the integration layer: the APIs, data stores, and downstream systems that agents interact with must emit enough signal to reconstruct the full execution path of any given agent run.

Evaluation takes three forms that must run in parallel. Functional evaluation assesses whether the agent achieved its goal within acceptable behavioural bounds — not just whether the final output is correct, but whether the path was consistent with policy constraints. Adversarial evaluation probes the agent under conditions designed to surface constraint violations: prompt injection attempts, goal misrepresentation, tool poisoning, and boundary-case inputs that are underrepresented in training data. Regression evaluation ensures that changes to the model, the orchestration logic, or the tool layer do not silently degrade previously validated behaviour.

Governance integration means connecting the QE pipeline directly to the risk and compliance framework. Test outcomes must be mapped to specific regulatory obligations — DORA Article 25 testing requirements, EU AI Act Article 9 risk management criteria, internal model risk policy thresholds — so that a QE report is not just a technical artefact but an audit-ready document. This requires the QE team to maintain a coverage map: which test scenarios correspond to which regulatory requirements, and what the acceptance threshold for each is.

Continuous monitoring closes the loop. Agentic systems in production must be monitored with the same instrumentation used in testing, and the monitoring must be tuned to the same risk indicators the evaluation phase identified. When behavioural drift exceeds a threshold — measured as a shift in tool call patterns, a drop in goal completion rate, or an increase in constraint violation frequency — the monitoring system must trigger re-evaluation, not just an alert. This is what Article 9's lifecycle-continuous requirement actually means in operational terms.

Maturity Benchmark: Where Most BFSI QE Teams Sit Today vs. Target State

Most BFSI QE teams approaching agentic systems for the first time sit at what can reasonably be called Maturity Level 1 or early Level 2. They have strong test automation capability for deterministic systems. They have CI/CD pipelines, API test frameworks, and in some cases model evaluation tooling for individual ML endpoints. What they typically lack is any of the cross-layer infrastructure that agentic validation requires: structured agent trace capture, inter-agent boundary testing, adversarial scenario libraries tuned to financial services contexts, or risk-scored evaluation outputs that map to regulatory frameworks.

Level 3 maturity — which should be the minimum acceptable bar for a production agentic deployment in a regulated institution — requires all four elements of the pipeline described above, plus a documented coverage map that links test scenarios to regulatory obligations. Level 4 adds closed-loop feedback from production monitoring into test generation: the system learns from production edge cases and automatically expands test coverage in response. Level 5 is continuous assurance — the evaluation pipeline runs without human initiation, risk scores are surfaced in real time to risk owners, and re-evaluation is triggered automatically when thresholds are crossed.

The gap between current state and minimum viable compliance is not a tooling gap. It is a process and capability gap. Most BFSI QE teams need to add three capabilities they have never needed before: adversarial scenario design for autonomous systems, cross-agent trace analysis, and compliance-coupled risk scoring. Those capabilities take time to build and require collaboration between QE, AI engineering, and risk functions that is not yet standard practice in most institutions.

How to Start: A 90-Day Transition Roadmap

A 90-day transition is achievable, but only if it is scoped honestly. The goal of 90 days is not to reach Level 5 maturity. It is to move from zero structured agentic test capability to a defensible Level 3 state — one where a go-live decision can be supported by documented, audit-ready evidence that the agentic pipeline has been validated against policy constraints, adversarial scenarios, and regulatory criteria.

Days 1 through 30 are the instrumentation and assessment phase. The first priority is to instrument the existing agentic pipeline — or the staging version of it — so that every agent run produces a structured trace. The second priority is a gap assessment: map current test coverage against the three evaluation types described above, and identify which regulatory obligations are currently unsupported by evidence. This phase should produce a coverage map and a risk-ranked backlog of test scenarios to be built.

Days 31 through 60 are the build phase. The QE team builds the adversarial scenario library — starting with the highest-risk scenarios identified in the gap assessment — and establishes the cross-agent observability layer. This phase also includes a pilot run of the governance integration: producing a test report for one agent workflow that is structured to map directly to DORA Article 25 and EU AI Act Article 9 requirements, and having it reviewed by the compliance or risk function for adequacy.

Days 61 through 90 are the integration and handoff phase. Continuous monitoring is configured for production. Thresholds are set and agreed with risk owners. The coverage map is finalised and included in the go-live documentation package. The QE team briefs the CISO and CTO on what the ongoing monitoring posture will be and what events will trigger re-evaluation.

This roadmap does not assume a large team or a greenfield system. It assumes a QE lead who has organisational access to the AI engineering team, a clear line to the risk or compliance function, and the authority to instrument the pipeline. Those are the real prerequisites.

The regulatory and technical case for agentic QE transformation is not speculative. DORA is in effect. The EU AI Act's high-risk provisions are in the enforcement calendar. The behavioural failure modes of agentic systems are well-documented. What remains is the operational question of whether QE functions in regulated institutions will build the capability before a compliance event forces the issue — or after.