The Audit Trail Fails First: Air-Gapped AI as a Structural Requirement Under RBI and EU AI Act

1. The Regulatory Forcing Functions

Air-gapped LLM deployment compliance for regulated financial institutions has moved from a theoretical architecture preference to a hard regulatory constraint — but most institutions have not yet mapped their LLM deployment topology against all three simultaneous obligations that now apply.

The Reserve Bank of India's data localisation requirements, most directly articulated through the Payment Systems guidelines and reinforced through successive RBI Master Directions on IT governance, require that data pertaining to Indian payment system participants be stored and processed on systems physically located within India. When a bank routes customer query prompts through a commercial LLM API, that prompt — which may contain account context, transaction signals, or identity fragments — traverses infrastructure the bank does not own, does not control geographically, and cannot audit independently. The residency obligation is not satisfied by a vendor's contractual assertion; it requires technical proof of where inference occurred.

The EU AI Act, applicable to high-risk AI systems under Annex III (which explicitly includes AI used in credit scoring, insurance underwriting, and financial supervisory decisions), imposes Article 9 risk management obligations, Article 11 technical documentation requirements, Article 13 transparency obligations, and — critically for this discussion — Article 17 quality management system requirements that mandate complete logging of system operation throughout the system lifecycle. Article 15 adds the requirement for accuracy, robustness, and cybersecurity controls that must be continuously verified. None of these obligations can be discharged if the inference environment is outside the deployer's direct control.

DORA, the EU's Digital Operational Resilience Act, adds a third vector. Under DORA Article 28 and the associated Regulatory Technical Standards on ICT third-party risk, financial entities must ensure that critical or important function providers are subject to full audit rights, concentration risk assessment, and exit strategy requirements. A commercial LLM API serving a credit decisioning or fraud detection workflow almost certainly qualifies as a critical ICT service under DORA's functional definition. Most commercial API agreements do not provide the audit rights DORA requires, and no hyperscale LLM provider offers contractual commitments on model version stability at the inference layer that would satisfy DORA's change management provisions.

2. Architecture of Air-Gapped Inference With Audit Trail Mechanics

A compliant air-gapped LLM deployment is not simply a self-hosted model. It is a system of four independently verifiable components that together produce a legally defensible audit record.

First, sovereign compute: model weights are loaded on infrastructure — on-premise servers, a sovereign cloud region with contractually verified single-tenant isolation, or an air-gapped private cloud — where the institution holds root access, controls the hardware security module, and can produce a chain of custody for every model version loaded into production. This is the foundation. Without it, the remaining components cannot be trusted.

Second, a network-isolated runtime: the inference process runs in a segment with no egress to external APIs, no telemetry callbacks to the model vendor, and no shared-tenant routing. Network isolation must be verifiable through firewall policy exports and packet-level logging — not through a vendor's dashboard. A diagrammatic description of this architecture shows three zones: an outer DMZ handling encrypted user-facing traffic, a middle application orchestration layer running retrieval-augmented generation pipelines and prompt routing, and an inner inference enclave with no outbound routes, where model weights reside alongside the cryptographic signing service. The enclave communicates with the orchestration layer only through a defined, logged API gateway. No zone has direct internet access.

Third, an immutable inference log: every prompt, every response, every model version identifier, and every timestamp must be written to an append-only log store — preferably one backed by a write-once storage medium or a cryptographic hash chain. This log is the artifact that regulators will request. It must be queryable on demand, exportable in a structured format, and retained for the periods specified under applicable regulations. For EU AI Act high-risk systems, the minimum retention expectation implicit in Article 17 is the system's operational lifetime plus post-deployment monitoring period.

Fourth, a cryptographically signed audit chain: each inference event should produce a signed receipt — hash of input, hash of output, model version hash, timestamp, and operator identity — stored separately from the inference log itself. In a supervisory review, the institution can produce the inference record and the independent receipt, and an auditor can verify that the log has not been altered. Without this component, the audit trail satisfies operational monitoring requirements but not forensic regulatory requirements.

The compliance gap table that belongs here maps each regulatory obligation to the architectural component it depends on. EU AI Act Article 17 maps to the immutable inference log and signed audit chain. RBI data localisation maps to sovereign compute and network isolation. DORA ICT third-party risk maps to sovereign compute, network isolation, and the audit chain. ISO 42001 Clause 8.4 maps to all four. NIST AI RMF GOVERN 1.7 maps to the audit chain and immutable log. Any architecture missing a component creates an unmapped obligation — a compliance gap that cannot be patched contractually.

3. Failure Modes of Cloud-Reliant AI in Regulated Environments

The three failure modes that most frequently surface in regulated financial services AI deployments are not theoretical.

The first is unlogged prompt telemetry. Commercial LLM APIs typically collect prompt data for safety monitoring, usage analytics, and — in some configurations — model improvement. Even where a vendor offers a data processing agreement, the institution cannot independently verify what telemetry leaves the inference call. This creates a gap between the institution's data processing record and the actual processing event, which is precisely what Article 17 of the EU AI Act requires institutions to close.

The second is cross-tenant model behaviour inconsistency. Hyperscale LLM providers update foundation model weights, inference optimisations, and safety layers on schedules that are not synchronised with the institution's change management or model validation cycle. A model that passed internal validation in one version may behave differently after an unannounced update. Under SR 11-7 principles (which Indian and European regulators have substantively adopted), model risk management requires that production models be the models that were validated — not a silently updated successor.

The third failure mode is the inability to produce a complete inference record on supervisory demand. When a regulator or internal audit function requests the full record of a specific credit decision made by an AI system, the institution must be able to produce the exact prompt, the exact model response, the model version, and the timestamp — and demonstrate that the record has not been altered. Cloud-hosted inference logs are controlled by the vendor. They are not always complete, they are not always retained on the institution's preferred schedule, and they are inaccessible if the vendor relationship ends. The audit trail does not just become difficult to produce. It may not exist.

4. Assurance Checklist Mapped to ISO 42001 and NIST AI RMF

The assurance checklist for air-gapped LLM deployment compliance maps obligations to verifiable evidence, not policy documents.

ISO 42001 Clause 8.4 (AI system operation) requires documented evidence that the AI system operates within defined parameters. The verifiable evidence is a combination of inference enclave configuration records, firewall policy exports showing network isolation, and the cryptographic audit chain showing model version consistency. Clause 9.1 (monitoring, measurement, analysis) requires ongoing evaluation — satisfied by automated log integrity checks and scheduled model performance evaluations run inside the air-gapped environment.

NIST AI RMF GOVERN 1.7 requires that roles, responsibilities, and accountability structures for AI risk be defined and exercised. In an air-gapped deployment, the institution must document who holds root access to sovereign compute, who controls the signing keys for the audit chain, and who is authorised to load a new model version into the inference enclave. These are not just organisational chart questions — they are technical controls questions that the assurance function must verify.

The NIST AI RMF MAP and MEASURE functions, when applied to air-gapped LLM deployments, require that the institution run its own adversarial evaluation, drift detection, and performance benchmarking inside the enclave. There is no vendor-side evaluation data to rely on. The institution owns the evaluation function entirely — which means it must resource it, schedule it, and produce evidence of it for audit.

Air-gapped LLM deployment compliance for regulated financial institutions is ultimately an assurance discipline, not an infrastructure project. The infrastructure is a precondition. The continuous evaluation of model behaviour, the integrity of the audit chain, and the mapping of every architectural control to a specific regulatory obligation — that is what converts a deployment into a defensible compliance position. Institutions that treat sovereign AI as a one-time infrastructure decision and not as a continuous assurance obligation will find that the audit trail, not the model, is what fails them first.