Learning point 1
What does the EU AI Act require when AI is used in lending decisions?
The EU AI Act (Regulation 2024/1689) is the world's most prescriptive AI lending regime. Annex III explicitly classifies AI systems used to evaluate the creditworthiness of natural persons or establish their credit score as high-risk AI systems - subject to the Act's full obligations from 2 August 2026. Fraud detection is the only credit-adjacent use case explicitly exempted from high-risk classification.
For high-risk credit AI, the Act requires: a documented risk management system (Article 9), data governance ensuring training and validation data is relevant and bias-checked (Article 10), technical documentation (Article 11), automatic logging (Article 12), transparency to deployers (Article 13), human oversight (Article 14), accuracy/robustness/cybersecurity (Article 15), a quality management system (Article 17), and registration in the EU database (Article 49). Penalties can reach up to EUR 35 million or 7% of global annual turnover, whichever is higher.
The AI Act conformity assessment may be combined with the supervisory review and evaluation procedure (SREP) under banking law (Recital 158), so EU credit institutions can integrate AI Act compliance into existing prudential frameworks rather than running parallel processes. The broader stack includes CRR III / CRD VI, DORA, CCD II, MCD, PSD2/PSD3, and EBA Guidelines on loan origination and monitoring.
Learning point 2
How does GDPR and the AI Act's data governance interact with AI agents in banking?
GDPR remains the privacy foundation for EU lending AI: lawful basis for processing, data minimization, purpose limitation, and data subject rights must be designed into the agent workflow. GDPR Article 22 is especially important because credit decisions qualify as legal or similarly significant effects; affected individuals must have the ability to obtain human intervention, express their view, and contest the decision.
AI Act Article 10 adds a separate data governance layer for high-risk AI. Training, validation, and testing data must be relevant, representative, free of errors to the extent possible, and examined for possible biases. This goes beyond GDPR in some respects because bias examination is mandatory and must be documented.
EDPB guidance on AI and data protection, plus national DPA enforcement, shapes how these obligations are applied. Enforcement is national and varies; CNIL in France has been particularly active on automated decision-making. Cross-border transfers to non-adequate countries require SCCs plus a Transfer Impact Assessment.
Learning point 3
How do I manage model risk for AI agents the way the AI Act and EBA expect?
EU deployments combine AI Act Articles 9 and 17 - risk management system and quality management system requirements - with the EBA Guidelines on loan origination and monitoring (2020/06), which govern model governance for credit decisioning. IRB banks also need to consider the ECB SSM Guide to internal models.
The deployer/provider split matters. A vendor providing the AI system performs provider-side ex-ante conformity assessment and technical documentation, but the bank as deployer owns day-to-day monitoring, logging, appropriate use, human oversight, and Article 26 obligations in production.
Model documentation for EU AI agents should map every credit decision workflow to the AI Act obligations: risk controls, dataset governance, automatic logs, human oversight points, performance monitoring, cybersecurity controls, and evidence that the system is used according to the provider's instructions.
| MRM role | Responsibility | Typically held by |
|---|---|---|
| Model Owner | Business accountability for agent outputs | Business head of the function the agent serves |
| Model Risk Manager | Independent validation, risk assessment | Risk or Compliance function |
| Technology Owner | Change management, version control | IT or Data team |
| AI Act Compliance Owner | Article 26 deployer obligations and CE-marking traceability | Legal or Compliance function |
Learning point 4
How does the AI Act require banks to test AI agents for bias and discrimination?
Yes, AI agents can exhibit bias - both the LLM they are built on and the data they are trained or evaluated on can encode historical patterns of discrimination. In EU lending, this risk sits under EU non-discrimination law and AI Act Article 10. The EU Charter of Fundamental Rights Article 21 provides the constitutional baseline.
The Race Equality Directive (2000/43) and Gender Equality Directive (2004/113) prohibit discrimination in access to goods and services, including credit. Protected characteristics under EU law are broader than US categories in important respects, including explicit protection of sexual orientation and religious belief.
The most common forms of bias in lending AI are demographic bias, proxy bias (using a variable like postal code or employer industry that is correlated with a protected characteristic), and feedback loop bias. AI Act Article 10 mandates examination of datasets used for high-risk AI training, validation, and testing for possible biases.
Learning point 5
What security risks come with deploying AI agents in a bank, and how are they mitigated?
AI agents in banking introduce three categories of security risk that do not exist with traditional software: prompt injection attacks (malicious inputs designed to override the agent's instructions), data leakage through the LLM layer (the model inadvertently revealing information from one customer's data in another's session), and supply chain risk from the LLM provider (the model vendor's systems being breached or the model being updated in ways that change agent behavior).
Prompt injection is the most immediately exploitable risk. An attacker could submit a loan application containing hidden instructions - in white text, in metadata, or embedded in a PDF - that try to override the agent's behavior. Well-designed agents have injection-resistant prompt structures that separate data inputs from instruction inputs architecturally, and validate that inputs are in expected formats before processing.
Data leakage is mitigated through session isolation - each agent interaction should operate in a clean context with no memory of previous sessions. DORA, in force since 17 January 2025, is the dominant cybersecurity and third-party risk framework for EU financial institutions. It requires a register of ICT third-party providers, critical provider classification, exit strategies, incident classification and reporting, and threat-led penetration testing for significant institutions.
AI vendors providing high-risk AI to EU banks may also be critical ICT third-party service providers under DORA, subject to direct oversight by the Lead Overseer, one of the European Supervisory Authorities. Major ICT incidents must be reported quickly after classification, with four-hour timing relevant for major incident reporting workflows.
