Hardly any German AI rule asks what the customer ends up with

On Monday, 19 October 2026, the UK Financial Conduct Authority (FCA) published a collection of good and poor practice in the use of AI by financial firms. Under the title “AI in financial services: good practice and areas for improvement”, it sets out 20 case studies labelled A to T: seven on the Consumer Duty, the UK requirement to deliver good outcomes for retail customers, nine on governance and accountability, and four on operational resilience. In its own words, the examples “do not create new requirements”. On the same day Jessica Rusu, the FCA’s chief data, information and intelligence officer, outlined in a drafted speech how the regulator is responding to the Mills Review on AI in retail financial services: through testing, assurance and surveillance carried out directly with firms rather than through another consultation. Six of the review’s seven recommendations, she said, are going straight into live work.

A comfortable reading is tempting in Germany. London works with examples, the EU with legislation, so whatever the FCA describes must long since have been settled here in binding form. For a large share of the cases that holds, and in consumer credit German customers even have enforceable rights where the FCA only describes practice. Sort the 20 cases by what they actually test, however, and the picture tilts. Input data, review processes and the inventory of ICT systems are governed by binding rules, some of them for years. The legal texts reviewed here contain hardly any rule that asks about the outcome for the customer, the yardstick the FCA applies to almost every case. What follows is derived from the legal texts and from the author’s own mapping of the cases. It is not a statement by any supervisor. Three of the failure patterns already featured in a LinkedIn carousel on 20 October. This piece covers all 20 and asks where each one meets a German rule.

The FCA’s 20 cases apply one yardstick: the outcome for the customer

According to the FCA, the cases draw on its engagement with firms, trade bodies, consumer organisations and technology providers, on the work behind the Mills Review and on its testing programmes, AI Live Testing and the Supercharged Sandbox. They are short, anonymous and free of figures. Four of the 20 describe a failure, the rest a practice the regulator considers sound. As the legal framework, the FCA cites only existing rules, from the Principles for Businesses through the Senior Managers and Certification Regime (SM&CR) to complaints handling. The collection could be dismissed as a set of British anecdotes for that reason. Yet it shows the yardstick a supervisor will bring to an AI application in future, and that yardstick is plainly different from a checklist.

In almost every case the FCA asks what reaches the customer. One firm tested, before deployment, whether indicators of vulnerability such as accessibility needs influenced fraud suspicion or claim acceptance (case B). Another had a small, varied group of customers read AI-drafted replies to complaints before introducing the tool, and measured the result against indicators such as complaint resolution (case E). A guardrail against model drift, by contrast, produced answers so generic that customers were steered towards unsuitable options (case C). The collection sums up the yardstick as a failure pattern: firms that monitored technical metrics rather than outcomes stood less chance of meaningful oversight. The model behind it is the Consumer Duty, which Jessica Rusu describes as “an outcomes-based regulatory framework”.

Sorted by that yardstick, the 20 cases fall into four layers. One case tests input data, eight test processes and human review, five test the inventory of systems and the exit from a provider, and six test the outcome for the customer. For each layer one can ask which German rule governs the same matter in binding form, and within which limits. The chart shows the result of that exercise. Three layers carry a binding rule, each with a scope considerably narrower than the British principles. For the fourth, the sources searched show no counterpart. The assignment of cases is the author’s own, and in places a case could just as well sit in the neighbouring layer. The input data layer rests on a single case, a test for bias.

A tick-box review in lending counts as an automated decision under Article 22 GDPR today

Case O is the hardest in the collection. In one firm the role of the human reviewer turned into a tick-box exercise over time: reviewers accepted AI recommendations without sufficient scrutiny, and unclear escalation criteria let outputs through even when confidence was low. The FCA does not name a use case. If the case plays out in a credit decision, Article 22 GDPR applies first in Germany, and it applies today: the right not to be subject to a decision based solely on automated processing. The EU AI Act follows in December 2027. In its non-binding 2017 guidelines on automated decision-making (Working Paper 251, WP251), the Article 29 Working Party stated that a controller cannot avoid Article 22 “by fabricating human involvement”. Anyone who routinely applies automatically generated profiles without any actual influence on the result is still taking a decision based solely on automated processing. Oversight must be “meaningful, rather than just a token gesture”, carried out by someone with the authority and competence to change the decision.

This sharpens the mapping in the 20 October carousel, which had assigned the case to the EU AI Act from December 2027. From 20 November 2026 a second rule follows, one that sits closer to the case. The German act implementing the Consumer Credit Directive (EU) 2023/2225, dated 12 May 2026, inserts Section 30(6) into the BDSG. Where the creditworthiness assessment for an ordinary (non-mortgage) consumer loan, a general consumer loan in the statute’s terms, involves “eine automatisierte Verarbeitung personenbezogener Daten”, the borrower may demand “das Eingreifen einer Person”, that is, intervention by a human. According to the statutory text this includes a clear explanation of the logic of the assessment, the right to state one’s own view and a review of the decision. The trigger is any automated processing, not only a decision based solely on it. The provision does not cover consumer mortgage loans. Banks have 24 days from the publication of this article until it takes effect.

Alongside it sits the new Section 37a BDSG on scoring, which entered the bill only at the committee stage in the Bundestag. Its list of excluded data applies only under one condition. According to the committee’s explanatory statement (Bundestag document 21/5381), it turns the former Section 31 BDSG “in eine Ausnahmeregelung vom Verbot des Artikels 22 Absatz 1”, an exception to the prohibition in Article 22(1) GDPR. A lender that bases a fully automated credit decision on this exception may not use, from 20 November, information on incoming and outgoing payments on bank accounts, nor age, sex or address data. Obligations to take income into account in the risk assessment remain unaffected, the same statement adds. Case G illustrates where the line can run. There an AI found spending patterns suggesting vulnerability, and an ethics committee decided whether the firm should keep lending. A decision taken by a committee is not a fully automated one, and Section 37a(2) does not apply.

The EU AI Act sends banks back to their own governance for monitoring

From 2 December 2027 the high-risk obligations of the EU AI Act apply to AI systems that evaluate the creditworthiness of natural persons or establish their credit score, with an exception for fraud detection. The exemption in Article 6(3) for systems that merely prepare a decision is not available here, because the last subparagraph states that such a system “shall always be considered to be high-risk” where it performs profiling of natural persons. Under Article 26(2) the deployer must assign human oversight to people with “the necessary competence, training and authority”, and before first use it must carry out a fundamental rights impact assessment under Article 27 and notify the result to the market surveillance authority, which in Germany is BaFin for financial institutions. Article 14(4)(b) requires that those overseeing the system can remain aware of the tendency to over-rely on its output, which the regulation calls “automation bias”. Case O describes exactly that tendency, case N a method of countering it.

The regulation does, however, limit its own reach for banks in two places that are rarely read together. The first is a referral. For financial institutions, the deployer’s monitoring obligation under Article 26(5) is deemed fulfilled if the institution complies with the financial-services rules on internal governance. Similar relief applies to quality management and documentation. The assignment of human oversight in Article 26(2) is not part of that relief. The second is grandfathering. Under Article 111(2), as amended by the Digital Omnibus, the regulation applies to high-risk systems in service before the deadline only if, from that date, they are “subject to significant changes in their designs”. A credit score running before December 2027 stays outside until it is substantially rebuilt. Case T, in which the underlying model was retired and the firm had to redesign its product, describes precisely such a moment.

A further boundary is still open. According to the Commission’s guidelines on the definition of an AI system, systems that merely improve established methods such as linear or logistic regression fall outside the definition. The guidelines are not binding, and the line is disputed. A traditional scorecard may therefore fall outside both regimes. It may not be a high-risk system under the EU AI Act, and if it falls within the scope of the Capital Requirements Regulation (CRR), the MaRisk model module expressly excludes it. Significant institutions under direct supervision by the European Central Bank have in any case fallen outside MaRisk since the 30 June 2026 version. The German framework for consumer credit is therefore dense but conditional on four things: the type of institution, the type of model, the date it went live and the degree of automation.

For letters and chatbots, German law governs the form of the answer

The picture changes where a language model generates text. Case E describes a tool that drafts replies to complaints, case F a customer service chatbot that flags signs of vulnerability to staff. The explanatory notes to MaRisk AT 4.3.4 define a model as a method that turns input data into “quantitative Schätzungen”, quantitative estimates, and the module covers models used for the processes the circular governs. BaFin has not decided whether a letter generator falls within that definition, and the wording at both points tends to speak against it. That does not leave the tool unregulated. BaFin’s circular 06/2018 on complaints management requires in paragraph 21 that communication with the complainant be conducted in clear and easily understandable language. MaRisk AT 8.2 requires an institution, before significant changes to its ICT systems, to analyse their effect on control procedures and to involve risk control, compliance and internal audit. That is the German counterpart to case J, in which a firm introduced AI before anyone was responsible for overseeing it.

Yet both rules govern how an answer must be framed and how a roll-out must proceed. Neither asks whether customers understood the answer. The firm in case E tested exactly that, with a varied sample of customers and with complaint-handling metrics. The EU AI Act adds little here. The transparency duties in Article 50 fall on the provider of chatbots and AI-generated content, and for a bank deploying a tool that drafts letters what usually remains is the duty under Article 4 to take measures supporting the AI literacy of its staff. The Digital Omnibus weakened that duty, which no longer requires a specific level of literacy to be guaranteed for any individual. The gap is widest for vulnerable customers. The FCA criticises firms that exclude customers with vulnerability indicators from AI journeys by default without offering them another route. No counterpart to the FCA’s guidance FG21/1 on the fair treatment of vulnerable customers turned up in the BaFin sources searched. BaFin’s mandate under Section 4(1a) of the Financial Services Supervision Act (FinDAG) concerns collective consumer interests and requires a serious, lasting or repeated breach of consumer law, not a poor outcome in an individual case.

The DORA inventory has no fields for an agent’s access rights or controls

For agentic applications, the FCA describes as good practice a central inventory in which firms register their agents, recording which are live, who owns them, which systems and data they can access and which controls apply. In IT Finanzmagazin in May this instrument appeared as the “Agent Registry”. No German rule requires a dedicated AI or agent register. DORA, however, requires in Article 8 an inventory of all ICT assets, and the regulatory technical standards in Delegated Regulation (EU) 2024/1774 set out the mandatory fields in Article 4: a unique identifier, the owner of the ICT asset, the business functions supported, the interdependencies between assets and the end dates of the provider’s support. In its guidance on ICT risks in the use of AI, issued on 18 December 2025, BaFin expressly counts models among them: the model is understood “als IKT-Asset (Software)”, as a software ICT asset. The guidance describes itself as non-binding assistance, while the inventory itself is a DORA requirement. Compared with the British example, the mandatory fields lack entries for an agent’s access rights and for the controls that apply to it.

The exit from a provider follows the same pattern. DORA requires exit strategies and transition plans under Article 28(8), but only for ICT services supporting critical or important functions. Nothing in the wording obliges a provider to announce model changes or the retirement of a model in advance, and the notification duties in Article 30(3) concern the provider’s ability to deliver the service. Delegated Regulation (EU) 2025/532 requires advance notice only for changes to subcontracting. The FCA cites as good practice firms that secured contractual notice of significant model changes. BaFin describes the same problem in its guidance as an obstacle to testing: unannounced model changes when using an AI model obtained from a third party. The support end date in the DORA inventory is where case T could announce itself in good time, provided someone records the retirement date of the model and not just that of the platform.

On incident reporting the FCA goes beyond the European framework, though so far only as an aim. According to the drafted text of Jessica Rusu’s speech, the duty to report major incidents “must extend to significant AI failures and near-misses”. DORA has no concept of a near miss, and the term appears neither in the regulation nor in its implementing acts on incident reporting. Under Article 3(8), an ICT-related incident requires the security of the systems to be compromised and an adverse effect on the availability, authenticity, integrity or confidentiality of data or on the services provided. A chatbot giving wrong answers without any system being compromised will often not qualify as an ICT-related incident at all. Internally, Article 17(2) does at least require all ICT-related incidents to be recorded. Where an AI failure does touch security, that is the obvious place to label it, as the BaFin guidance recommends.

BaFin described a staged model in 2021 that resembles the FCA’s traffic-light thresholds

Two of the FCA’s cases will look familiar to German readers, because BaFin described a similar pattern years ago. In case M the responsible senior manager had to approve an incident playbook before sign-off, with red, amber and green thresholds and predefined steps up to a temporary shutdown. In case P a firm translated its risk appetite into such thresholds, agreed between the business and model risk. On the author’s reading, they match the structure of a pattern BaFin described on 15 June 2021 in its paper “Big Data und künstliche Intelligenz: Prinzipien für den Einsatz von Algorithmen in Entscheidungsprozessen”: a staged model in which a human signs off from the first stage onwards and, at the last stage, sign-off is initially withheld under a “Stopping-Rule” that instead triggers a review of the model. On management responsibility the paper stated that the board remains responsible for material business decisions even when they rest on algorithms.

The paper never became binding. BaFin itself called it preliminary considerations and a basis for discussion that was nonetheless meant to serve as guidance. A different part of the thinking entered MaRisk in 2024, namely the extension of the model module to artificial intelligence together with a requirement of adequate explainability, then in AT 4.3.5 and since the 30 June 2026 version in AT 4.3.4. The staged model with its stopping rule remained guidance. The FCA takes the opposite route: it writes no rule and instead points to firms that work this way. The two systems also differ on accountability. Under the SM&CR, AI risk can be allocated to a named Senior Management Function (SMF), and case L describes a firm that made the SMF of the affected business line accountable for each use case, with a separate model owner for the technical build. The German Banking Act (KWG) assigns managing directors responsibility for proper business organisation under Section 25a(1), and MaRisk AT 3.1 sets out a collective responsibility that applies “unabhängig von der internen Zuständigkeitsregelung”, regardless of the internal allocation of duties. A duty to allocate AI risk to named individuals did not turn up in the sources reviewed. Pedro Machado, ECB representative to the Supervisory Board, described the same problem in February: “fragmented ownership, with responsibility split across IT, data science teams, business lines and control functions, without a clear accountability framework”.

An override rate close to zero is a finding

Between the British outcome yardstick and German process law there is one metric that serves both. MaRisk AT 4.3.4 requires in paragraph 4 rules on how model results are used, including overrides where relevant, and in paragraph 5 a validation that reviews whether model results are handled appropriately. Anyone who has run a trading book knows the limit that never binds. Either it was set too generously, or nobody checks whether it would bite. An override rate close to zero sends the same signal. It may mean the model is good, or that reviewers simply follow it, as case O describes. Only a comparison of the kind set up in case N tells the two apart. While the system was being introduced, staff there carried out affordability assessments without seeing the system’s proposal, so that independent human judgement could be compared with the AI’s.

The German framework is admittedly stricter than the British examples in three places: Article 22 GDPR for a tick-box review in lending, the complaints circular for AI-drafted replies, and MaRisk AT 8.2 for a roll-out without governance. All three, however, govern input data or process, and none asks about the outcome for the customer. The FCA intends to keep its collection updated, and in the drafted text of her speech Jessica Rusu describes it as “not intended to be static guidance”. For German institutions, whether an AI application does right by the customer does not become a legal duty as a result. It remains the question that a review of overrides comes closest to answering, and one that can be asked before any supervisor does.

Recommendations for model risk, compliance and ICT risk management

Glossary

Christian Schablitzki – the agentic banker

Strategy & Management Consultant · Agentic AI expert for financial institutions

More than 20 years in investment banking and derivatives trading, followed by over 10 years advising financial institutions. Currently a Partner at Infosys Consulting in Germany. Certified as Claude Partner – Claude Code (Anthropic), as well as in Google AI, Generative AI Leader (Google Cloud) and IBM RAG and Agentic AI.

LinkedIn profile → · agenticbanker.ai →