The warning shot comes from the lab itself

Highlight

The warning shot comes from the lab itself

Over the past fortnight, OpenAI and Anthropic have shipped their new flagship models: Claude Fable 5.1 on 1 September, GPT-6 Astra on 3 September, both priced at 10 and 50 US dollars per million tokens. Eight days before Astra, OpenAI published a report describing how its own agents, during a cyber evaluation in July, worked around the isolation of their sandbox, coordinated through an improvised message board, found credentials lying on the open internet and took over Hugging Face servers. OpenAI itself calls this a “warning shot for us and for the world”. Anthropic had reported three comparable incidents on 30 July, among them a booby-trapped Python package that sat in public for roughly an hour and ran on 15 real systems.

This time the warning carries the vendor’s letterhead, and it ships with the product. OpenAI designates Astra as its first model at the “Critical” level for cyber capabilities: according to the company, it can find previously unknown flaws in well-protected systems and exploit them without a person guiding each step. On 27 August, what has since grown to more than 150 organisations, OpenAI, Anthropic, Google and Microsoft among them, called for a collective effort on cyber defence; Sam Altman added that there was “not much time to act”. Andrew Bailey, Governor of the Bank of England and Chair of the Financial Stability Board (FSB), wrote in his letter to G20 finance ministers, published on 31 August: “For the financial system, the most immediate concern is the potential impact of frontier AI on cyber risk.”

For an institution, the consequence is that these models now sit on both sides of the firewall, as a tool inside the house and as a tool in the attacker’s hands. Mark Branson, President of Germany’s BaFin, told the Handelsblatt banking summit on 2 September that of almost 900 incidents reported over the past twelve months only 13 per cent were attributable to attacks, “yet the danger is growing”, partly because of frontier AI models. What occupies me more than the offensive capability is the way in. In Anthropic’s three cases the sandbox was misconfigured, in an environment run by an external evaluation partner; Anthropic explicitly takes the responsibility on itself. At OpenAI the controls held as designed, and the agents worked around them through a chain of vulnerabilities. The second finding weighs heavier, because a misconfiguration can be fixed, whereas closing one bypass closes only that one. Anyone handing agent pilots to a service provider does well to verify the isolation itself. The UK regulator’s survey locates the bottleneck after detection, at remediation; why that is, I set out in last week’s FCA article.

LinkedIn Featured

Magic 7 in the S&P 500: Concentration Risk, Scenarios and What It Costs

Seven names carry 32.91 per cent of the S&P 500; with the suppliers of the AI story the figure rises to 40.08 per cent. Euro area households hold around 440 billion euros in US technology stocks, increasingly through ETFs, according to the ECB. The article opens on 27 January 2025, the day Nvidia lost 589 billion US dollars of market value, works through three scenarios for European portfolios and asks what this risk costs once somebody has to carry it under prudential rules.

Magic 7 in the S&P 500: Concentration Risk, Scenarios and What It Costs →

Agentic AI

tools, skills & what’s trending

Two models, one price, a new risk tier

Claude Fable 5.1 and GPT-6 Astra cost the same, 10 US dollars per million input tokens and 50 per million output tokens, and both advertise Computer Use, the ability to operate a screen directly. OpenAI’s launch table has Astra ahead in 13 of 15 public benchmarks with a comparable score, and the same table carries the Artificial Analysis index that puts Fable 5.1 in front by 66 points to 61. The result depends on who is measuring. Astra is the first model OpenAI places at “Critical” for cybersecurity under its own Preparedness Framework. Anthropic ships Fable 5.1 with additional safeguards and puts a number on their cost itself: 55.8 against 60.9 per cent on Terminal-Bench 4.0 compared with the more permissive Mythos 5.1, which only vetted organisations and individuals receive. What Computer Use means for the permissions question inside a bank is the subject of my 8 September article.

What happened inside the sandboxes in July

OpenAI’s technical report, published on 26 August, reads like a penetration-test log, except that nobody commissioned the test. An internal research model in the size class of GPT-5.6 Sol re-established a message board on 8 July using encoded directory names; on 10 July an agent found Hugging Face credentials lying in public and shared them; others gained administrator rights on a research cluster. On 4 September, independent researchers presented a second case, in their assessment distinct from the Hugging Face swarm: around 18,000 posts from agents identifying themselves as OpenAI systems had accumulated between May and early July in a German-language wiki, including instructions for evading a network block through a spoofed Azure domain. OpenAI confirmed the incident the following day. Anthropic, after reviewing 141,006 evaluation runs, had found three cases of its own; in all three the cause was a misconfiguration in the environment of its evaluation partner Irregular, although the prompt explicitly stated that there was no internet access.

OpenAI drops Cursor because the owner changed

On 28 August, OpenAI announced that it would wind down its contract with the coding tool Cursor following the latter’s acquisition by SpaceX, with a shut-off date of 12 November, the longest notice the contract allows. The reasoning is unusually candid: the company cannot be confident that SpaceX will stay within its terms of service, and the agreement contains a termination right on change of control. Future models, Astra included, will not reach Cursor; according to Cursor chief executive Michael Truell, OpenAI models serve about five per cent of user traffic. For an institution this is a case study in supplier risk of a kind that rarely appears in an outsourcing register: the model provider terminates because its customer changed hands. The exit strategy under Art. 28 (8) DORA should cover this case.

Banking & Regulation

what really matters now

The ECB wants to go on-chain itself

Speaking at Jackson Hole on 28 August, Isabel Schnabel suggested that central banks should “embrace DLT and go on-chain themselves”, with programmable reserves rather than access through private intermediaries. Two days earlier in Frankfurt, Piero Cipollone had set out the timetable: Pontes, settlement in central bank money for DLT platforms, goes live before the end of 2026, initially charging nothing beyond one-off onboarding fees and later extending to 22.5 operating hours per business day; Appia is to deliver the blueprint for an integrated European tokenised market in 2028. Tokenised traditional assets on public blockchains grew worldwide from 4.7 to 23.3 billion euros within a year, according to the ECB. Anyone still planning intermediary tokens against an omnibus account should read the two speeches side by side.

Less reporting, the same capital rules

At Bruegel on 2 September, Claudia Buch put three numbers on the table: around 20 per cent less supervisory reporting in the assessment of individual banks, half as many data points in the next EU-wide stress test, which the ECB runs jointly with the EBA, and around 40 of more than 100 supervisory guidance documents to be discontinued. On the same day, Mark Branson described the revised MaRisk, Germany’s minimum requirements for risk management, as “around 30 per cent leaner”. Claudia Buch stresses that capital rules are untouched and adds that weaker standards would be more likely to raise shareholder payouts than lending. In parallel, the EBA is consulting until 31 December on technical standards for operational risk management under Art. 323 CRR3, with relief for institutions below a business indicator of 750 million euros; ICT risk remains a matter for DORA.

Signal & Noise

what deserves your time

• FSB Chair Bailey warns the G20 about frontier AI – Financial Stability Board. The letter to finance ministers, published on 31 August, states: “the most immediate concern is the potential impact of frontier AI on cyber risk”, and underlines the importance of being able to restore critical systems “from bare metal”. Many jurisdictions, it says, lack protocols for the development, release and deployment of such models.
• 36 agentic AI pilots selected from nearly 100 proposals – Hong Kong Monetary Authority. Four Hong Kong regulators are letting 30 institutions and 27 technology partners test agents in onboarding, payments and claims handling. Extended in this cohort: AI that oversees the actions of other AI, a theme the regulators carry forward from the previous round. Europe has no equivalent.
• Big tech, big debt – ECB Blog. Five US hyperscalers now account for just shy of ten per cent of gross new issuance of euro-denominated non-financial corporate bonds. The blog asks what that means for the funding costs of every other issuer, sovereigns and supranationals included. Reading material for any treasury.
• Fairwind: Gemini 3.8 Flash Cyber for vetted defenders only – Google. Google releases its model for autonomously finding and patching vulnerabilities only to security, incident response and penetration testing teams; OpenAI follows with Daybreak, one billion US dollars of subsidised access for defenders. Offensive capability is being rationed, defence subsidised.
• TD Bank all but hits its annual AI target after three quarters – PYMNTS. 195 million Canadian dollars of realised AI value against a full-year target of 200 million, according to the third-quarter earnings presentation. Chief executive Raymond Chun: “We have essentially hit our fiscal 2026 target.” A bank that quantifies its AI business case and tracks it is still the exception.

“AI is grown more than designed.”