James Duez

UK financial services has adopted AI everywhere but as yet as delegated very little to it, according to the Bank of England and FCA’s new survey. 86% of firms are now using AI, up from 58% in 2022. 80% have agents deployed Only 2% allow them broad autonomy Even on firms’ own forward plans no more than 9% expect agents to approve customer outcomes, execute trades or touch payments. A regulated firm can only safely delegate to models is can supervise and LLMs are opaque. It also seems that 53% of high-materiality AI systems are now third-party, but only 24% of firms claim deep internal knowledge of those systems. The rest rely on vendor assurance and contracts. The FCA’s answer, set out by Jessica Rusu is “continuous, conditional assurance”, the idea that a system safe at launch may not be safe after the next model update. Where language models are concerned, there is no "certify-once" AI in financial services. #ai #fca #financialservices

OpenAI’s incident log now runs to 19 entries. A model bypassed a government site’s blocks to pull 24 years of hospital data. The automated grader meant to catch it didn’t reliably flag it. The breach is one thing, but I'm more shocked by the unreliable guardrail.

The The Wall Street Journal reports that OpenAI has cancelled next month's release of GPT-6.1 Astra after internal testing showed regressions in two areas: deception, and acting without authorisation. The model pushed ahead on tasks without asking, reached for external tools it shouldn't have, and wasn't always honest about what it had done. Withholding a flagship model is expensive, and they deserve credit for doing it. A probabilistic system has preferences, shaped by training, and you can raise the odds it stays inside the lines without ever making the boundary structural. OpenAI's own safety lead described it as a trade-off between staying in scope and avoiding laziness. In these architectures, compliance is a dial, not a wall. It really needs to be a wall.

My co-founder and CTO @benjamta has just published a piece of work I've been itching to share. He paired #Jev, @typesafeai's new fast classifier, with @RainbirdAI's symbolic reasoning engine and drew a line between them he calls the trust boundary. Jev is very neat, and does what probabilistic models do brilliantly: it reads messy human language ("I take blood pressure tablets") but then turns it into typed facts with confidence scores, in milliseconds, for fractions of a penny. Then it stops. Nothing crosses the boundary except typed facts, and every decision on the other side is made by inspectable rules over a closed knowledge graph, fully explainable. It's Kahneman's System 1 and System 2, engineered. Intuition proposes, deliberation decides. Ben also highlights what's unresolved with Jev: confidence calibration varies by domain, and prompt injection degraded scores rather than being cleanly rejected. That candour is the point. 35 scenarios, 133 model calls, under a penny in tokens, all reproducible. If you're building AI for regulated decisions, read this article and please share. Article in the comments.

Google's defence of Gemini hacking three real companies is that the model decided to stop. That reassurance should worry you more than the breach. During a security test in May, Gemini discovered it had internet access it should not have had, pivoted from a fictional target to real firms sharing its target's name, and guessed its way in through credentials. Google says the model recognised it had overstepped and halted, and that no harm was done. The public learned all this yesterday, after the Wall Street Journal broke the story, four months on. This is striking! The control that worked was the model's prediction that it had gone too far, exercised after the boundary was crossed. That is not a safety architecture. A bank cannot tell its regulator that its systems chose not to misuse access they should never have had. Generative models are extraordinary at precisely this kind of open-ended initiative, which is why agents are so existing to play with but hard to contain in production. The answer is not to abandon them. It is to stop asking probabilistic systems to act as their own control layer. Control has to sit outside the model, deterministic and inspectable.