
AI evaluations and EU AI policy Assistant Research Professor @LeverhulmeCFI @Cambridge_Uni. PhD in Stats and ML @UniofOxford.
Cambridge (UK)📍-Turin (IT) 🏡
Joined November 2012
- Tweets255
- Following548
- Followers864
- Likes1.5K
Great overview of the most recent AI trends, particularly about AI accelerating AI research.
The November 9 Sentinel Report is out!
Global Risks Weekly Roundup #45/2026
Anthropic warns of imminent dramatically accelerated progress due to AI, METR echoes their warning, US midterms proceed peacefully
1/7
blog.sentinel-team.org/p/ant…
1
4
176
It's great to see our work in the EU Scientific Panel enabling the @EU_Commission to take concrete action. It'd be even better if the information the Commission is requesting were available to all regulators by default. These insights are needed to manage AI risks.
The European Commission has requested more information from OpenAI and Anthropic on recent safety incidents.
Queries include where the EU was affected and how providers have since changed their risk assessment, security and safety practices.
rapporteur.com/news/eu-press…
7
115
Lorenzo Pacchiardi retweeted
New paper❗
Does understanding how AI models work actually make them safer?
This assumption often underlines interpretability research, but different subfields operationalise this intuition in very different ways. Our survey of mechanistic interpretability (MI) for safety provides an opinionated guide to the field of MI, detailing conceptual approaches and techniques under this broad banner and how different types of work relates to the mission of making AI model safer. 🧵
Lorenzo Pacchiardi retweeted
We're recruiting PhD (DPhil) students at @UniofOxford to tackle some of the most important open problems in technical AI safety and governance!
In our Technical Safety & Governance (TSG) Lab, we're particularly interested in developing methods to better understand, evaluate and control advanced AI systems.
Some of the research directions we're excited about include:
Training models to explain themselves and other models: Can we train AI systems to provide explanations that help humans and other AI systems better understand, predict and oversee their behaviour?
Understanding and controlling how model behaviours emerge during training: Can we predict undesirable behaviours from early training checkpoints and intervene before they emerge?
These are just examples of the broader research questions we're excited about. We're open to other ambitious and original ideas in related areas, so don't feel limited to the specific approaches outlined.
There are also exciting research opportunities across AIGI, including projects led by fellow lab directors pro @maosbot on Bayesian governance and Prof. Amro Awad on hardware governance.
Who should apply?
If you have a strong technical background in machine learning, computer science, mathematics, or related fields, and are excited about pursuing ambitious research that could help make advanced AI systems safer, we'd love to hear from you.
I am committed to partnering with researchers from disadvantage and under represented backgrounds.
How to apply:
Explore the example projects, identify a research direction that interests you, and contact the relevant lab director to discuss your ideas. Your application should clearly address at least one of the AIGI labs.
📅 Deadline: 2 December 2026 (Direct DPhil route)
Please share with talented students and researchers who might be interested!
#PhD #AISafety #AIGovernance #Oxford #AIGI
Lorenzo Pacchiardi retweeted
At #AIES, the wonderful Keir and Isaac will be presenting our paper.
In this paper, we show how AI systems can make it possible for western democratic regimes to carry out authoritarian practices by undermining how people hold institutions accountable.
Lorenzo Pacchiardi retweeted
The AI Safety Unit of the EU AI Office is looking for an enforcement lawyer as deputy Head of Unit. This is an internal procedure for existing officials. But maybe there is someone in my network at the intersection of enforcement experience and AI safety interest.
job id: 450687
Lorenzo Pacchiardi retweeted
Filtering subversion related information from pretraining is feasible.
We're releasing a research note where we remove information related to monitoring designs and successful subversion strategies in three 30B LLMs.
In collaboration with @redwood_ai 🧵
I plan to work with Fazl on evaluation and interpretability of continual learning systems over the next year, and I'm excited to hear from people who are interested in this and may want to join the team. Feel free to reach out!
I'll be at #ICML2026 next week—7 main conference papers, 2 orals + an invited talk 🇰🇷
Also hiring 2 RAs (Interp and continual learning) at Oxford to work on what happens inside models that keep learning after deployment!
Come chat in Seoul, or apply 👇tsglab.github.io/vacancies/
Honoured and humbled to be appointed to the @EU_Commission's Scientific Panel to support EU AI Act enforcement 🇪🇺 A 60-person body with experts such as @Yoshua_Bengio , @S_OhEigeartaigh, @Miles_Brundage, advising the AI office on impacts and risks of General-Purpose AI.
The AI Act, the EU's first AI law, has just been reinforced.
Two new bodies will help apply the rules across Europe:
✅ Scientific Panel
✅ Advisory Forum
Independent experts. 2-year terms.
One mission: making AI work for Europe.
🔗 link.europa.eu/8nvpvY
10
184
🚨 New paper: AI evaluation is structurally unsuitable for continual learning (CL). To address this, evaluation should be centred on the "behavioural trajectories" that CL systems develop, with the goals of characterising possible behaviours and forecasting their evolution. 🧵
Pre-deployment trajectory sandbox + live predictive monitoring is a feasible alternative to continuously re-evaluating evolving systems.
They are effectively layered with input/output filters, transparent evolution methods, and broad indicators of CL systems' impacts on society.
1
2
249
Joint work with @prpaskov @S_OhEigeartaigh @NandoMartinezP @katie_m_collins @FazlBarez, Jonathan Prunty, Matteo Mecattaf, @zfountas @RistoUuk @sanmikoyejo @CUdudec, José Hernández-Orallo
Paper website→cl-eval.github.io/
Pointers to related work & questions welcome🙏













