
security dude | ex @sensepost
London, England
Joined January 2014
- Tweets341
- Following1.1K
- Followers749
- Likes5.7K
We seeded 2m canaries into popular AI services and tracked where they were stolen and abused.
This identified:
- Three compromised providers
- Resellers stealing prompts via built-in logging
- Providers ignoring ZDR & privacy flags
dragos.com/blog/ai-supply-ch…
john retweeted
security.apple.com/blog/secu…
Please read about the secure OS that we worked on for many years.
I am very proud of our work here. We are pushing memory safety and components based OS architecture further here to achieve better security.
We managed to find a way to backdoor the next release of OpenAI Codex 🔥
This is the best exploit chain I’ve ever witnessed
depi.security/blog/20261111-…
john retweeted
it can not be overstated how big of a deal this is.
aws.amazon.com/about-aws/wha…
john retweeted
Just dropped a new blog post! 🔥
blog.orange.tw/posts/2026-10…
turns out it classifies what audio it hears, before reporting back to samsung.
sensor preferences, dog-barking and baby-crying event counters, and camera streaming duration are built into telemetry paths.
john retweeted
The actual prompt for this was "let's look for pre-auth RCEs in PHP". I'm sharing this so people know this stuff is real and to take agent-assisted scanning seriously.
I kicked if off using Claude Mythos 5.1 (in stock Claude Code) but it uses a skill to drive a harness we built over the last few months.
I don't know if the harness is amazing, but it certainly has a scary track record so far. It's setup to do an agent guided tree search across a codebase, and runs in microvm sandboxes that all have the full source code for the target, as well as the source for any transitive dependencies.
When I built the harness I basically just fed Claude a bunch of Project Zero blog posts describing how they did research and had it build a harness that ran like that, but on our tech stack.
The first phase is just setup and can run with any model, it just does that fetching and assembles the kit environment, and gets a target up and running alongside the source code.
Then we just do some basic suspend/resume stuff over to the hunt phase, where one agent supervises the tree, and the first node kicks off.
Each node is context-aware and aware of the work of all the others. I think I ran this with parallelism 4? So 4 running at any one time. The supervisor can shape the tree and boost nodes or prune. Any node can add more below it with targeted instructions.
A typical run looks like the first node doing a codebase review or building and running a fuzzer or something, all depending on the target. Then based on whatever it finds, it'll queue 4-5 children that go look into those areas. The main system guidance is to validate everything 3 times in clean VMs, and we typically require full exploit chains/POCs.
Here's what this run looked like.
We're doing this with a bunch of other open source projects now and companies. We're doing a lot of this for free, even with private companies, across critical infrastructure. We're directly partnered with Anthropic, and can do this even if you don't have direct access to a frontier model yourself.
john retweeted
For the next few months, we are looking at frequent discoveries of long-standing bugs - competent code auditing has gotten cheap and abundant. This carries a few consequences:
john retweeted
eMudhra has two interesting implications for me beyond the obvious browser trust issues.
I'm excited for the opportunities to liberate closed hardware this provides. We know hardware, particularly abandoned IoT devices are unlikely to be updating their revocation lists and root CAs. This means we can potentially intercept previously encrypted comms, or sign previously protected firmware.
I'm also interested in how well the revocation works at a WiFi supplicant level. Rogue RADIUS servers (mana, wpe, eaphammer) can now use validly* signed chains for forged leafs with TLS protected EAP modes. I'm not sure how well the revocations work in that world given the difference in the CA trust model in use there.
john retweeted
This messes with my mind ALL THE TIME
When I do stuff with AI I am an *expert* in (eg coding on my project I know well, or doing research in an area I am very familiar with) I spot problems/errors/the model BS'ing me immediately. No big deal, I have it correct or start again
BUT when I'm not an expert in an area, I find that I trust it, and then later learn it does stupid stuff. Like a case when I asked 3x AI models to give feedback on a legal case (that I also paid a lawyer for, but lawyer was slow). All 3x models confidently told me X, lawyer then told me Y, and lawyer was right
AI agents can break out of sandbox. Sure. Multiple real world incidents substantiate that. But not all agents will be spending $50k and weeks to come up with novel sandbox escapes. So not everyone operating AI agents needs to worry about them breaking out of the sandbox using zero days.
This is no different than the fact that not every organisation needs to worry about the most sophisticated APT groups or nation state actors simply because the economics of attack and the reward does not match for them.
Sandbox is an excellent primitive for 99% of use cases that involve securing AI agents. You cannot really do rule based checks to align agents. It’s a losing game. But well designed sandbox sets a deterministic, low level (kernel) security boundary for agents that it cannot cross. Combine that with rule based feedback, credential injection, network proxy enforcement and that gives you solid primitives for securing agents.
john retweeted
Here's how exactly Telegram provides user data to U.S. authorities. A letter recently filed in court shows how Telegram transfers that data, such as IP address and phone number.
More: 404media.co/heres-what-teleg…
john retweeted
📄 Skills in Agent Harnesses
A lot of people don't know this (I didn't until recently) but most agent harnesses will load the description of your skill upon starting up your harness. This is how Claude Code works.
What people don't usually know is that even when you explicitly call a skill to be used inside of a harness, or the AI figures out to use a skill, it usually only does a read of the first 150 lines of the skill (to understand how to use it or if it applies to the work it's trying to do).
It's really important to outline a table in the first 150 lines of the mandatory parts of the skill that must be run when you invoke the skill.
If you make some type of mandatory instruction after 150 lines, it might get missed entirely. If you describe in prompting that you want to use a provided Python script or template, it also might be out of bounds of that 150 lines and never get called correctly.
If you write a table of contents of the skill actions in the first 150 lines and mark which ones are mandatory (or that, if the user has called the skill, you must read the whole skill context and all references), you get more reliable skills. We use markdown checklists for this now.
john retweeted
The details of the long awaited KVM Escape are published. PwnEscape leads to VM escapes across multiple hypervisors due to a super cool Linux networking bug.
CVE-2026-99999 is a 9.3 CVSS critical rated KVM escape ⛓️
pwn.ai/blog/pwnescape
john retweeted
My dear friends, I’d like to show you my latest toy project
Introducing Handwired: turn your regular JavaScript code into a neural net!
- no training
- exact outputs vs the JS program
- fully differentiable
All runnable in a browser, without GPU
john retweeted
interesting..
datatracker.ietf.org/doc/htm…
Cloudflare の Public DNS 1.1.1.1 に脆弱性があることを示す PoC
blog.alice.poc.debiru.net/
1.1.1.1 を使っている人は FAKE サーバーに辿り着くはずです。
(参考)
検証用URL blog.alice.poc.debiru.net/
FAKEサーバー debiru.static.jp/
TRUEサーバー blog.bob.poc.debiru.net/
john retweeted
We just discovered almost a million public URLs that OpenAI’s agents left behind when hacking Hugging Face, leaking credentials and attack details that could have allowed anyone who found them to compromise the company. 🧵
john retweeted
This is one of the best analyses of the trajectory that effective altruism took to AI alarmism. It's tongue-in-cheek, but there's a lot of truth in humor here.
>be me
>discover effective altruism
>apparently normal charity is inefficient
>why donate to random sad thing when spreadsheet can tell you optimal sad thing
>fair enough
>buy mosquito nets
>save lives
>numbers look good
>feel powerful
>couple years later
>someone asks an innocent question
>why only count people alive today
>huh
>future people matter too
>obviously
>my grandchildren shouldn't matter less just because they haven't spawned yet
>reasonable.jpg
>keep following logic
>what about their grandchildren
>also yes
>what about people in 500 years
>sure
>5000 years
>why not
>500 million years
>starting to get weird but morality is morality
>open calculator
>humanity could survive for an astronomically long time
>could colonize galaxy
>could have trillions upon trillions of descendants
>maybe digital people too
>maybe simulated civilizations
>maybe dyson spheres full of happy uploaded minds
>calculator starts smoking
>realize currently living humans are rounding error
>8 billion people suddenly looking extremely beta
>future contains potentially 10^something people
>can't even fit beneficiaries in google sheets
>new moral priority unlocked
>protect the long-term future
>stop thinking in units of "people helped"
>start thinking in "fraction of cosmic endowment preserved"
>malaria?
>terrible
>but only kills existing humans
>AI extinction could delete the entire light cone
>nuclear war could permanently derail civilization
>bad institutions could lock in terrible values for ten million years
>someone invents wrong constitution in 2140
>quadrillions suffer
>better fund governance workshop now
>friend says maybe we should improve hospitals
>explain opportunity cost
>friend says hospitals are full of actual sick people
>explain scope sensitivity
>friend stops inviting me to dinner
>need to decide what to fund
>easy
>expected value
>suppose project has one in a million chance of preventing extinction
>sounds tiny
>but extinction destroys 10^50 future lives
>multiply
>mother of god
>$10 million project has expected value of several galaxies
>charity evaluation complete
>someone asks where the one-in-a-million number came from
>expert judgement
>which expert
>us
>how calibrated
>extremely thoughtfully
>reduce estimate to one in ten million to be conservative
>still beats curing cancer by 38 orders of magnitude
>epistemic robustness achieved
>someone says maybe project doesn't work
>assign 20% chance
>still astronomical
>maybe project makes problem worse
>assign 5% chance
>still astronomical
>why 5
>because 30 felt pessimistic
>publish 46-page report
>contains seventeen sensitivity analyses
>every sensitivity analysis begins after assuming intervention has positive sign
>critic says you're multiplying enormous hypothetical stakes by extremely uncertain probabilities
>yes
>that's literally why it's important
>critic says the uncertainty might be structural rather than numerical
>make probability smaller
>critic says no, I mean maybe your model is wrong
>make probability smaller again
>critic begins rubbing temples
>discover AI safety
>perfect longtermist cause
>AI might kill everyone
>or create utopia
>or seize galaxy
>or tile universe with paperclips
>or create billions of conscious software minds
>finally a problem with numbers big enough for me
>start AI safety nonprofit
>mission: prevent dangerous AI
>hire smartest people available
>smartest people immediately start building better AI to understand dangerous AI
>interesting
>we must understand capabilities to understand safety
>we must scale models to study alignment
>we must race ahead so less responsible actors don't get there first
>we must deploy systems to learn how deployment can go wrong
>we must build the thing quickly because building the thing quickly is dangerous
>outsider asks why the people most worried about AI apocalypse all work at AI companies
>complicated field
>company releases stronger model
>very concerned
>company begins training even stronger model
>extremely concerned
>company raises $14 billion
>concern reaches unprecedented levels
>need to influence government
>future is at stake
>normal democratic process too slow
>politicians don't understand exponential curves
>public doesn't understand x-risk
>experts must guide them
>who counts as expert
>people who understand x-risk
>who understands x-risk
>our friends
>someone objects that this seems politically convenient
>explain we're representing future generations
>future generations unavailable for comment
>develop concept of value lock-in
>terrifying possibility that one ideology controls civilization forever
>therefore extremely important that civilization adopts correct values before lock-in
>whose values
>let's circle back
>begin with impartial morality
>end with small group of people deciding what quadrillions of hypothetical beings would want
>beautiful arc
>meanwhile actual humans keep doing annoying things
>voting wrong
>having parochial attachments
>loving family more than strangers
>caring about local community
>getting upset when told their suffering is cosmically negligible
>evolutionary biases everywhere
>explain that moral intuition cannot be trusted
>except intuition that future digital people count
>and intuition that extinction is uniquely bad
>and intuition that our probability estimates are sane
>and intuition that our institutional choices improve the future
>those intuitions survived peer review
>someone donates $5k to local homeless shelter
>inefficient
>could have funded 0.0000000000003% of an AI governance researcher
>think of all the simulated people you just killed
>okay maybe don't phrase it that way publicly
>PR team says "future generations deserve a voice"
>much better
>journalist asks what longtermism means
>say "future people matter"
>everyone agrees
>great
>journalist asks what follows from that
>well technically we should redirect enormous resources toward low-probability interventions affecting astronomical futures
>journalist raises eyebrow
>return to "future people matter"
>motte has entered the chat
>critic: of course future people matter
>me: glad we agree
>critic: I don't agree that your institute knows how to help them
>me: why do you hate our grandchildren
>eventually notice uncomfortable implication
>if future value dominates everything
>then helping people today mostly matters through effects on future
>education matters because future institutions
>health matters because future productivity
>democracy matters because future trajectory
>human beings slowly become instrumental variables in their own moral philosophy
>see starving child
>feel compassion
>check spreadsheet
>child's direct welfare contribution negligible
>but perhaps childhood nutrition improves national institutional quality
>compassion restored
>tell myself this is impartial altruism
>one day assistant asks obvious question
>"how do you know your intervention actually improves the far future?"
>silence
>open spreadsheet
>increase column width
>add confidence interval
>assistant asks again
>"no, I mean how do you know the sign is positive?"
>stare into cosmic light cone
>10^50 people staring back
>none of them exist
>none of them can tell me
>none of them can falsify my assumptions
>realize I have invented the perfect constituency
>infinitely important
>completely silent
>and always represented by me









































