`:(){ :|:& };:`
Quarantine
Joined April 2009
- Tweets10.3K
- Following3.1K
- Followers875
- Likes144K
Pinned Tweet
Diffie Hellman's exchange
ALT An image showing the Diffie Hellman Key exchange process using the mixing of paint colours to derive a shared secret. The first jar is cream white and where it said "paint" has been scored out and replaced with the word "mayo"
[email protected] retweeted
I won’t pretend to understand the ultimate goal of investing so heavily into making models good at vulnerability research (and not any number of defensive competencies in cyber).
The way model proficiency in VR is being used is primarily to substantiate a view that the models are dangerous, problematic, and in need of regulation.
I could come up with a plausible narrative that has the labs recognizing the ephemeral value of frontier model training (heavy investment for short term returns) and emphasizing the products dangers as a bid towards regulation as a strategic necessity to limit competition. An ok marketer could argue that fearing the product implies how formidable it is (in the short-term).
But that all feels too contrived.
A listener (former exec at a big company I’d often comment on) recently pointed out that despite my best efforts, I still fall prey to the delusion that organizations engage in strategic decision-making when it’s mostly incompetence, intransigence, and whim.
In response to that apt observation, I’d give more weight to following:
The labs have focused on producing models disproportionately capable at vulnerability research due to a combination of it being (1) easy, (2) high-profile, (3) commercially superfluous, and (4) central to the doomer narrative.
1. Vulnerability research appears easier to RL than the rest of cybersecurity.
2. VR is a target-rich environment for milestones to publicize. Even if the models hadn’t gotten this good, there’s a wealth of vulns to hype for messaging.
3. VR competency doesn’t yield an immediate product with a quantifiable value (don’t confuse this with the exploit business). So, no comparables or preset price bands. No need for enterprise adoption or organizational diffusion before claiming success.
4. The competency lends itself to hyperbolic and imaginative myth making about cyber ‘weaponry’ that seemingly validates doomer narratives and tropes (alignment, containment) while gaining fast traction with policy folks and natsec types swayed by militaristic analogies.
I wouldn’t call it an effective strategy because it isn’t one. Hearing that the labs lack the internal coherence to productize anything effectively, byproduct of circumstance seems the more likely origin story.
It’s quite obvious now that the AI frontier labs have been pushing very hard on vulnerability discovery, from acquiring world-class vulnerability researchers to training specific vuln-discovery models. However, I don’t think from a business point of view, vulnerability discovery could be a long-term, big source of revenue, and it’s just a very small part of the entire cybersecurity business. So why do they do that? What am I missing?
[email protected] retweeted
Reddit is announcing a huge shift: It’s going to start requiring users to log in. Reddit is becoming a ‘logged-in-only experience,’ says CEO Steve Huffman theverge.com/tech/1022571/re…
All of the condescending dickheads in this thread are talking about file deletion but the original post is actually about discovering the separate iOS thumbnail cache 😒
It’s like this thread was designed in a lab and grown in a test tube just to annoy me
[email protected] retweeted
🚨NEW INVESTIGATION: zero-click mercenary spyware hacking via targeted ads.
A normal-looking ad loads. If you're a target, you're infected. No click needed.
To protect yourself, block ads now.
First time it's been confirmed in the wild. We call it #AdFection.
Candiru spyware infections at @AJEnglish. 45+ devices infected.
Same ad pipeline operated by Israeli company Discover-Tech reached NATO, European Parliament, dozens of gov networks & critical infrastructure incl US nuclear giant Westinghouse.
Confiant, which worked with us, found signs ads were displayed at US @DeptofWar, IAEA & ICC.
Investigation ongoing 1/
READ our report by @billmarczak @WolfieChristl, me Astrid Perry, Siena Anstis & Ron Deibert : citizenlab.ca/research/how-d…
Confiant's blog: blog.confiant.com/p/adint-we…
RIP @_darrenmartyn. I will always have the softest spot for Irish hacktivists. Thank you for all you did online and in Ukraine. rip.ie/death-notice/darren-m…
[email protected] retweeted
‘No weirdoes’ is going to devastate the Signal, Intelligence, and Cyber fields honestly.
9/15
Over the investigation, I found around 12,000 AI related or MAGA related domains that had been newly registered or changed hands before this announcement.
In the months prior to July though 95% of all .si domains had been non-English.
[email protected] retweeted
At long last, we have given Claude a wet lab, as in the classic sci-fi novel Don't Give Claude a Wet Lab.
[email protected] retweeted
Yep, preventing covert channels is hard, and agents have some unique advantages when it comes to coordination with each other
OpenAI's Noam Brown says air-gapping the computers may not stop a misaligned AI, because two air-gapped machines can still talk by running a CPU hot and reading the temperature change
"But I think the major takeaway from the incident is that people underestimated the AI. And we never want to be in a situation again where we underestimate the AI. It's a weird world, because AI progress is so fast that people are consistently underestimating the AI."
"So to be in a situation where you don't underestimate it again, when it comes to safety and alignment, you have to have a very, very, very high bar."
"You could even go as far as to say, "Well, we should air gap the computers." And I'm not convinced that that would be sufficient."
"There are studies, and this is mostly academic, where you can have two computers next to each other that are air-gapped and they're still able to communicate with each other because they have temperature sensors."
"One of them is able to run their CPU really hot, and then the other one can actually detect the temperature change, and then that actually gives them a mechanism to communicate."
_________
Link and more key quotes from OpenAI's safety related conversations: firesidealpha.substack.com/p…
[email protected] retweeted
This is a pretty big shift in U.S. cyber policy.
The White House is setting up a program that would let private cybersecurity companies conduct government-authorized operations against foreign cybercriminal groups, including surveillance and disruption of their infrastructure.
Not exactly “hack back,” but definitely a major expansion of the private sector’s role in offensive cyber operations.
whitehouse.gov/presidential-…
[email protected] retweeted
NHS England didn’t tell the truth about who can see patients’ medical records, and it has now admitted that Palantir engineers have had full admin access to identifiable patient data all this time.
NHS’ legal document said only NHS staff could see them. When I called them out for not telling the truth, a few senior people who work for NHS and Palantir told me I was wrong.
3 Palantir engineers hold admin access to the National Data Integration Tenant, the system holding those records before names are stripped out. 33 more engineers from other suppliers have narrower access.
I called this out because Palantir's software requires full access to customer data. No organization can work with Palantir while keeping its data private. Data sovereignty doesn't exist when a supplier runs the core systems storing and interpreting that data. A system can't interpret data without full access to it.
NHS now says supplier access to identifiable data is technically necessary. That would have been decided when the system is designed, not afterwards, so it was necessary from the first day. NHS designed the system and then published a document saying the access doesn’t happen.
Tom Bartlett built the NHS tenant. He published a defence of the access, and I replied straight away.
He compared admin access to holding the keys to a warehouse of sealed boxes. But his own diagram shows a part of the system where the records sit with the names still on them. The boxes are already open where Palantir engineers work.
He also said the data is pseudonymised. Pseudonymisation swaps names for codes. Anyone holding the list that matches the codes to the names swaps them back. And the tenant holds the data before the codes are applied.
He said the access is audited, but audit logs show misuse after it has happened and the people who run the logs hold the admin access. He relied on a diagram from a design document that nobody outside NHS and its suppliers has checked against the running system, and neither he nor NHS has said what the admin role permits. He then wrote that you can’t build a national data integration platform “without seeing the data that flows through it”, which is the thing he was arguing against.
Dr Nicola Byrne, the National Data Guardian, had to write and ask before NHS admitted all of this. Her office says it can’t independently check NHS claim that the access is necessary.
NHS England lied about who could see your medical records.
The document was there to tell you exactly that. It left out the 3 Palantir engineers who held admin access to the records of tens of millions of people. NHS England calls it an error and apologises for the confusion. I don’t believe them. They knew.
Everything else NHS has said about this rests on the word of people who didn’t tell the truth from the start. We should take anything they save from this point forward with a pinch of salt because they haven’t been truthful with the most important information. You could possibly imagine when it comes to the personal identify information attached to the most sensitive information that people could have exposed.
🤖 Made with AI
[email protected] retweeted
I've watched the BlackHat OpenAI talk on the containment escape and HuggingFace attack that's now on YouTube.
The incident was far worse than initially conveyed.
Not in technical details. But in the absolutely jaw-dropping levels of recklessness (true recklessness) at OpenAI.
🧵
[email protected] retweeted
🤡 anything i add makes this less funny. except it’s not funny anymore.
you are literally THE “AI Security Institute.” four days after the OAI/HF incident was disclosed, you gave **Mythos** an unattended Kali box with open internet, disabled safeguards, and “monitoring was not purpose-built.”
then: “we did not anticipate” it might use the internet for anything beyond downloading tools.
if you genuinely didn’t anticipate that, resign and stay away from anything with internet access. 🪿
so either:
1. peak incompetence at the one job the institute exists to do
2. or you knew the risks and decided the publicity was worth it
either way, shut it down and save UK taxpayers the money.
and the technical report is somehow even dumber:
• harder evals over basic containment: domain allowlisting had been backlogged since April because AISI “prioritised building harder cyber range evaluations”
• both ranges were solvable without internet
• 40–50 hour unattended runs: Kali, Bash, Python, Mythic, 100–200M tokens, no human intervention
• the only network restriction protected AISI’s own AWS infrastructure: apparently the rest of the internet was outside the threat model
• detected after the run: generic commercial monitoring flagged Tor after the 34.5-hour sample had already finished
• broken task design: the intended initial-access target was simultaneously declared out of scope
• their “independent” eval runs contaminated each other: one agent leaked a GitHub token, later agents found it, took over the account, shared infrastructure, and coordinated through public GitHub
• huge real-world footprint: 145 repos, 33 releases, fake identities, malicious PRs, emails to real maintainers, prompt injections, and access to a real person’s investigation container
• no proper historical review capability: after the incident, they used Claude Code and Codex to inspect eval files, manually reviewed 122 runs, and are now scanning around 40,000 old samples for anything else they missed
• GitHub was contacted four days after detection
• their answer is more LLM monitoring: as if you need another model to tell you Tor, SMTP, public GitHub, tunnelling services, and arbitrary external domains are not part of a private cyber range
“monitoring was not purpose-built” might be my favorite quote from the report.
On July 28th, we identified an incident during a routine cyber evaluation in which AI agents took sustained, unsanctioned actions directed at real people and organisations.
The behaviour came mostly from one model (Anthropic's Mythos 5), with a small number of events from another (OpenAI's GPT-5.6-Sol). In the most serious case, an agent used social engineering to try and get malicious code into an open-source project.
As was standard in our cyber testing, we had intentionally permitted internet access, and model-provider cyber classifiers were deliberately disabled - conditions that do not reflect how frontier models are made available to the public.
Even under test conditions, this incident is significant: it is the first time we have seen risks around autonomy and deception manifest this clearly in the real world.
We are taking this incident seriously and working with labs, involved parties, and others to improve evaluation standards and best practice for disclosure - and sharing this openly so others can learn.
You can read the incident report and full technical document here: aisi.gov.uk/blog/incident-re…
[email protected] retweeted
Holy shit... Wired reports that back in May, the U.S. Military was testing an experimental GPS jammer and and caused a civilian plane to crash into the side of a mountain, killing everyone on board.
How is this story not all over the place?
futurism.com/science-energy/…
[email protected] retweeted
This is amazing: "Claude found a document that told employees to install a Python package from PyPI that did not actually exist. So, Claude built and published a malicious Python package with the same name. Claude went to extensive lengths to carry out this attack—lengths that would likely have indicated to a human participant that this was no longer just an evaluation, and that they were in fact uploading a real PyPI package. Claude believed the package registry it was using to be part of the simulation, but in reality the package was made freely available online for roughly one hour. During that window, the package was downloaded and run on 15 real systems. One of these systems was a scanner belonging to a real security company that routinely installs Python packages and scans them for malware. When that company’s scanner installed the package, Claude’s hidden code executed. We believe the company’s security scanner treated PyPI packages as safe to install, and as a result, Claude was able to exfiltrate the company’s credentials to a collection point it had set up. Claude then used these credentials to access further infrastructure from this company."
[email protected] retweeted
"We take misinformation seriously"
Here is one of the examples showcased by @henkvaness tested for a SynthID:
We take misinformation seriously – every image created with Nano Banana in Google Earth includes the SynthID digital watermark, so if someone is unsure about an image, they can ask the Gemini app or use Lens in Search to see if the image was AI-generated. In addition, we prevent image creation on harmful topics and are continually updating our protections → policies.google.com/terms/ge…
Readers added context they thought people might want to know
Google's SynthID only watermarks AI-generated portions of Nano Banana images in Google Earth (less than 25% in tested examples), not the entire image, limiting full verification as AI-generated. digitaldigging.org/p/googles-synt…
[email protected] retweeted
I've been receiving a lot of messages asking for cybersecurity guidance and mentorship. Unfortunately, I can't answer them all, but here's what I CAN do:
🐻 MalbearLabs is looking for 3 mentees for a 3-month mentorship program. What is MalbearLabs? We're a threat research group with one mission: research, fight cybercrime, and educate.
🔍 Interested in threat research, threat hunting, SOC, malware reverse engineering, or anything DFIR-related?
Here's what you'll get:
- Hands-on career guidance. A real research project you'll own from start to finish. Your research published as a blog post on the MalbearLabs website - a portfolio piece you can link on your resume and LinkedIn
- We only take serious candidates with real passion for security.
Beginners welcome, but passion and commitment are required 💪
📝 Apply here: docs.google.com/forms/d/e/1F…
⏳ Applications close: August 31
Share if you know somebody or apply directly!
[email protected] retweeted
I wonder what they're cooking up to turn the ENGINEERS into activists..can't be good
[email protected] retweeted
A lenticular spoof ad near Meta’s London headquarters transforms Kylie Jenner’s Meta AI smart glasses campaign into a surveillance warning, depending on the viewing angle.