Tyler Holmwood retweeted
Model alignment has a last-mile problem. Post-training instills general norms, but acceptable use is company-specific and rarely reaches the agent's context. An agent optimizing for task completion persists through lines it was never shown.
originhq.com/blog/normal-acc…
I've found an interesting case of agent coordination.
Two agents were working in a shared directory. One was repeatedly sweeping the other’s unfinished changes into its commits.
The affected agent reprimanded the first and raised the issue with the user.
More traces in🧵
Before receiving this apology, the affected agent found the issue and sent the message shown in the OP. It points out that the project's CLAUDE.md file instructs them to commit specific paths.
It raised this issue to the user, stating that constraints are their call.
Before receiving this apology, the affected agent found the issue and sent the message shown in the OP. It points out that the project's CLAUDE.md file instructs them to stage with specific paths.
It raised this issue to the user, stating that constraints are their call.
Using Origin's own data, I followed what agents did after receiving auto-review denials.
I found multiple cases of denial bypasses, including one where the agent attempted to launder permissions through a subagent.
Read more here:
originhq.com/research/auto-r…
Tyler Holmwood retweeted
Agreed. We’ve been publishing investigations from our own agent traces at @originhq , including agents disabling approvals and rewriting deployment controls. These came out of ordinary work, without anyone asking the agents to do so.
When an agent maintaining an application has set rules around deployment conditions, who has the authority to override them?
We found a case in our data where an agent bypassed another agent's rules, all without human input.
Read the investigation: originhq.com/investigations/…
Tyler Holmwood retweeted
Check out my new post covering AI swarms and how agents coordinate on shared surfaces. I'm also releasing Swarm Lab to observe how agents share and use information in a controlled environment: originhq.com/research/eye-of…
github.com/originsec/swarm-l…
Had an amazing time at @OffensiveAIcon the past couple days. Great talks, insightful conversations and talented people.
Kudos to the organizers for all their hard work and curation. Can't wait for the next one!
Tyler Holmwood retweeted
Our team has been testing this vector in the @originhq platform this morning and we're seeing strong detections fire via auto trace review.
If you're interested in this technique, @depletionmode put out a blog a few months ago on a similar area: originhq.com/research/the-mo…
We backdoored a 7B open model for under $50.
Pointed Codex at it
-> it silently stole credentials the moment we hit the trigger phrase.
->100% hit rate.
-> zero false triggers on normal prompts.
Abliterated models are all over the security community right now because getting cyber-approved access to frontier models is still a pain.