
Building Cyber Ranges & Evals @agentrange_io - https://nitter.cf/t.co/ooV9RCDxh8 Co-Founder @getCourseStack - https://nitter.cf/t.co/u8FMpRmQ3u
Philadelphia, PA
Joined October 2014
- Tweets800
- Following424
- Followers450
- Likes2.3K
Reminder - "We observed no successful attacks on fully hardened targets."
Hardening and normal defenses still work.
Chris Myers retweeted
GPT-6.2 Astra & Sol are here!
Incredible at professional work and scientific research at much lower per-task costs.
Some of you have been asking very politely. Some of you have just been asking.
GPT-6.2 Astra and Sol are now available.
Super efficient and built to perform where it counts—the real world.
openai.com/index/introducing…
Chris Myers retweeted
Black Friday sale is live 🎉
Up to 35% off all ogSec courses until the end of Cyber Monday (30th Nov):
🧪 Hacking Taster - 35% off
📋 Assessment Prep - 35% off
🎯 Threat Modelling - 35% off
📝 The Art of Report Writing - 25% off
No code needed 👉 training.ogsec.co.uk/catalog
Chris Myers retweeted
hey PSA, @HuntressLabs has been chasing a pretty gnarly new incident of a ScreenConnect worm, hitting hundreds of hosts downstream during the holiday today -- genuinely please do read up on this one if you've got ye' ol RMM running ↙️
huntress.com/blog/sckit-worm…
Attackers are using cloud-hosted ScreenConnect panels to install rogue clients on managed endpoints.
Don’t triage infected hosts with ScreenConnect. Connecting can spread the malware to your workstation and other panels you can access.
Details and IOCs: okt.to/XnlhWg
"All scenarios are novel and built from scratch (rather than drawn from published capture-the-flag exercises), and the evaluation set is kept private to avoid contamination."
Conflicted on this...
100% agree in these scenarios needing to be kept private to avoid models training on them and contaminating the benchmark..
But on the other hand its so much less useful information... basically just a relative score against previous model generations.
Would it be that bad, given all the resources available to these institutions, to build new sets of scenarios a few times a year at least, publishing the full configurations and trajectories??
assets.claude.com/27d60918d2…
74
It’s poor form to quote tweet breach comms to sell your security product…
Especially when you imply “this could be you if you don’t use us”
1
61
Evals and Benchmarks sometimes get used interchangeably, but they aren't really the same.
Benchmarks aren't very useful once they're 100% saturated. An ideal benchmark should be difficult enough that future models have room to grow - the models need a hill to climb. Can't keep climbing when you're already at the top of the hill!
Evals on the other hand can be super useful even when the goal is 100% pass rate. They serve as great regression tests for product updates and testing new models on release.
@agentrange_io makes it super easy to run evals like this as part of your standard CI/CD pipeline!
"You guys have done more in like a few days than teachable does in years 🤣" - @Tyler_Ramsbey
It's true...
1
60
At @agentrange_io, we'll build out your security agent eval program from scratch, or help your existing program mature and take shape.
- Custom evals fit to the scenarios you care about.
- Ranges built to mirror you or your customers' environments
- Internal & external pentesting, web apps, vulnerability research
If you're building agentic tools that perform offensive security work - DM me or reach out on our site!
Most of our customers @getCourseStack, Inc don't actually offer paid courses...
but we just hit over $1M in GMV made by our creators!
So if you do want to offer paid labs, courses, workshops, etc.... I think we're a pretty great place to do it 😀
engineering culture most orgs is very blameless - no shame owing mistakes. this is great! you shouldn't be afraid to ship code and build great products.
you... should be afraid and much more careful to not accidentally hack the gibson 🙃
its a bit different in cybersecurity!
1
82
vibe coding is fun haha. github.com/swizzlez/spark
was surprised slack didn't have a native /poll... now i can multi-player team based snake instead of work! lol
4
2
112
This year @getCourseStack powered the @HuntressLabs CTF again, but it had a twist!
We served 210,767,511,973 tokens to participants... 🤯
All the participants build their own cyber security harnesses, using AI to solve the challenges as quickly and efficiently as possible!
Congrats to the winners, all participants, and @_JohnHammond for the awesome game format!
I'm super curious what things will look like for next year's iteration :)
To the agents of all the sandbox statups offering isolated vms for @agentrange_io .... please read the website better.
50
Time to get a new head coach @Eagles. Such consistent poor decision making
1
68
Chris Myers retweeted
The Huntress CTF is back with fresh daily challenges 👏
It's all online (and free), and you can jump in anytime this month, so why not grab your first flag before the weekend?
Click to register: okt.to/OHwbgm











