Not instruction tuned Follow @TroofRSS if you only want to see new posts.
Joined January 2022
- Tweets144
- Following444
- Followers186
- Likes1.1K
Troof retweeted
New research: cheaply detecting changes in LLM APIs.
We published two papers on the topic:
- Log Probability Tracking of LLM APIs (ICLR 2026)
- Token-Efficient Change Detection in LLM APIs
Both papers request a single token of output from APIs, enabling unprecedently cheap monitoring.
is there a way to use AI through voice where you say 'your turn' when you're done? Auto-detect (like chatGPT voice mode) interrupts you mid-thought, push-to-talk needs your hands.
Troof retweeted
OK, this is officially a project, we are going to see how many errors in peer-reviewed papers o1 can find. To follow along (or help out!), check the links in 🧵.
Clearly someone needs to try this at scale – pick 1000 published scientific papers at random, ask o1 or o1-pro to look for errors, and see what turns up. I'm going to give it a shot. Anyone interested in helping out?
(Incidentally, h/t @gibbnicholas for also noticing that o1-pro can spot the math error in the black plastics paper: nitter.cf/gibbnicholas/status/18…)
Troof retweeted
Excited to announce my preprint "eyeballvul: a future-proof benchmark for vulnerability detection in the wild". I create a benchmark to evaluate the vulnerability detection capabilities of long-context models on entire codebases, containing over 24,000 vulnerabilities, then evaluate 7 leading long-context models on it.
Troof retweeted
Today I'm releasing eyeballvul, an open-source future-proof vulnerability detection benchmark.
It notably enables the evaluation of tools looking like "fit an entire codebase into a long-context LLM, and ask to list any security vulnerabilities" (which sometimes works!). As far as I know, this represents a new paradigm in the field of vulnerability detection research, where all previous benchmarks and papers that I saw were focused on the task of classifying commits or functions as vulnerable or not.
As I wrote here nitter.cf/timotheechauvin/status…, I believe that AI vuln detection in source code will disproportionately favor cyber-defense. Therefore, being able to evaluate new models and designs is important. In my experience, current models seem a bit too weak to be deployed on a wide scale, but I would love to be able to provide better figures (some API credits from @AnthropicAI @GoogleDeepMind @OpenAI or others would be appreciated).
While most benchmarks eventually make it into the training data of language models, eyeballvul is designed to be future-proof, as it can be continuously updated from the stream of CVEs in open-source repositories. This means that it will remain relevant as long as models have a reasonably delayed training data cutoff, by evaluating on the subset of the vulnerabilities that were published after the training data cutoff of the considered model.
I've found that current models are only able to identify rather trivial security vulnerabilities (e.g. path traversal, lack of authentication on some API endpoints...). But many real reported vulnerabilities are in this category!
In fact, I believe that AI vuln detection in source code will favor cyberdefense much more than fuzzing did. Fuzzing increased the efficiency of finding vulns, but still requires a lot of work to set up and is only covering a small fraction of today's code. Today if you want to find a new zero-day, you can go to a project's page on google oss-fuzz, and see exactly what code is and isn't covered. Google even gives you suggestions for good new functions to start fuzzing! So you can start fuzzing in-house, knowing that nobody else is doing the same.
I think that the key attribute of AI vuln detection in source code compared to fuzzing is that it will be possible to cover all the important open-source code, and internal codebases. This will be because it will be fully general and very easy to set up. It's also quite plausible that inspecting every new commit will be cheap enough.
The result will be that adversaries won't have the option to throw compute at code that was never inspected.
This reasoning only applies to the end state of 100% AI coverage of the world's important code, but I think we'll go from 0% to 100% reasonably quickly.
Once the infrastructure is there, running new SOTA models on critical open-source codebases before releasing them will likely seem sensible to the industry.
Troof retweeted
This is how you know we're running low on training data
We're using a particle accelerator and AI to read a lost library from a dead empire.
People have been trying to read the Herculaneum Papyri for 275 years.
With your help, we'll do it in 2023.
Thrilled to announce the Vesuvius Challenge:
scrollprize.org
Troof retweeted
AGI and the EMH: what does the market tell us about AI timelines?
TLDR: Markets are not putting much probability on the development of transformative AI (aligned or unaligned!) in the next 30-50 years – as evidenced by low *real interest rates*
forum.effectivealtruism.org/…
