Gutenberg retweeted
What features constitute AI-generated text? We used our data-centric interp SDK on @pangram's public dataset and found some interesting features
Gutenberg retweeted
New paper! We used Sparse Autoencoder (SAE) embeddings to understand how agent behavior actually changes during post training. We analyzed thousands of rollouts in a Diplomacy environment and uncovered generalized reward hacking, weird roleplay, and the root cause of an unsuccessful training run. All of which our LLM baseline couldn't find.
Gutenberg retweeted
I’m excited to share a bit about what I’ve been working on in data-centric interpretability. It’s a paradigm with great potential, and it’s ready for application.
Our production system has already been tested on real customers’ data to improve their engagement metrics. But we can’t share that, so we’ve made a synthetic dataset to demonstrate. Basically, pretend you’re a marketing firm trying to generate messages that engage users, and they have this hidden preference in the data. LLMs and embeddings fail to catch it, but our system can.
Data-centric interp is efficient and good at discovering unknown unknowns. We’ve demonstrated it works on short texts.
But what’s really exciting is to extend this to longer contexts and larger datasets. Monitoring agent traces is costly. Training datasets are too massive to even try. But data-centric interp gives you rich representations of data, which can systematically uncover insights, anomalies, and other qualities over large datasets.
We can help find these hidden features in your data that will drive your metrics.
I’ll be at NeurIPS and am always down to chat data-centric interp


