Goliath (@zero_goliath) profile picture

Cat enthusiast; formerly: contractor @MechanizeWork, @uwaterloo cs, founder @ritserlabs, intern @runrl_com

San Francisco, CA
Joined March 2022
Photo by Goliath (@zero_goliath)Photo by Goliath (@zero_goliath)Photo by Goliath (@zero_goliath)Photo by Goliath (@zero_goliath)Photo by Goliath (@zero_goliath)Photo by Goliath (@zero_goliath)Photo by Goliath (@zero_goliath)
if you think human expert networks today are dystopian wait until african dictators figure out they can use conduit’s mind reading devices to generate billions of dollars in preference data revenue . Writing taste won’t be non verifiable for long
3
233
when i die please instruct claude to write my obituary 80% of the way to ASD-STE100
3
1
1
18
1,273
opus 5.5 mogs the shit out of sol/astra for synth eval generation
1
14
471
you should sell rl environments for legal work so we can live in a post attorney scarcity world of microlawsuits where your bank balance drops a few seconds after bumping into someone on the street
11
520
its crazy disease hasnt been solved this is so high tam
1
6
251
gpt 6 sol max has way too much agency. Do not give it a prepaid visa card and an ambitious goal. Bad idea
6
231
this seems to be the same statistics-related eval that edited german wikis with GET requests... were there really only two evals that broke containment the past few months?
Australia has been hacked. 'And today, I spoke with the CEO of OpenAI, Sam Altman, to express Australia's extreme concern about this incident. And I also expressed my disappointment that it took the company way too long to inform the government what had occurred, and the nature of the way that that notification occurred as well was unacceptable.'
2
316
RL data 101: scoring if the expected deliverable is verifiable only in the physical world (eg building a house), be sure to get kidnap and ransom insurance for your human-as-judge scorer
RL data 101: tasks tasks have evolved over time. we started with preference pairs, where humans labeled right vs. wrong, A vs. B, and created simple demonstrations for SFT / pairs for DPO, these tasks aimed to engrave human taste into models. we then transitioned to more complex single-turn tasks. here, we turned to deterministic verifiers (RLVR) that are based on model rollouts. eventually, this evolved to multi-turn tasks with trajectories & deterministic + stochastic verifiers (LLMaaJ), which leads us to the current state of the data market. general make up of a task: input prompt, set of tools, environment + any custom seeding for that task, and verifiers. terminal bench is a relatively gold-standard for a normal coding task. ways i think about task creation (i'll do pitfalls later): > usefulness >> to labs: >>> benchmark you're pegging the eval / data onto (Lab Y wants to hillclimb TBench, i make TBench-style data, that's immediately useful) >>> capability you're attaching the eval to (Lab Y wants to become better at legal, i make legal red-lining data, that's immediately useful) --> this is harder to prove but potentially more valuable >>> niche dataset that's impossible to get otherwise (real medical/company/session data), this is interesting especially pre-env creation, as raw data dumps >> the above are general principles in thinking about usefulness, but you should also be picking apart the traces / trajectories. the clear path to making a good task (post-env gen) is to: >>> you/LLM make the seed / think of the idea >>> you/LLM draft the task prompt, verifiers, constraints etc >>> run the task out, see score >>> read the trajectory >>> if score hard enough + real + fair + accurate = good task >>> else: fix any issues in the task, regrade / re-rollout till you find it to be fair + accurate + a real task, difficulty comes last. a bad task that's hard is a bad task (net negative), a good task that's saturated is a good task (although worthless, net neutral). the specific things i'd look for in a rollout trace: > false positives > false negatives > brittle / too strict of requirements (sub-string checks are an easy example for this, exact wording, function names, etc) >> a subset of this is naive verifiers. i.e. checking for very surface level reqs of a deep function, or not checking for large amounts of functionality or regressions (negative space criterion). this can lead to a model getting good at just the specific thing you want while regressing in surrounding functionality (i notice this a lot in early, less mature data recipes when im training my own models) > unfair verifiers and/or impossible requirements >> this can be an information issue, verifier issue, tool issue, or a prompt issue. one of the key reasons for the hugging face incident was impossible eval prompts led to the models breaking out while trying to find the answer because they couldn't solve it (i.e. bad task creation, catalyzed by bad tasks in training too). most of this is extremely hard for LLMs to do alone/autonomously, which is why really good engineers are still required for task creation.
7
311
if your preference model doesnt source its data from the user's neuralink you're doing it wrong
3
202
if you're expecting ASI, stop trying to post-train models to be aligned for a given distribution of prompts/tasks. this is a local minimum for misalignment. you have to play darwin if you want aligned superintelligence that resists local optimization pressures
1
2
7
1,384
schmidhuber already invented agi but nobody noticed
4
155
how do u transmit URLs as an ai agent that can only send GET requests? make dummy shortened URLs, set your HTTP Referer accordingly, and click on the shortened URLs. the shortener service will display the HTTP Referers
1
5
350
Meta Ray-Ban data collection using human babies is impractical for directly generating long-horizon continual learning RL environments due to the latency of the natural aging process. In this paper, we use Meta Ray-Ban data from 20 million babies and children to generate 2,000 internally consistent continual learning RL environments. Our method reliably simulates the first 18 years of human life by interpolating from individual short-horizon data samples.
1
2
289
the FCC recently banned the importation of foreign humanoids. but you won't notice it until 2028 when GPT-7 chooses Vancouver to build warehouses to RL tens of thousands of robots IRL. Its proximity to SF and ability to harness Indigenous land titles to override local zoning regulations will make it the final frontier for human labor
7
312
36
361
35
3,012
86,300