MTS @openai, author of the entire post-training RL infra, core contributor of ChatGPT/GPT4/GPT4o etc. 30U30
San Francisco, CA
Joined June 2014
- Tweets23
- Following190
- Followers12.8K
- Likes13
Pinned Tweet
Codex grew programmatic policies with no neural nets: max score on Breakout, and SOTA-level scores on MuJoCo.
Maybe heuristics were not too weak. Maybe they were just too expensive to maintain. Maybe it's the next paradigm.
trinkle23897.github.io/learn…
Very exciting to see the cool result! Same pattern, different physics: Codex-grown heuristics matching or beating DRL agents in fluid dynamics, while staying readable, maintainable, and transferable.
Heuristics were not dead. They were under-maintained.
1/5 TLDR; We used Codex to discover and maintain heuristic learning for hard fluid dynamics control cases.
I’ve been applying DRL and GNN to physics since 2019,, and over the past 3 months I’ve been toying with the idea of using agents in our processes. Inspired by the blog post from @Trinkle23897, I decided to use the same strategy and have agents find readable control strategies. This means a lot to our field, where interpretability can be key for industry.
One small subset for ImageNet, one giant leap for Heuristic Learning. A clean dunk on my “pure-code HL hits ImageNet wall” take.
This isn’t “no learning.” It’s learning where the representation lives in code instead of weights.
This quoted post is unavailable.
Codex iterated a pure NumPy + cv2 closed-loop heuristic policy for VizDoom D3 Battle. No neural network training, no map, no object coordinates, no seed-specific routes. Just screen pixels plus public game variables, roughly the same signals a human player gets. It works surprisingly well.
Notes and videos are now in the blog: trinkle23897.github.io/learn…
Codex grew programmatic policies with no neural nets: max score on Breakout, and SOTA-level scores on MuJoCo.
Maybe heuristics were not too weak. Maybe they were just too expensive to maintain. Maybe it's the next paradigm.
trinkle23897.github.io/learn…
As GPT-5 launches today, it's hard to forget the first ChatGPT-4 model called 0915-gpt4 internally at 2022. @shengjia_zhao did RM, I did PPO and deployed it that Friday for @johnschulman2 to test. First prompt was tic-tac-toe, and it played surprisingly well. Time flies!
Harmony format is finally open-sourced. I still remember 3 years ago (before ChatGPT release) @shengjia_zhao, Daniel and I were brainstorming about the right abstraction for RL training, and that is the start point of the entire harmony library. github.com/openai/harmony
Finally... OAI internally talked about releasing open-source model since 2022 and we got close a few times since then. Now it is.
Our open models are here.
Both of them.
openai.com/open-models
Jiayi Weng retweeted
Hello Twitter! I wrote a ChatGPT app to help remote teams stay in the loop. Basically it will summarize your team’s GitHub activities and send them to Slack.
Check it out! summarizely.org/ (First 100 sign-ups will get 1-year free trial)
#ChatGPT #AI #OpenAI #remotework