research @AnthropicAI | phd from @StanfordAILab🌲| proud @Cal bear 🐻 | taught w @BerkeleyML
San Francisco
Joined November 2014
- Tweets444
- Following1.3K
- Followers2.2K
- Likes10.2K
Rohan Taori retweeted
Introducing Project Glasswing: an urgent initiative to help secure the world’s most critical software.
It’s powered by our newest frontier model, Claude Mythos Preview, which can find software vulnerabilities better than all but the most skilled humans.
anthropic.com/glasswing
Rohan Taori retweeted
A statement on the comments from Secretary of War Pete Hegseth.
anthropic.com/news/statement…
Rohan Taori retweeted
I'll be discussing work on using AI for pharmacovigilance (led by @claudeai 😉 ) as a Spotlight in #Agents4Science today at 11:00am PST!
It was really fun participating in @james_y_zou's experiment on what happens if we let an LM lead a research project agents4science.stanford.edu
Rohan Taori retweeted
It was a dream come true to teach the course I wish existed at the start of my PhD. We built up the algorithmic foundations of modern-day RL, imitation learning, and RLHF, going deeper than the usual "grab bag of tricks". All 25 lectures + 150 pages of notes are now public! 🧵
Rohan Taori retweeted
I think CS336 has one of the best LLM problem sets of any AI/LM class thanks to our incredible TAs (@nelsonfliu,@GabrielPoesia,@marcelroed,@neilbband,@rckpudi).
We're making it so you can do it all at home, and it's one of the best ways to learn LLMs deeply.
Want to learn the engineering details of building state-of-the-art Large Language Models (LLMs)? Not finding much info in @OpenAI’s non-technical reports?
@percyliang and @tatsu_hashimoto are here to help with CS336: Language Modeling from Scratch, now rolling out to YouTube.
Rohan Taori retweeted
Introducing Claude 3.7 Sonnet: our most intelligent model to date. It's a hybrid reasoning model, producing near-instant responses or extended, step-by-step thinking.
One model, two ways to think.
We’re also releasing an agentic coding tool: Claude Code.
Rohan Taori retweeted
Claude 3.5 Haiku is now available on our API, Amazon Bedrock, and Google Cloud's Vertex AI.
Haiku is fast and particularly strong at coding. It outperforms state-of-the-art models—including GPT-4o—on SWE-bench Verified, which measures how models solve real software issues.
ALT An illustration for Claude 3.5 Haiku.
Rohan Taori retweeted
Sahara AI raised $43M to build a collaborative AI economy!
Co-led by @PanteraCapital, @BinanceLabs, and @polychain, this funding brings us closer to a future of open, equitable, and secure AI through our decentralized AI blockchain platform.
We also saw participation from @SamsungNext, @Sequoia, @matrixvc, @daofive, @geek_cartel, @NomadCapital_io, @SCB10X_OFFICIAL, @CanonicalCrypto, @mirana, @ForesightVen, @aegis_ventures, @DispersionVC, @tangent_xyz, @alumniventures, among others.
Moreover, we're proud to have the support from the brightest minds in AI like:
- Laksh Vaaman Sehgal from @MothersonGroup
- Rohan Taori (@rtaori13) from @AnthropicAI & @StanfordAILab
- Teknium (@teknium) from @NousResearch
- Vipul Prakash (@vipulved) from @togethercompute
- Elvis Zhang (@elvzhangg) from @midjourney, and more
Whether you're an AI enthusiast, a data scientist, a business leader, or simply passionate about the potential of technology to change the world, there's a place for you in the collaborative AI economy!
Read more on how you can join our mission:
saharalabs.ai/blog/sahara-ai…
Rohan Taori retweeted
Join us at the co-located <AI / ALL> summit on Aug 15, with the social party in the evening!
lu.ma/mxcx5bia
co-hosted with @SCB10X_OFFICIAL @SambaNovaAI
sponsored by @awscloud
participated by folks @AIatMeta @google @CohereForAI @togethercompute
Arriving in Bangkok for @aclmeeting! 😃
Will be sharing our recent work on logical scaffolding, model uncertainty expression & multi-hop entailment inference w/ folks @nlp_usc + @KaitlynZhou +friends @allen_ai
I'm also helping on the <AI / ALL> summit
w/ @saharaai
👇👇
Rohan Taori retweeted
I’m excited to share a project I’ve been working on for over a year, which I believe will fundamentally change our approach to language models.
We’ve designed a new architecture, which replaces the hidden state of an RNN with a machine learning model. This model compresses context through actual gradient descent on input tokens. We call our method “Test-Time-Training layers.”
TTT layers directly replace attention, and unlock linear complexity architectures with expressive memory, allowing us to train LLMs with millions (someday billions) of tokens in context.
Our instantiations, TTT-Linear and TTT-MLP, both match or beat the strongest Transformers and Mamba. Arxiv: arxiv.org/abs/2407.04620
Rohan Taori retweeted
Introducing Claude 3.5 Sonnet—our most intelligent model yet.
This is the first release in our 3.5 model family.
Sonnet now outperforms competitor models on key evaluations, at twice the speed of Claude 3 Opus and one-fifth the cost.
Try it for free: claude.ai
ALT Benchmark table showing Claude 3.5 Sonnet outperforming (as indicated by green highlights) other AI models on graduate level reasoning, code, multilingual math, reasoning over text, and more evaluations. Models compared include Claude 3 Opus, GPT-4o, Gemini 1.5 Pro, and Llama-400b.
Rohan Taori retweeted
I am really excited to introduce DataComp for Language Models (DCLM), our new testbed for controlled dataset experiments aimed at improving language models. 1/x
Rohan Taori retweeted
2 (very different 😅) recent works!
#1 arxiv.org/pdf/2403.09603 Popular model training services lack user trust due to possible 😈 intent, but 3rd-party auditing suffers w/ GPU nondeterminism. @simran_s_arora, Dan Boneh, and I control this to build a robust verification scheme 🙂
simplicity of DPO but using your reward model 👇 great work!!
In the words of Billy Idol, give a "rebel yell" for REBEL: a strikingly simple RL algorithm (it's just regression! no clipping / critics!) that scales to generative models (both LLMs and Diffusion Models!) and has deep theoretical interpretations: arxiv.org/abs/2404.16767. [1/8]