Staff Research Scientist at https://nitter.cf/t.co/WEMkSSRVeZ. Formerly research scientist at Google, postdoc at Stanford, and PhD student at Columbia.
Brooklyn, NY
Joined May 2009
- Tweets55
- Following261
- Followers1.1K
- Likes63
Pinned Tweet
So excited about this work and to be a part of this awesome team @cartesia !!
H-Nets provide a new building block for AI models that enables "chunking" the input in the most appropriate way given the context. This enables training language models directly on bytes, thereby eliminating the need for a tokenizer. It also allows the model to dynamically allocate compute to the most difficult to understand/predict parts of the input/output, making the model more efficient.
Really pumped to keep pushing the boundaries of what these models can do, and also to develop new cool algorithms for efficiently training and inferencing these models.
I've always believed that intelligence should come from simple, general building blocks that learn their own abstractions directly from raw data; in other words, the bitter lesson. I think we're finally seeing a real step towards AI that can learn from all data.
Language modeling on raw bytes is a microcosm of this ideal, and to me it's a central problem in AI: if we can't build LLMs that learn end-to-end from bytes, how will we handle the rest of the world's data that we understand even less?
Last year we introduced hierarchical networks (H-Nets) as a first step. Our new version finds up to 3-5x efficiency gains in controlled scaling law experiments, and at larger scale matches an open-source model trained on 11T tokens with ~10x less data and compute. The posts below walk through what we found and why we think it works.
Avner May retweeted
I've always believed that intelligence should come from simple, general building blocks that learn their own abstractions directly from raw data; in other words, the bitter lesson. I think we're finally seeing a real step towards AI that can learn from all data.
Language modeling on raw bytes is a microcosm of this ideal, and to me it's a central problem in AI: if we can't build LLMs that learn end-to-end from bytes, how will we handle the rest of the world's data that we understand even less?
Last year we introduced hierarchical networks (H-Nets) as a first step. Our new version finds up to 3-5x efficiency gains in controlled scaling law experiments, and at larger scale matches an open-source model trained on 11T tokens with ~10x less data and compute. The posts below walk through what we found and why we think it works.
Avner May retweeted
We @togethercompute believe intelligence should be abundant, not expensive.
Today we announced our Series C funding of $800m @ $8.3B valuation, to continue to build the world's most efficient platform for generative AI.
Thanks @nikogallogly for telling our story in @nytimes!
shorturl.at/SooOP
I'm at @iclr_conf in Rio, and am excited to be presenting our Speculative Speculative Decoding (SSD) work tomorrow from 10:30 am - 1 pm (Pavilion 3 P3, #1803).
Come by to learn how we're attaining 30% speedup on average over SGLang 🚀
Collab w/ @tanishqkumar07 @tri_dao.
Excited to announce our new LLM inference algorithm, speculative speculative decoding (SSD)!
It is fast 🚀 — up to 2x faster than state-of-the-art inference engines (vLLM, SGLang).
Working on this with @tanishqkumar07 and @tri_dao was a blast.
Details in thread:
Excited to announce our new LLM inference algorithm, speculative speculative decoding (SSD)!
It is fast 🚀 — up to 2x faster than state-of-the-art inference engines (vLLM, SGLang).
Working on this with @tanishqkumar07 and @tri_dao was a blast.
Details in thread:
One thing that excites me about this algorithm is that the large improvement in latency does not come at the cost of lower throughput. This is possible because—unlike other approaches like tree verification—SSD does not require additional compute by the target model. It only increases the compute required for the speculator, which is much faster and cheaper.
To learn more, please take a look at our paper, and feel free to reach out with any questions!
arXiv: arxiv.org/pdf/2603.03251
code: github.com/tanishqkumar/ssd
Avner May retweeted
What if your LLM inference automatically got faster the more you used it?
Introducing ATLAS from the Together AI Turbo research team.
Read more: togetherai.link/atlas
Here’s Together AI Founder and Chief Scientist @tri_dao introducing ATLAS:
Avner May retweeted
Tokenization is just a special case of "chunking" - building low-level data into high-level abstractions - which is in turn fundamental to intelligence.
Our new architecture, which enables hierarchical *dynamic chunking*, is not only tokenizer-free, but simply scales better.
Excited that Together has just released TogetherChat!
TogetherChat is a way for anyone to ask questions to Deepseek-R1 and other leading open-source models, with an awesome UI, with a very similar user experience to using ChatGPT or Claude models. Check it out at chat.together.ai!
Avner May retweeted
🚀 Big news! We’re thrilled to announce the launch of Llama 3.2 Vision Models & Llama Stack on Together AI.
🎉 Free access to Llama 3.2 Vision Model for developers to build and innovate with open source AI. api.together.ai/playground/c…
➡️ Learn more in the blog together.ai/blog/llama-3-2-v…
Avner May retweeted
🙌 We love that Llama has gone multimodal! We're excited to partner with @AIatMeta to offer free access to the Llama 3.2 11B vision model for developers. Can't wait to see what everyone builds!
Try now with our Llama-Vision-Free model endpoint.
Sign up here: api.together.ai/playground/c…