ml @pytorch model optimization, distributed training.
San Francisco, CA
Joined May 2012
- Tweets56
- Following1K
- Followers398
- Likes652
Hamid Shojanazeri retweeted
Reflection AI’s Beam 1.0 is out today, a 501B MoE with 23B active under Apache 2.0. We ran a brief RL post-training run on it in AC2 that tripled its score on a challenging long-horizon agentic coding task in 40 steps, 0.07 to 0.215.
We’re offering day 0 support in AC2 for inference and post-training.
Beam is now available across platforms, with open weights under the Apache 2.0 license - in collaboration with an ecosystem of 40+ partners.
We're releasing:
- Weights, tokenizer, and reference implementation
- BF16, FP8 and NVFP4 weights and a DFlash2 drafter
- The open source harness we built and trained Beam on: Mirror CLI
Weights and model card on Hugging Face: huggingface.co/reflection
Hamid Shojanazeri retweeted
LIVE STREAM in minutes! Hear from the Reflection team about how they built Beam.
Stream: youtube.com/watch?v=dGhUFqeI…
Beam is now available across platforms, with open weights under the Apache 2.0 license - in collaboration with an ecosystem of 40+ partners.
We're releasing:
- Weights, tokenizer, and reference implementation
- BF16, FP8 and NVFP4 weights and a DFlash2 drafter
- The open source harness we built and trained Beam on: Mirror CLI
Weights and model card on Hugging Face: huggingface.co/reflection
Here is a quick demo on Llama 4 long context!
code can be found here! github.com/meta-llama/llama-…
Llama 4 now supports 10M context length — yes, 10 million tokens!
We just ran a 900k-token GitHub repo through it and generated a guide in under 3 minutes. This can unlock many applications!
Checkout the code: github.com/meta-llama/llama-…
Hamid Shojanazeri retweeted
Exciting times for "doing actual RL" with Tulu3 and likely Llama 3.3! Heading to #NeurIPS to present VinePPO at MATHAI, where we built the right RL framework for LLM reasoning. Let’s chat about
- RL training of LLMs
- Learning to search
- Test-time scaling
DM me if interested!
Hamid Shojanazeri retweeted
Llama 3.2 Multimodal benchmarks
MMMU for 11B 60.3 vs Claude Haiku 50.2
MMMU for 90B 60.3 vs GPT 4o mini 59.4
90B looks extremely powerful!
🚀 We’re live!
The next-gen image reasoning capabilities with powerful 11B & 90B models, plus sleek on-device 1B & 3B SOTA models!
Explore how to use the Llama 3.2 models now 👇
📄 Llama Vision Guide: llama.com/docs/how-to-guides…
📲 Llama Stack Apps: github.com/meta-llama/llama-…
📣 Introducing Llama 3.2: Lightweight models for edge devices, vision models and more!
What’s new?
• Llama 3.2 1B & 3B models deliver state-of-the-art capabilities for their class for several on-device use cases — with support for @Arm, @MediaTek & @Qualcomm on day one.
• Llama 3.2 11B & 90B vision models deliver performance competitive with leading closed models — and can be used as drop-in replacements for Llama 3.1 8B & 70B.
• New Llama Guard models to support multimodal use cases and edge deployments.
• The first official distro of Llama Stack simplifies and supercharges the way developers & enterprises can build around Llama to support agentic applications and more.
Details in the full announcement ➡️ go.fb.me/229ug4
Download Llama 3.2 models ➡️ go.fb.me/w63yfd
These models are available to download now directly from Meta and @HuggingFace — and will be available across offerings from 25+ partners that are rolling out starting today, including @accenture, @awscloud, @AMD, @azure, @Databricks, @Dell, @Deloitte, @FireworksAI_HQ, @GoogleCloud, @GroqInc, @IBMwatsonx, @Infosys, @Intel, @kaggle, @NVIDIA, @OracleCloud, @PwC, @scale_AI, @snowflakeDB, @togethercompute and more.
With Llama 3.2 we’re making it possible to run Llama in even more places, with even more flexible capabilities. We’ve said it before and we’ll say it again: open source AI is how we ensure that these innovations reflect the global community they’re built for and benefit everyone. We’re continuing our drive to make open source the standard with Llama 3.2.
🔥 New Fine-tuning Recipes are here! 🔧
⚡ TorchTune: github.com/pytorch/torchtune…
📄 Llama Recipes: github.com/meta-llama/llama-…
Drop your feedback on the repos!
Big thanks to @huggingface, @vllm_project, and all our incredible launch partners! 🙌
It was so inspiring to see the enthusiasm surrounding #Llama3, thanks everyone for hacking with us over the weekend in #Llama3Hackathon! so many cool projects!
nitter.cf/hashtag/Llama3Hackatho…
The focus on #agentic workflows was a positive indicator of the community's organic direction, unlocking numerous opportunities!
Thanks to all the partners! @cerebral_valley @awscloud,@GroqInc,@wandb,@LambdaAPI,
@huggingface,@llama_index, @ollama,@brave,
@OctoAICloud
Hamid Shojanazeri retweeted
Super excited to be releasing AlphaFold 3 today, developed by @IsomorphicLabs and @GoogleDeepMind: our next generation AI model for predicting the biomolecular structures and interactions of proteins, DNA, RNA, small molecules, and more: bit.ly/44yfaCw
1/
Hamid Shojanazeri retweeted
While wrapping up my FSDP + QLora Llama 3 blog, I noticed that ~80 samples (stacked to 3k sequence length) are enough for Llama 3 70B to "converge" to the instruction templates. Thats impressive!🤯
If you are using fine-tuned GPT-3.5 models, you might want to look into Llama 3 70B! 🚀
Hamid Shojanazeri retweeted
Easily Fine-tune @AIatMeta Llama 3 70B! 🦙 I am excited to share a new guide on how to fine-tune Llama 3 70B with @PyTorch FSDP, Q-Lora, and Flash Attention 2 (SDPA) using @huggingface build for consumer-size GPUs (4x 24GB). 🚀
Blog: philschmid.de/fsdp-qlora-lla…
The blog covers:
👨💻 Instructions to set up the environment & preprocess the dataset
🥇 Uses @PyTorch FSDP, Q-LoRA, and Flash Attention v2 (SDPA) for efficient distributed training
🧠 Build with @huggingface TRL, Transformers, PEFT, and Datasets
💾 Tested on @nvidia H100 and A10G GPUs (4x 24GB)
🚀 Can be adjusted to other open LLMs, like Mixtral
What are you building today? 🤗
Hamid Shojanazeri retweeted
Doing a back-of-the-envelope calculation, a 7B Llama 2 model costs about $760,000 to pretrain!
And this assumes you get everything right from the get-go: no hparam tuning, no debugging, no crashes and restarting. The real cost, including experimentation, is probably millions!
It makes me appreciate all these openly available models!
Math:
- The total number of GPU hours needed is 184,320 hours.
- The cost of running one A100 instance per hour is approximately $33.
- Each instance has 8 A100 GPUs.
That's 184320 / 8 * 33 = $760,000
Get out of the box acceleration and memory saving for @huggingface decoder models with @PyTorch 2.0 and Accelerated Transformers with one line of code: model = model.to_bettertransformer() !
Read more in this blog post.pytorch.org/blog/out-of-the-…
Some of speedups and memory savings:
Memory footprint savings on GPU during training range from 20% to 110%+.
Speedups during training range from 10% to 70%.
Speedups during inference range from 5% to 20%.
Interested to contribute the next model support? Please start here.huggingface.co/docs/optimum/…
Out-of-the-box performance (up to 50% speedup on inference) of Stable Diffusion on @huggingface Diffusers with @PyTorch 2.0.
Using the accelerated scaled dot-product attention in PyTorch and torch.compile(), we achieved up to 50% inference speed up across different hardwares
No code changes required to take advantage of scaled dot-product attention, because diffusers uses it when it's available. You can also compile the UNet for an additional performance boost:
`pipe.unet = torch.compile(pipe.unet)`
It works best on newer GPUs.