Anthony Russell (@russler) header
Anthony Russell (@russler) profile picture
Joined May 2007
Photo by Anthony Russell (@russler)Photo by Anthony Russell (@russler)Photo by Anthony Russell (@russler)Photo by Anthony Russell (@russler)Photo by Anthony Russell (@russler)Photo by Anthony Russell (@russler)Photo by Anthony Russell (@russler)Photo by Anthony Russell (@russler)
Anthony Russell retweeted
Launch and return are fundamental techniques for Starship’s fully and rapidly reusable design
938
4,162
517
32,403
3,026,736
Anthony Russell retweeted
Intel just dropped the most powerful 7B model ever! - Top 7B on Huggingface's leaderboard. TL;DR: - Base model: Mistral 7B. - Fine Tuned on: SlimOrca - DPO: LLaMA-13B vs ChatGPT Gens (Prefer ChatGPT) --- - Model: huggingface.co/Intel/neural-… - SFT Dataset: huggingface.co/datasets/Open… - DPO Dataset: huggingface.co/datasets/Inte… - Code: github.com/intel/intel-exten… - License: Apache 2 --- --- Unique Hardware: Trained on Habana Labs's 8x Gaudi2 mezzanine accelerators! I didn't get to try the accelerators myself yet. To the best of my knowledge | Hardware TL;DR: - Comparible in performance to A100. - 96GB on device RAM. Read More: habana.ai/habana-labs-data-d… --- Base model Mistral 7B [1] For those not familiar, mistral is considered to be among the best 7B base models nowadays. > Future deep post about mistral it coming soon. [1] Paper arxiv.org/abs/2310.06825 --- Supervised Fine Tuning (SFT) Trained on SlimOrca [1][2] - SlimOrca is a subset selection of Orca that only includes ~500k GPT-4 completions. - The selection was done using GPT-4 to remove answers which appear wrong based on the orinigal FLAN targets. [1] Orca Paper: arxiv.org/abs/2306.02707 [2] SlimOrca Dataset: huggingface.co/datasets/Open… --- Direct Policy Opimization (DPO) TL;DR DPO [1]: Given a prompt and a pair of completions, train the model to prefer one over the other. We currently witness that most of the most powerful models over the past month were trained with DPO. This model was trained on prompts from SlimOrca's dataset where each has one GPT-4 completion and one LLaMA-13B completion. The model trained to prefer GPT-4 over LLaMA-13B. [1] DPO Paper: arxiv.org/pdf/2305.18290.pdf --- Intel's Codebase To be honest: I was skeptical when I found out about Intel's claims. Usually new hardware is not trivial to work with. But I must give credit where credit is due. Intel fully support Huggingface's trainer for all stages of modern LLM training: - Causal Language Modeling - Supervised Fine Tuning - Reinforcement Learning From Human Feedback (Both PPO and DPO) - And LoRA. Everything has easy and Minimalistic (thank you!!!) code examples and you can run whatever you want with a push of a button. The performance is state-of-the-art as written above. I will try to reproduce their whole pipeline experiment end-2-end over the next few days and report back to you. --- In Addition! Habana's hardware is developed in Israel! 🇮🇱
43
313
28
1,727
467,990
Anthony Russell retweeted
last week we shared how to borrow against real property as an NFT this week is a video demo on how you can go from paper deed to NFT in a couple of minutes of work 📜 --> 🏡 check it out
2
2
12
715
Question for the experts 🙋‍♂️ Will the extraordinary energy requirements of the newest generative AI models help slow their progress, and potentially alleviate safety concerns related to runaway capability? Cc: @Scobleizer @elonmusk @ESYudkowsky @ylecun @karpathy @amasad
10
21
2
129
24,567
The last decade has been a pivotal one in the AI revolution 🤖 Cc: @ylecun @Scobleizer @erikbryn @amcafee @paulg @DavidSacks
54
954
92
3,519
656,775
Sometimes your the frog and sometimes your the fly.
This quoted post is unavailable.
I have now been back in Australia for a year and I am amazed at how bad the @Uber product is here. After years of amazing @GrabSG in Singapore I am amazed at the completed failure of Uber in the market.
Anthony Russell retweeted
AEMO projecting a collapse in residential and commercial gas demand in eastern Australia- falling by 65% by 2040 This is because electrification now far cheaper and more efficient than gas for cooking, heating and hot water. #auspol aemo.com.au/-/media/files/ga…
5
39
5
84
Ad spending per vehicle sold: Genesis: $1,955 Lincoln: $1,553 Jaguar: $1,235 Lexus: $984 Cadillac: $908 Honda: $183 Dodge: $78 Mitsubishi: $43 Tesla: $0 Source: Ad Age (2020 spending) inside.com/business?utm_sour…
Wonderful to see the crowd at the @wallabies V @FranceRugby in Brisbane all be COVID aware and wearing masks. Again all the COViD concerns are from returning Australians. Not those in country.
1
So we’ll written Jason Wilson writing in an opinion piece in ⁦@guardian⁩ - When I look at Australia from the US, I see an insular nation that has turned its back on its own theguardian.com/commentisfre…
Could not come sooner - let’s hope we see more governments following quick on the NHS’s heals on this. theguardian.com/society/2021…
2
Time to geek out and go down a rabbit hole…. @elonmusk and Sandy Munro …… missed this earlier in the year but very interesting. youtu.be/YAtLTLiqNwg
Could not have said it better @WSJ - The World Relies on One Chip Maker in Taiwan, Leaving Everyone Vulnerable wsj.com/articles/the-world-r…
1
Great to see Australia even talking about risk based assessments with Covid. 8:45 into this @abcnews interview. Hopefully they will start allowing more people from Singapore in soon. youtu.be/6PXJpy378Zw
Anthony Russell retweeted
Incredible photo of this piece of paradise this winter, shared by the North Bondi RSL. Happy Weekend 😍🐳🌊
4
9
3
130
We need to consider allowing more immigration from our surrounding neigbours. After living in Asia for many years and seeing the professional nature of the Filipino nurses we should consider allowing more into Australia smh.com.au/national/it-s-hor…