🅻🅸🆅 (@ollywogs) header
🅻🅸🆅 (@ollywogs) profile picture

I make things go beep boop beep | creativity and machine learning | @hseas @medialab @MIT_CSAIL @idmMIT

Cambridge, MA
Joined January 2020
Photo by 🅻🅸🆅 (@ollywogs)Photo by 🅻🅸🆅 (@ollywogs)Photo by 🅻🅸🆅 (@ollywogs)Photo by 🅻🅸🆅 (@ollywogs)Photo by 🅻🅸🆅 (@ollywogs)Photo by 🅻🅸🆅 (@ollywogs)Photo by 🅻🅸🆅 (@ollywogs)Photo by 🅻🅸🆅 (@ollywogs)Photo by 🅻🅸🆅 (@ollywogs)Photo by 🅻🅸🆅 (@ollywogs)
🅻🅸🆅 retweeted
HOLY MOTHER OF MATHEMATICS!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Google is working on a math-focused variant of its DeepThink model, and its raw thoughts are pretty funny.
400
783
312
14,372
1,485,637
Introducing Gemma Scope 2 🤗Largest open release of interpretability tools (over 1 trillion parameters trained!) 🔬Works as a microscope to analyze all Gemma 3 models' internal activations 🗣️Advanced tools for analyzing chat behaviors
22
89
19
942
78,443
🅻🅸🆅 retweeted
I pulled a random street from Google Maps. Turned it into a 3D world with World Labs. Dropped a Unitree G1 into it and hooked it up to our server-side MuJoCo setup. Now the G1 is actually walking around a real Malaysian street in my browser. Three.js is doing the rendering. MuJoCo is running the physics and walking policy on the server. A simple WebSocket keeps everything locked together. Real street → 3D world → server physics + policy → live control in the browser. Kinda feels like cheating. Physical AI is getting wild. Huge shoutout to @theworldlabs. Marble makes this stuff way too fun. #robotics #MuJoCo #WorldLabs #simulation #unitree #sim2real
59
308
31
2,726
167,201
Made a site comparing the sizes of living things :) The great Julius Csotonyi spent 5 months painting over 60 illustrations for the site, no ai used
321
5,946
246
50,282
1,095,656
turns out it works with paintings too 🤯
adding camera control to the list of things Qwen Image Edit is great at + with a specialized multi-angle LoRA it's even better✨ > rotate the camera > tilt between bird’s-eye and worm’s-eye views > adjust lens (wide, close-up) of course we built a demo for it 🤝📹
24
108
12
1,289
130,647
🅻🅸🆅 retweeted
omg.. this cant be real China’s 4DV AI just dropped 4D Gaussian Splatting, you can turn 2D video into 4D with sound.. imagine.. we will be able to change camera angle, zoom in/out while watching movies 5 examples:
704
3,681
862
35,004
3,617,469
Mapping the Lickability of the Periodic Table
448
4,624
853
38,460
4,259,933
Crazy to see how far AI video has come in the last few months The handling of shadows and light here feels like a new standard 👇 (from @endlesstaverns, made with Runway Gen-3 + Suno)
14
114
34
604
252,041
3D Gaussian is great, but how can you interact with it 🌹👋? Introducing #PhysDreamer: Create your own realistic interactive 3D assets from only static images! Discover how we do this below👇 🧵1/: Website: physdreamer.github.io/
13
79
18
379
61,080
🅻🅸🆅 retweeted
Announcing 𝐕𝐨𝐢𝐜𝐞𝐂𝐫𝐚𝐟𝐭🪄 SotA for both speech editing and zero-shot text-to-speech, Outperforming VALL-E, XTTS-v2, etc. VoiceCraft works on in-the-wild data such as movies, random videos and podcasts We fully open source it at tinyurl.com/3r8fvyzk
28
132
33
663
160,500
🅻🅸🆅 retweeted
I am really excited to reveal what @GoogleDeepMind's Open Endedness Team has been up to 🚀. We introduce Genie 🧞, a foundation world model trained exclusively from Internet videos that can generate an endless variety of action-controllable 2D worlds given image prompts.
129
531
265
2,388
833,738
🅻🅸🆅 retweeted
In our new preprint, we ask: Do multilingual LLMs trained mostly on English use English as an “internal language”? - A key question for understanding how LLMs function. “Do Llamas Work in English? On the Latent Language of Multilingual Transformers” arxiv.org/abs/2402.10588
12
112
33
636
115,311
Aya model is a new open-source massively multilingual language model. It was instruction fine-tuned by people from all over the world through one year! Aya is the unique model supporting 101 languages! It's the next step in building truly multilingual models. 1/10
1
4
1
18
1,826
Great week for ML and multilingualism! - French with CroissantLLM hf.co/posts/manu/47030765853… - Basque with Latxa nitter.cf/Hitz_zentroa/status/17… - Chinese with Qwen 1.5 hf.co/collections/Qwen/qwen1… - Dutch with GEITje hf.co/posts/BramVanroy/67922…
We are happy to present the Latxa family of open language models, which includes the largest and best model for Basque. It is based on @Meta's Llama models ranging between 7-70 billion parameters, with the open license Llama-2.
2
15
80
12,376
With collaborators @Google we're announcing 💫 ZipLora 💫! Merging LoRAs has been a big thing in the community, but tuning can be an onerous process. ZipLora allows us to easily combine any subject LoRA with any style LoRA! Easy to reimplement 🥳 link: ziplora.github.io/
30
217
35
1,250
366,195
🅻🅸🆅 retweeted
TL;DR: We use SDS from text-to-video models to animate vector-graphics sketches! Please check our project page for more details, and more penguins: livesketch.github.io/
27
234
36
1,118
174,539
🅻🅸🆅 retweeted
Stop using text-to-image. this blender + real-time latent consistency workflow is way more fun, and shows how you can use Generative AI collaboratively, instead of as a creative slot machine try it yourself here huggingface.co/spaces/radame…
11
60
8
379
68,048
🅻🅸🆅 retweeted
Enjoy:
9
41
31
301
59,963
🅻🅸🆅 retweeted
The fact that most individual neurons are uninterpretable presents a serious roadblock to a mechanistic understanding of language models. We demonstrate a method for decomposing groups of neurons into interpretable features with the potential to move past that roadblock.
114
976
285
5,649
1,750,121
OpenAI's GPT-4 details have apparently been leaked! Looks very detailed and I suspect it's the real deal - given all I know about how these systems work. Here is a summary (extractive+abstractive) I made based on the original thread (see bottom of the post) + some additional pointers from me. (Disclaimer there are probably errors here) ---ARCH--- * GPT-4 is more than 10x the size of GPT-3 (175 B). We believe it has a total of ~1.8 trillion parameters across 120 layers. Mixture Of Experts (16 experts, each ~111B). Not a dense transformer like e.g. PaLM (or GPT-3). They use MQA instead of MHA (classic at this point). * Each forward pass (generation of 1 token) only utilizes ~280B parameters and ~560 TFLOPs. This contrasts with the ~1.8 trillion parameters and ~3,700 TFLOP that would be required per forward pass of a purely dense model. ---DISTRIBUTED--- * To parallelize across all their A100s GPUs They utilized 8-way tensor parallelism. Beyond that, they are using 15-way pipeline parallelism. Also apparently they used DeepSpeed ZeRo Stage 1 or block-level FSDP. (You can check out my video on all of these strategies here: youtube.com/watch?v=hc0u4avA… 3D parallelism is what you're looking for & ZeRo) ---VISION--- They have a separate vision encoder from the text encoder, with cross-attention. The architecture is similar to Google DeepMind's Flamingo (I used to work on this project :) ). This adds more parameters on top of the 1.8T of GPT-4. It is fine-tuned with another ~2 trillion tokens, after the text-only pre-training. ---DATA--- * Trained on ~13T tokens (multiple epochs, not unique). Plus millions of rows of instruction fine-tuning data from ScaleAI & internally (I guess acquired through ChatGPT + their API before they changed the policy). * 8k context length for the pre-training phase. The 32k seqlen version of GPT-4 is based on fine-tuning of the 8k after the pre-training. See e.g. MosaicML's blog on how to achieve this: github.com/mosaicml/llm-foun…) ---COST--- * OpenAI’s training FLOPS for GPT-4 is ~2.15e25, on ~25,000 A100s for 90 to 100 days at about 32% to 36% MFU. Part of this extremely low utilization is due to an absurd number of failures requiring checkpoints that needed to be restarted from. If their cost in the cloud was about $1 per A100 hour, the training costs for this run alone would be about $63 million. (Today, the pre-training could be done with ~8,192 H100 in ~55 days for $21.5 million at $2 per H100 hour) ---INFERENCE--- OpenAI might be using speculative decoding on GPT-4's inference. See this paper: arxiv.org/abs/2211.17192 The inference runs on a cluster of 128 GPUs. There are multiple of these clusters in multiple datacenters in different locations (it'll be hard for Elizier to nuke these xD). 8-way tensor parallelism and 16-way pipeline parallelism. Original thread: archive.is/2RQ8X (strictly speaking the original one has been removed).
19
186
22
994
330,278