๐Ÿ…ป๐Ÿ…ธ๐Ÿ†… (@ollywogs) header
๐Ÿ…ป๐Ÿ…ธ๐Ÿ†… (@ollywogs) profile picture

I make things go beep boop beep | creativity and machine learning | @hseas @medialab @MIT_CSAIL @idmMIT

Cambridge, MA
Joined January 2020
Photo by ๐Ÿ…ป๐Ÿ…ธ๐Ÿ†… (@ollywogs)Photo by ๐Ÿ…ป๐Ÿ…ธ๐Ÿ†… (@ollywogs)Photo by ๐Ÿ…ป๐Ÿ…ธ๐Ÿ†… (@ollywogs)Photo by ๐Ÿ…ป๐Ÿ…ธ๐Ÿ†… (@ollywogs)Photo by ๐Ÿ…ป๐Ÿ…ธ๐Ÿ†… (@ollywogs)Photo by ๐Ÿ…ป๐Ÿ…ธ๐Ÿ†… (@ollywogs)Photo by ๐Ÿ…ป๐Ÿ…ธ๐Ÿ†… (@ollywogs)Photo by ๐Ÿ…ป๐Ÿ…ธ๐Ÿ†… (@ollywogs)Photo by ๐Ÿ…ป๐Ÿ…ธ๐Ÿ†… (@ollywogs)Photo by ๐Ÿ…ป๐Ÿ…ธ๐Ÿ†… (@ollywogs)
๐Ÿ…ป๐Ÿ…ธ๐Ÿ†… retweeted
HOLY MOTHER OF MATHEMATICS!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Google is working on a math-focused variant of its DeepThink model, and its raw thoughts are pretty funny.
400
783
312
14,372
1,485,636
๐Ÿ…ป๐Ÿ…ธ๐Ÿ†… retweeted
Introducing Gemma Scope 2 ๐Ÿค—Largest open release of interpretability tools (over 1 trillion parameters trained!) ๐Ÿ”ฌWorks as a microscope to analyze all Gemma 3 models' internal activations ๐Ÿ—ฃ๏ธAdvanced tools for analyzing chat behaviors
22
89
19
942
78,443
๐Ÿ…ป๐Ÿ…ธ๐Ÿ†… retweeted
I pulled a random street from Google Maps. Turned it into a 3D world with World Labs. Dropped a Unitree G1 into it and hooked it up to our server-side MuJoCo setup. Now the G1 is actually walking around a real Malaysian street in my browser. Three.js is doing the rendering. MuJoCo is running the physics and walking policy on the server. A simple WebSocket keeps everything locked together. Real street โ†’ 3D world โ†’ server physics + policy โ†’ live control in the browser. Kinda feels like cheating. Physical AI is getting wild. Huge shoutout to @theworldlabs. Marble makes this stuff way too fun. #robotics #MuJoCo #WorldLabs #simulation #unitree #sim2real
59
308
31
2,726
167,201
๐Ÿ…ป๐Ÿ…ธ๐Ÿ†… retweeted
Made a site comparing the sizes of living things :) The great Julius Csotonyi spent 5 months painting over 60 illustrations for the site, no ai used
321
5,946
246
50,282
1,095,656
๐Ÿ…ป๐Ÿ…ธ๐Ÿ†… retweeted
turns out it works with paintings too ๐Ÿคฏ
adding camera control to the list of things Qwen Image Edit is great at + with a specialized multi-angle LoRA it's even betterโœจ > rotate the camera > tilt between birdโ€™s-eye and wormโ€™s-eye views > adjust lens (wide, close-up) of course we built a demo for it ๐Ÿค๐Ÿ“น
24
108
12
1,289
130,647
๐Ÿ…ป๐Ÿ…ธ๐Ÿ†… retweeted
omg.. this cant be real Chinaโ€™s 4DV AI just dropped 4D Gaussian Splatting, you can turn 2D video into 4D with sound.. imagine.. we will be able to change camera angle, zoom in/out while watching movies 5 examples:
704
3,681
862
35,004
3,617,468
๐Ÿ…ป๐Ÿ…ธ๐Ÿ†… retweeted
Mapping the Lickability of the Periodic Table
448
4,624
853
38,460
4,259,932
๐Ÿ…ป๐Ÿ…ธ๐Ÿ†… retweeted
Crazy to see how far AI video has come in the last few months The handling of shadows and light here feels like a new standard ๐Ÿ‘‡ (from @endlesstaverns, made with Runway Gen-3 + Suno)
14
114
34
604
252,041
๐Ÿ…ป๐Ÿ…ธ๐Ÿ†… retweeted
3D Gaussian is great, but how can you interact with it ๐ŸŒน๐Ÿ‘‹? Introducing #PhysDreamer: Create your own realistic interactive 3D assets from only static images! Discover how we do this below๐Ÿ‘‡ ๐Ÿงต1/: Website: physdreamer.github.io/
13
79
18
379
61,080
๐Ÿ…ป๐Ÿ…ธ๐Ÿ†… retweeted
Announcing ๐•๐จ๐ข๐œ๐ž๐‚๐ซ๐š๐Ÿ๐ญ๐Ÿช„ SotA for both speech editing and zero-shot text-to-speech, Outperforming VALL-E, XTTS-v2, etc. VoiceCraft works on in-the-wild data such as movies, random videos and podcasts We fully open source it at tinyurl.com/3r8fvyzk
28
132
33
663
160,500
๐Ÿ…ป๐Ÿ…ธ๐Ÿ†… retweeted
I am really excited to reveal what @GoogleDeepMind's Open Endedness Team has been up to ๐Ÿš€. We introduce Genie ๐Ÿงž, a foundation world model trained exclusively from Internet videos that can generate an endless variety of action-controllable 2D worlds given image prompts.
129
531
265
2,388
833,738
๐Ÿ…ป๐Ÿ…ธ๐Ÿ†… retweeted
In our new preprint, we ask: Do multilingual LLMs trained mostly on English use English as an โ€œinternal languageโ€? - A key question for understanding how LLMs function. โ€œDo Llamas Work in English? On the Latent Language of Multilingual Transformersโ€ arxiv.org/abs/2402.10588
12
112
33
636
115,311
๐Ÿ…ป๐Ÿ…ธ๐Ÿ†… retweeted
Aya model is a new open-source massively multilingual language model. It was instruction fine-tuned by people from all over the world through one year! Aya is the unique model supporting 101 languages! It's the next step in building truly multilingual models. 1/10
1
4
1
18
1,826
๐Ÿ…ป๐Ÿ…ธ๐Ÿ†… retweeted
Great week for ML and multilingualism! - French with CroissantLLM hf.co/posts/manu/47030765853โ€ฆ - Basque with Latxa nitter.cf/Hitz_zentroa/status/17โ€ฆ - Chinese with Qwen 1.5 hf.co/collections/Qwen/qwen1โ€ฆ - Dutch with GEITje hf.co/posts/BramVanroy/67922โ€ฆ
We are happy to present the Latxa family of open language models, which includes the largest and best model for Basque. It is based on @Meta's Llama models ranging between 7-70 billion parameters, with the open license Llama-2.
2
15
80
12,376
๐Ÿ…ป๐Ÿ…ธ๐Ÿ†… retweeted
With collaborators @Google we're announcing ๐Ÿ’ซ ZipLora ๐Ÿ’ซ! Merging LoRAs has been a big thing in the community, but tuning can be an onerous process. ZipLora allows us to easily combine any subject LoRA with any style LoRA! Easy to reimplement ๐Ÿฅณ link: ziplora.github.io/
30
217
35
1,250
366,195
๐Ÿ…ป๐Ÿ…ธ๐Ÿ†… retweeted
TL;DR: We use SDS from text-to-video models to animate vector-graphics sketches! Please check our project page for more details, and more penguins: livesketch.github.io/
27
234
36
1,118
174,539
๐Ÿ…ป๐Ÿ…ธ๐Ÿ†… retweeted
Stop using text-to-image. this blender + real-time latent consistency workflow is way more fun, and shows how you can use Generative AI collaboratively, instead of as a creative slot machine try it yourself here huggingface.co/spaces/radameโ€ฆ
11
60
8
379
68,048
๐Ÿ…ป๐Ÿ…ธ๐Ÿ†… retweeted
Enjoy:
9
41
31
301
59,963
๐Ÿ…ป๐Ÿ…ธ๐Ÿ†… retweeted
The fact that most individual neurons are uninterpretable presents a serious roadblock to a mechanistic understanding of language models. We demonstrate a method for decomposing groups of neurons into interpretable features with the potential to move past that roadblock.
114
976
285
5,649
1,750,119
OpenAI's GPT-4 details have apparently been leaked! Looks very detailed and I suspect it's the real deal - given all I know about how these systems work. Here is a summary (extractive+abstractive) I made based on the original thread (see bottom of the post) + some additional pointers from me. (Disclaimer there are probably errors here) ---ARCH--- * GPT-4 is more than 10x the size of GPT-3 (175 B). We believe it has a total of ~1.8 trillion parameters across 120 layers.ย Mixture Of Experts (16 experts, each ~111B). Not a dense transformer like e.g. PaLM (or GPT-3). They use MQA instead of MHA (classic at this point). * Each forward pass (generation of 1 token) only utilizes ~280B parameters and ~560 TFLOPs. This contrasts with the ~1.8 trillion parameters and ~3,700 TFLOP that would be required per forward pass of a purely dense model. ---DISTRIBUTED--- * To parallelize across all their A100s GPUs They utilized 8-way tensor parallelism. Beyond that, they are using 15-way pipeline parallelism. Also apparently they used DeepSpeed ZeRo Stage 1 or block-level FSDP. (You can check out my video on all of these strategies here: youtube.com/watch?v=hc0u4avAโ€ฆ 3D parallelism is what you're looking for & ZeRo) ---VISION--- They have a separate vision encoder from the text encoder, with cross-attention. The architecture is similar to Google DeepMind's Flamingo (I used to work on this project :) ). This adds more parameters on top of the 1.8T of GPT-4. It is fine-tuned with another ~2 trillion tokens, after the text-only pre-training. ---DATA--- * Trained on ~13T tokens (multiple epochs, not unique). Plus millions of rows of instruction fine-tuning data from ScaleAI & internally (I guess acquired through ChatGPT + their API before they changed the policy). * 8k context length for the pre-training phase. The 32k seqlen version of GPT-4 is based on fine-tuning of the 8k after the pre-training. See e.g. MosaicML's blog on how to achieve this: github.com/mosaicml/llm-founโ€ฆ) ---COST--- * OpenAIโ€™s training FLOPS for GPT-4 is ~2.15e25, on ~25,000 A100s for 90 to 100 days at about 32% to 36% MFU. Part of this extremely low utilization is due to an absurd number of failures requiring checkpoints that needed to be restarted from. If their cost in the cloud was about $1 per A100 hour, the training costs for this run alone would be about $63 million. (Today, the pre-training could be done with ~8,192 H100 in ~55 days for $21.5 million at $2 per H100 hour) ---INFERENCE--- OpenAI might be using speculative decoding on GPT-4's inference. See this paper: arxiv.org/abs/2211.17192 The inference runs on a cluster of 128 GPUs. There are multiple of these clusters in multiple datacenters in different locations (it'll be hard for Elizier to nuke these xD). 8-way tensor parallelism and 16-way pipeline parallelism. Original thread: archive.is/2RQ8X (strictly speaking the original one has been removed).
19
186
22
994
330,278