Founding member @theworldlabs I fight the devil in the details 🧐
San Francisco, CA
Joined March 2017
- Tweets526
- Following491
- Followers2.7K
- Likes2.3K
Pinned Tweet
We cooked a new model at @theworldlabs! Atlas, our next-generation world model, is out today and I am so excited to finally talk about this one! 🔥🌍
The main idea and unique feature of Atlas is camera-conditioned sparse frame generation: give it a one or more images and the exact camera poses you want, and it generates frames at those poses, consistent in 3D with everything it has seen. You can do also many things with this model, like
(1) Sparse reconstruction: even 2-3 photos of an apartment or warehouse are enough for making a "digital twin".
(2) Keyframing for film-making: place a handful of frames in 3D space and direct a 1 minute video at 1440p through them. You can frame every shot yourself instead of pulling the lever of a slot machine.
(3) Bullet time (my favorite!): You can freeze the time mid confetti explosion and fly the camera through it, from footage of 3-5 cell phones on tripods. The Matrix needed a warehouse of cameras for that shot, now it fits in a backpack.
Because every task is just a sequence of frames + cameras, we trained Atlas on a big mix of tasks and data. I personally drew a lot of satisfaction in pouring SO MUCH DATA into this big boy. Due to task generality, Atlas can also do text-to-image, 360 panoramic generation, outpainting, depth estimation, etc. Only my teammates know how many bernese mountain dogs I have generated with this model in text-to-image mode. XD
A huge shoutout to the entire team at World Labs, I feel so lucky to work with such talented people and have made long-lasting friendships being in the trenches together, day in and day out - @KeunhongP @_mbanani @BhamidipatiPan1 @_nikunj_gupta @gowthami_s @rmgarg02 @vibhaa14 @wendlerch @hyl0m0rphism @DrTunnels @bardienus and many more (not all are on X).
Blog: worldlabs.ai/blog/atlas
See thread below for details 🧵⬇️
Karan Desai (KD) retweeted
Meet Mistral Large 4, aka Le Chonk.
• 1T parameters, natively multimodal. 49B active.
It is the best open weights model from US or Europe on aggregated benchmarks.
• State-of-the-art on critical workloads, including cyber defense, manufacturing and finance and it surpasses closed frontier models on visual grounding.
• Forged in Europe end-to-end and is deployable from Europe via our own Mistral Cloud infrastructure.
• Available to all via API today. Working with cybersecurity partners privately.
Open weights release end of October.
🤖 Made with AI
My friends at @theworldlabs know the rule: if I like your prompt, I steal it and add Bernese mountain dogs. @BenMildenhall's Atlas montage had 198 prompts and zero dogs. Fixed it now. 🐶
Karan Desai (KD) retweeted
Atlas is also a great text-to-image model.
Under our unified formulation, T2I is just single-frame generation. The same operation as everything else Atlas does, just with no observed frames to condition on.
Every frame in this video was generated by Atlas.
I love the text-to-image capabilities of Atlas! I didn't have prior experience in image generation when I joined @theworldlabs, I am glad I learned how to train a model for strong image generation capabilities while working on Atlas. Pretty pixels! 🎨
Karan Desai (KD) retweeted
Atlas excels at handling challenging camera trajectories. With just two input images, you can circle the wreck and explore the sunken ship 👀🚢
Bullet time ability of Atlas!
worldlabs.ai/blog/atlas
Atlas can control space and time to create "bullet time" videos.
Having worked on dynamic 3D reconstruction during my Ph.D, it's shocking how easy this is with Atlas.
Our interns @DrTunnels @HaoZhang623 @davidpantera_ @ZeCh452318 had a lot of fun smashing things.
I have worked with @jcjohnss and @_mbanani since 2019 at the University of Michigan, and everyday I work with them at @theworldlabs. We trust each other, and we know how to work effectively with each other. This kind of trust and camaraderie matters a lot when you're working on technically hard problems. Seven years and counting, I work with them and feel grateful for many more people I met along the way so far. A truly rewarding experience :)
Karan Desai (KD) retweeted
Atlas understands both the spatial structure of the world and how it evolves over time.
This lets us turn a handful of ordinary cameras into a "bullet time" multiview capture studio, capturing dynamic moments frozen in time.
It was Atlas :)
nitter.cf/kdexd/status/209208994…
Karan Desai (KD) retweeted
🧐A question I've long been interested in: how can we learn from human hands and transfer that directly to robots?
Our new work, HUG, makes it possible in three simple steps: (1) collect human grasps at scale, (2) learn from them, and (3) retarget for deployment.
Karan Desai (KD) retweeted
60 million Gaussian splats. One massive dark fantasy world ready to explore! ⚔️
Created entirely with Marble, this persistent world is brought to life in-browser via our Spark 2.0 LoD system and Three.js
Fly through it yourself and learn more about how it was made 👇
Karan Desai (KD) retweeted
We're excited to be rolling out two model updates today!
Marble 1.1: Improves lighting and contrast, with a major reduction in visual artifacts.
Marble 1.1-Plus: Our new model built for scale. Create larger, more complex environments than ever before.
Karan Desai (KD) retweeted
Big congrats to the Open Art team on their new launch. Powered by the World Labs API. 🌎
Today, we’re launching a new way to create with AI.
With OpenArt Worlds, you can generate a fully navigable 3D environment from a single prompt or image, step inside it, and capture shots exactly the way you envision them.
No more starting over.
No more inconsistent scenes.
You build the world once - and create inside it.
• Move through your scene freely
• Find your angles
• Add characters and elements
• Capture production-ready shots
It's time to keep building and marching forward 🙏
World Labs has raised $1 billion in new funding. We are grateful and excited to partner with our investors, including AMD, Autodesk, Emerson Collective, Fidelity Management & Research Company, NVIDIA, and Sea, among others.
worldlabs.ai/blog/funding-20…
My friend @KeunhongP asked me to buy a domain of my name before he buys it first and hoards it forever, so now my personal webpage is karandes.ai -- glad this domain was still available 😀
Karan Desai (KD) retweeted
Introducing Any4D, a unified transformer for fully feed-forward, dense, metric-scale 4D reconstruction from flexible inputs!
Any4D regresses per-pixel motion + geometry across frames in one pass — 15× faster, 2–3× more accurate reconstructions ⚡📈
Details + code below 👇
Exciting collab with @Nik__V__ @YuchenZhan54250 Tanisha Gupta @akashshrm02 @smash0190 @RamananDeva
We cooked something new at @theworldlabs! I am so excited to share our first product with everyone! Create, edit, and export large and persistent worlds now, signup at marble.worldlabs.ai
Introducing Marble by World Labs: a foundation for a spatially intelligent future.
Create your world at marble.worldlabs.ai
We cooked some Real Time Frame Models at @theworldlabs! Try them out :)