Come visit our CVPR Booth (@sudo_robotics #638, next to Waymo) to challenge our robot with your own items!
#sudo – at Denver, CO
6
457
Chao Xu retweeted
A first look at what we’re building at Sudo AI. Introducing #sudo R1.
We train robots entirely in simulation, then deploy them directly to hardware - no data collection, per-object tuning, or calibration.
In this first release shared in our blog, #sudo R1 runs 60 minutes uncut and reliably picks objects it has never seen before: glass, fabric, metal, irregular shapes - under changing lighting, clutter, and physical interference.
Simulation-trained systems rarely hold up in practice. We think that’s starting to change.
If you’re building in robotics or thinking about where this is going, follow along at sudo.ai
Chao Xu retweeted
Yesterday the hyped Genesis simulator released. But it's up to 10x slower than existing GPU sims, not 10-80x faster or 430,000x faster than realtime since they benchmark mostly static environments
blog post with corrected open source benchmarks & details: stoneztao.substack.com/p/the…
Two years ago today, we had DreamFusion. Since then, the field has advanced rapidly, and it’s been great to take this wild ride. At #ECCV2024 , I will present Sparp onsite with a short oral pre on CV4Metaverse today, a poster on Wild3D tomorrow, and a main conf poster on Friday.
Single-image-to-3D is ill-posed. What if we had a few more views? We introduce 𝗦𝗽𝗮𝗥𝗣, a framework that understands spatial relationships between 𝘂𝗻𝗽𝗼𝘀𝗲𝗱 views to predict camera poses and reconstruct the underlying 3D objects. #ECCV2024
Project: chaoxu.xyz/sparp
Thanks to @_akhaliq for featuring our #ECCV paper!
🌋Project: chaoxu.xyz/sparp
🤗Demo: huggingface.co/spaces/sudo-a…
A quick intro to the paper in my pinned post: nitter.cf/_ChaoXu/status/1825986…
#AIGC #GenAI #Imageto3D #Hillbot #sudoAI
SpaRP
Fast 3D Object Reconstruction and Pose Estimation from Sparse Views
discuss: huggingface.co/papers/2408.1…
Open-world 3D generation has recently attracted considerable attention. While many single-image-to-3D methods have yielded visually appealing outcomes, they often lack sufficient controllability and tend to produce hallucinated regions that may not align with users' expectations. In this paper, we explore an important scenario in which the input consists of one or a few unposed 2D images of a single object, with little or no overlap. We propose a novel method, SpaRP, to reconstruct a 3D textured mesh and estimate the relative camera poses for these sparse-view images. SpaRP distills knowledge from 2D diffusion models and finetunes them to implicitly deduce the 3D spatial relationships between the sparse views. The diffusion model is trained to jointly predict surrogate representations for camera poses and multi-view images of the object under known poses, integrating all information from the input sparse views. These predictions are then leveraged to accomplish 3D reconstruction and pose estimation, and the reconstructed 3D model can be used to further refine the camera poses of input views. Through extensive experiments on three datasets, we demonstrate that our method not only significantly outperforms baseline methods in terms of 3D reconstruction quality and pose prediction accuracy but also exhibits strong efficiency. It requires only about 20 seconds to produce a textured mesh and camera poses for the input views.
Single-image-to-3D is ill-posed. What if we had a few more views? We introduce 𝗦𝗽𝗮𝗥𝗣, a framework that understands spatial relationships between 𝘂𝗻𝗽𝗼𝘀𝗲𝗱 views to predict camera poses and reconstruct the underlying 3D objects. #ECCV2024
Project: chaoxu.xyz/sparp
SpaRP generalizes well in the real world, accurately predicting camera poses and reconstructing 3D objects effectively from both Amazon product images and real-life photos captured on mobile devices.
1
1
249
I am honored to work on this project with Ang Li, @LinghaoChen97, @yulinnliu07, @eliphatfs, @haosu_twitr, @MinghuaLiu_.
2
207
✈️En route to #NeurIPS2023
Happy to chat about anything 3D🍻
Poster (One-2-3-45): TUE 10:45 AM - 12:45 PM
3
434
Thanks for sharing! 🚀 Our latest project is a cutting-edge framework born from the embrace of the multiview-based feed-forward paradigm in One-2-3-45 for Image-to-3D. Upgraded multiview and 3D recon modules. A perfect fusion of speed and quality – the best of both worlds! #AIGC
One-2-3-45++: Fast Single Image to 3D Objects with Consistent Multi-View Generation and 3D Diffusion
paper page: huggingface.co/papers/2311.0…
Recent advancements in open-world 3D object generation have been remarkable, with image-to-3D methods offering superior fine-grained control over their text-to-3D counterparts. However, most existing models fall short in simultaneously providing rapid generation speeds and high fidelity to input images - two features essential for practical applications. In this paper, we present One-2-3-45++, an innovative method that transforms a single image into a detailed 3D textured mesh in approximately one minute. Our approach aims to fully harness the extensive knowledge embedded in 2D diffusion models and priors from valuable yet limited 3D data. This is achieved by initially finetuning a 2D diffusion model for consistent multi-view image generation, followed by elevating these images to 3D with the aid of multi-view conditioned 3D native diffusion models. Extensive experimental evaluations demonstrate that our method can produce high-quality, diverse 3D assets that closely mirror the original input image.
Project page: sudo-ai-3d.github.io/One2345…
GitHub: github.com/SUDO-AI-3D/One234…
6
268
Chao Xu retweeted
📢Excited news from sudoAI (@sudoAI_): The interactive demo of our 3D Generative AI model is online! (alpha test for desktop, mobile / pad coming soon)
✨Transform images & text into stunning 3D models in just 60 seconds!🚀
Try it now!
👉sudo.ai
🚀 Excited to release our official Gradio demo for Zero123++! Check it out here: huggingface.co/spaces/sudo-a…
Chao Xu retweeted
Excited to release Zero123++, an image-conditioned diffusion model for generating high-quality, 3D-consistent multi-view images from a single input view!
Paper: arxiv.org/abs/2310.15110
Code: github.com/SUDO-AI-3D/zero12…
🤗Demo: huggingface.co/spaces/sudo-a…
Chao Xu retweeted
Transforming a single image into a consistent set of multi-view images can be the cornerstone of numerous 2D-to-3D applications. Check out our new model, Zero123++, and start crafting your own magic today! github.com/SUDO-AI-3D/zero12… @eliphatfs











