Anton Kovsharov

🪸 [Call for contributors!] 🪸 New release: "Google Docs for Coral Reefs" is live!! You can now explore the reef with your dive buddies, select coral colonies, tag species, add comments, and search for visually similar colonies across the reef, all in 3D! 🔵 Why? Coral reefs are such precious, beautiful, incredibly complex, and threatened ecosystems. They are dying fast. A lot of amazing people are fighting to keep them alive. We need to deploy more solutions faster. That requires a deep understanding of how coral reefs function. What works, what doesn't, to coordinate precise action. The evidence is buried in a humongous amount of data scattered across organisations, hard drives, and disconnected tools. We need one place to share the data, examine it, analyse it, and turn it into action. That’s why we’re building this! 🔵 Ground truth is everything. We start with the highest-signal data: time-series 3D. You see each colony from many angles, at true scale, and track how it evolves over time. But the data is enormous. Even small survey plots are hundreds of gigabytes. We already process and stream it directly in the browser. Now we turn it into a workspace. 🔵 Dive buddies. Annotating hectares of reef is soo lonely. So we built multiplayer! Bring your friends and annotate together in 3D. See where everyone is looking in real time and split up the work. Leave comments at exact locations. Use @ tags for species, health, etc. Pretty much like Google Docs or Figma, but for a 3D reef. 🔵 Smart selection. Corals are painfully hard to annotate! Their shapes and colours are all different, and it's hard to see where one colony ends and another begins. Even amazing 2D models like SAM struggle with reef photos and orthos. Lousy masks and vague point annotations create bad training data. Those errors wreck downstream AI models. Luckily, we have an unfair advantage. Each coral appears in many photos from different angles. Our selector works in a shared 3D embedding space, across every view at once. Just Shift+click, drag out a semantic radius, and select anything directly in 3D. 🔵 Semantic search. Start with a colony, coral fragment, or disease patch. Hit the ≈ button and find visually similar things across the reef. Create a category and add the right matches. Keep going until you’ve mapped it. Then move on to the next one. 🔵 Under the hood. We project DINOv3 features from the photos into the 3D splats. Then we use LDA to adapt them to your categories and make the model lightweight enough to work in the browser. 🔵 Please break it. This demo is one tiny patch of reef, held together with duct tape and full of bugs. But even in this state, no other browser tool lets a team annotate a 3D reef together like this. 🔵 Huge thanks to the superstar Matthew Osborne, creator of The Living Atlas, for leading the development! The fantastic Jan Paulini built our first species list. And to the amazing people doing the real work in the water, who kindly shared this reef data with us: @alderdice_r, @reefgenomics, @emmafcamp, @PaigeStrudwick, @ChristineDRoper, and many others. 🔵 Please annotate a few colonies. It would help us a lot! Together, we map every reef on Earth! 🤗 🌍

Wildflow Coral -- our first product, a comprehensive digital twin of coral reef ecosystems based on multimodal foundation models for biodiversity. 🔵 Why? Billions of years of evolution created Earth's vibrant ecosystems -- they are inconceivably rich and incredibly complex! We have so much to learn from this beautiful complexity. We want to build a future where humans and nature thrive and enrich each other. That’s why we chose to start with coral reefs. They are the most complex, beautiful, endangered, and important ecosystem. They are home to 25% of marine life while occupying only 0.1% of the ocean. Sadly, we’re on track to lose 90% of coral reefs by 2050. The time to act is now. Over half a billion people rely on coral reefs. Already vulnerable communities are the first to suffer the consequences. We have to make a lot of critical decisions about them! They are a key to model entire biosphere (protect nature) down the road. 🔵 How? We take all the data across all modalities, such as 3D photogrammetry, bioacoustics, underwater videos, remote sensing, eDNA, environmental data like currents, and more, to create the ultimate digital twin of any coral reef ecosystem and make it available to the world. And do it at a planetary scale. Core tech is multimodal foundation models for biodiversity. 🔵 What? This enables deep modelling of complex ecosystem dynamics, such as population dynamics, predator-prey dynamics, energy transfer, phenological events (like spawning), and computes the ecosystem's health and resilience metrics. Via rigorous analysis of multimodal data, it uncovers precise mechanisms driving ecosystem change, offering humans irrefutable evidence to steer their actions. It shows quantitatively how coastal development, agriculture runoff, invasive species, pollution, increasing water temperatures, and other pressures affect the coral reef ecosystem. It shows what the coral reef ecosystem gives us back through its services, such as coastal area protection and oxygen production. It coordinates conservation and restoration efforts worldwide. We know which practises work and which don’t. It guides human activities to understand, protect and restore coral reef ecosystems. 🔵 Next steps: ↳ ✅ 3D "Street View" for coral reefs prototype [DONE] ↳ 3D digital twin of coral reefs ↳ multimodal twin of coral reefs ↳ model ocean ecosystems ↳ model biosphere ↳ become the first generation that actually leaves behind nature better than we found it! 🐳 🌎

Sneak peek into the larger 3D "Street View for Coral Reefs" model: - ❌ geometry is not good - ❌ no cleanup of wacky splats - ✅ used 200 photos from 2 GoPros (almost one swim-through)