Investing in open source physical AI
Boca Raton, FL
Joined December 2007
- Tweets5.2K
- Following1.3K
- Followers10.4K
- Likes8.9K
Pinned Tweet
Some promising early results of my enhanced Blender MCP compared to the standard Blender MCP. It was able to generate and render a UV unwrapped, low poly spaceship in 30% of the time with 15% of the tokens.
Home school: 12 year old is remixing smash brothers with his own characters using omarchy, Hermes, opensmash, and remesher
Day 1 and he got Emerl from Sonic Battles in Open Smash. We used Hermes + bpy.dev to clean up the rig that the Hermes remesher plugin generated.
Imagine trying to compete against open source in 2026. Why swim upstream when you can accelerate downstream?
@higgsfield_ai sent us a cease and desist last night, asking us to shut down our Open Source Higgsfield challenge.
Our read on why they sent it. When the apps people pay for can be rebuilt in the open in two weeks, that’s an uncomfortable thing for a closed platform to watch.
So here’s us reiterating our promise to every builder in the challenge and community, and we’re keeping it:
• The challenge stays open
• Final submissions close tomorrow, Oct 30
• Winners are announced Nov 6, with prizes paid as promised
You showed up for each other. We’re showing up for you.
If we win, we all win!
Comfy out 🫡
Setting up an old laptop from 2019 with a busted battery for my 12 YO to build video games on. Omarchy installed in less than 2 minutes.
Clever approach
I may have cracked how to animate non humanoid assets with AI! H3 Max generates a video from front and side then we use a finetuned DINOv2 model to track joints! The vid explains the whole process
Open source: github.com/patch-gg/anymove
Take a few hours this weekend and set up Hermes on an old machine. You won’t regret it.
Jensen closed his GTC Berlin 2026 keynote with this Hermes Agent demo as the finale.
He introduced it by saying the way to build an agent is to ask an agent to build it for you, so he installed and set up a customized Hermes in one prompt.
Then he used Hermes, running on @nvidia RTX Spark, to create a simulated training environment for the @huggingface Microduck. Hermes planned an obstacle course with Nemotron and had GPT-6 Astra build it using FreeCAD, Omniverse libraries, Blender, Newton, and Isaac Lab. Hermes trained thousands of simulated ducks on that course until the validated policy was ready to run on the real robot in the physical world.
At Nous, we believe open source is a practical tool as much as a philosophical ideal. It gives users the ability to build powerful, complex workflows chaining numerous local+cloud models, devices, and tools themselves instead of waiting for a company to enable them.
Michael Gold retweeted
🎮 Introducing Matrix-Game 4.0 — the world’s first open-source video model game that runs in real time on a single consumer GPU.
Instead of merely watching an AI-generated world, you can now actually play inside one.
⚡ 480p at 60 FPS
⏱️ 112ms end-to-end response
🚀 Up to 83.4 FPS generation
🖥️ Runs on a single NVIDIA RTX 5090
🧠 Coding Agent turns natural-language ideas into playable game logic
🌍 Game engines maintain world states and rules, while the video model renders every frame in real time
From shooters to racing games, the same world model can support entirely different gameplay—by changing the code, not retraining the model.
Matrix-Game 4.0 completes the loop:
Player input → Game engine updates the world → Video model renders the next frame → Player interacts again
Code, model weights, and demos are now open source. Build on it, remix it, and create your own playable worlds.
🔗 GitHub: github.com/SkyworkAI/Matrix-…
🤗 Hugging Face: huggingface.co/Skywork/Matri…
🌐 Project page: matrix-game-v4-0.github.io/p…
From controllable video to runnable worlds—the barrier to creation is getting lower, and the space for imagination is getting bigger.
#SkyworkAI #MatrixGame #WorldModels #GenerativeAI #AI #OpenSource #Gaming
Michael Gold retweeted
I think this is big. 🌠
FastAPI's objective is to provide what you always need for real prod apps. And not provide what you don't.
You always want metrics, there's just not enough time to always set it all up. Now it's just there, by default.
FastAPI now has built-in OpenTelemetry support, natively 📈
No extra code needed.
See how your app behaves in production.
Go and upgrade to "fastapi[standard]" version 0.142.2 ⬆️
Read the docs: fastapi.tiangolo.com/advance…
Michael Gold retweeted
Yesterday, my boys and I vibe coded a game together. They tell me what they want in their game and I prompt it into Opus 5.5 to build it.
And of course they picked "Zelda Breath of the Wild" inspired game, where they can explore an open world, do dungeon puzzles, fight mini bosses and a final boss. The twist is that the hero holds a giant paintbrush as a weapon, each color had a different attack and effect (red = fire, blue = ice, yellow = bounce, green = vines). And oh they also wanted to fly on the paintbrush.
We spent our whole saturday playing and iterating on the game. It was great fun.
I told them, the future is all about Imagination, whatever they can imagine, it will be manifested in a digital experience.
At the end of the day, the game had:
- 2 biomes (1 island and a bunch of floating islands)
- Paintbrush-based combat (4 colors representing different effects and mechanics)
- 3 dungeons with puzzles
- An undeground cave
- Unlockable flying paintbrush when you activate 4 crystals around the island
- 30+ enemies and elites
- 1 giant boss
As you can see in the gameplaye video below, its not the most polished video game but it's so satisfying to see my boys getting bewildered when whatever they asked for, the game is built before their eyes.
Of all the games I have vibe coded so far, this was the most satisfying because my boys were actively co-creating it and playing it.
This is definitely not the last game we're making together, they have already started brainstorming their next game, hint: "Peak with grappling hooks and TNTs."
You can play the game here, works only on desktop:
wildbrush.vercel.app/
I’ll be in SF this week. Would love to connect with people working on Physical AI software or research.
Testing out low-poly modeling with a very opinionated MCP and reference art generated by @DeemosTech
Michael Gold retweeted
I said "auto-regressive LLMs, in and of themselves, will not lead human-level AI"
That statement is still totally true.
First, the reasoning abilities of current AI systems are based non-auto-regressive search (which is what I have always advocated for). But AFAICT, they do it in token space, which is limited and inefficient. I have claimed that human-like reasoning must be a search in continuous representation space. It looks like the industry is moving towards that.
Second, the self-improvement methods, as currently practiced, only work for domains where the quality of outputs can be scored without human intervention, such as mathematics, code, and scenarios that can be simulated accurately. Not anything else. Humans and animals learn new skills way more efficiently than current RL methods.
Third, the multimodal capabilities of current AI assistants generally use separately-trained encoders (that are not LLMs). This is also what I've been advocating. Except that I think the best way to do this is with JEPA trained with self-supervised learning. The research community is clearly moving towards that (3000 papers on JEPA in just 4 years).
Fourth, if LLMs were a path to human-level AI, we would have domestic robots and Level-4 or Level-5 self-driving cars for consumers by now. And we don't. We certainly don't have cars that can learn to drive in 20 hours or practice like any teenager. We're still missing something pretty huge to claim human-level intelligence (let alone superhuman).
Sure, we now have computer systems that are impressive, very useful, and whose performance is superhuman in an increasing number of domains (coding being one of them).
But that's true of the entire history of progress in computer technology.
Lastly, there is a basic confusion about what intelligence actually is.
It is not the mere accumulation and regurgitation of existing declarative knowledge (which is essentially what LLMs do).
As Jean Piaget famously said, "intelligence is not what you know, it is what you do when you don't know."
It is your ability to solve new problem without any prior training, to act in previously-unknown scenarios, and to adapt very quickly to new situations with minimal training.
We're still far from that.
Just use both. Frontier models are so good for coding and supervising small models that are still very drunk / hungover.
