After doing a custom modular diffusers integration of MotionBricks, which I want to use for doing a smooth transition between character motions, I couldn't resist to do a PoC game to show what this model can do in realtime inside a game.
if you want to test it yourself, I made a public repo of it, you can download it from the releases: github.com/asomoza/tactic-op…
the modular diffusers custom blocks are here: huggingface.co/OzzyGT/motion…
I'm making a series of tutorials and guides for using modular diffusers with different models to make PoC games, since most of the ones I see are just low effort copies, I want for people that want to learn how to use different tools to build their game and not just let the AI do it all. Also I highly recommend to use human taste for the art direction because it makes a big difference.
One of this models that it's really underrated is kimodo soma from @nvidia which can let you do all the motions you need for your game.
for Moss-Image by @Open_MOSS I did a quick test using a reference image so I didn't have to write the prompt, same as krea-2, and it works pretty good with diffusers, now I don't have to write most of the prompt if I want to use another image as a base
if you want to reproduce what I did, here is the code for it: github.com/asomoza/diffusers…
Alvaro Somoza retweeted
FLUX 3 Image Klein is here.
Open Weights with the quality of a large frontier model. Customizable and fast.
5.6B parameters and runs on a single consumer GPU.
Native 4K, bounding boxes, and pixel-perfect editing.
Another pretty good use for Minimax-H3 I found is generating SFX for games. You can feed it a clip of whatever you want the SFX for, and let the model generate the sound for it
So I needed a model that could position each part I want in a specific position, like the engines and mounted guns because for a game you need this, there is currently only one model that can do this. I still don't quite understand why people don't like bounding boxes...
I posted that you can extend videos with Minimax-H3, but did you know you can do the same with an image?
It doesn't need to have the same aspect ratio. You can use any image, mask it, and let the model outpaint the rest. And this model is damn good at it.
In case you didn't know, Minimax-H3 can spatially outpaint videos without any LoRA, you just need to use masks.
If this isn't something everyone knows and you want to see how to do it with Diffusers, let me know and I can make an example.
Following the prediction previews I did for Krea-2, I did the same for Minimax-H3 as a video example.
This can be useful if you have a clear composition in mind, since you can cancel the generation early if it doesn't match what you want.
huggingface.co/OzzyGT/minima…
Not sure how many people know this, but with Krea-2 you can start from a reference image, use its composition as inspiration, and then change what you want from it.
You don't need to describe the whole scene in the prompt.
And that's only one way to use reference images with Krea-2. There are also LoRAs for style transfer by @ostrisai and identity editing by Conrad Locke:
huggingface.co/conradlocke
I wrote a short post covering all three approaches, with examples and code:
huggingface.co/blog/OzzyGT/u…
Following up on my requested Diffusers examples, I made another set of custom blocks with Modular Diffusers, this time to show how you can change the previews from noise to the actual image prediction, like most other image generation apps do.
huggingface.co/blog/OzzyGT/p…
A long-standing question in Diffusers has been how to use multiple LoRAs with masks so they don't bleed into each other.
With Modular Diffusers, I created custom blocks to showcase how to do it and wrote a short blog post explaining the process.
huggingface.co/blog/OzzyGT/k…
I added 3 new objects and modified another one in this image. Can you guess which ones? And which model did I use?
Learn how I built this with Modular Diffusers and custom blocks for differential diffusion inpainting:
huggingface.co/blog/OzzyGT/i…
