Experimenting with Tech even though I have no clue what I’m doing
Phoenix, AZ
Joined October 2018
- Tweets61
- Following22
- Followers15
- Likes31
Pinned Tweet
Tried making a music video entirely with AI tools. Way harder than I expected, and way better than I expected. More info on how 👇
AI video kept making the small background face twitch during a slow zoom out, so I built a tool to fix it.
It tracks her face through the camera move and locks it in place, frame by frame.
Using my same character for a reimagined taylor swift video. Before vs After 👇
How it works, if you want to try it (Python + OpenCV + ffmpeg):
1. Pick one clean frame where her face looks right, and draw a box around the face.
2. Track the area around that box through the clip (feature tracking) to get how the camera moves, scales and turns each frame.
3. Smooth that motion over time so the tracking itself doesn’t jitter.
4. Paste the face from your clean frame into every frame, warped to follow the camera, with soft feathered edges.
5. Snap it onto the real face each frame (OpenCV ECC alignment) so it sits exactly in place.
The result: the camera still moves, but her face stops twitching.
A lot of tools had to come together. You need a character, then good starting images, a video generator that does lip sync, and of course the actual song too. Consistency with content really is difficult to tackle.
Happy to answer any questions
okay this is actually cool. images, then video, now worlds you can walk around in. i need to try this. now make it run locally lol
Has anyone gotten deep into the agents world? I’ve tested open claw, dots, grok bot. Just not super sold yet. I’d love to be convinced to use them more.
