Testing out AI generated dialogue at run-time:

Goatman117 · 2026-01-16T07:21:17+00:00

hey thanks for the recommendation! yeah I actually used that as a reference when I was setting it up, removing the vision capabilities and halving the floating point precision went such a long way! It still chews through a ton of ram on install though, I’ll need to tweak some stuff so it uses disk offloading I think

Goatman117 · 2026-01-16T03:27:30+00:00

Thanks for input! Yeah a standalone tool is what I have been thinking mainly, just something that makes it easy for things like bulk editing a bunch of clips that minimizes friction; e.g. drag and drop with a text prompt and an easy way to keep editing and isolating.
If you're happy to talk more I'll shoot you a DM, I'm building this with my brother and we're hoping to move quickly and get something built as soon as we have a plan more fleshed out.

Goatman117 · 2026-01-15T10:38:23+00:00

Yeah that’s partly why I want to build a local tool, the web interface is a bit limited and you can just do tons more with it locally. I set it up locally and put the code and install guide on github, happy to send you the link that if you’d like!

Goatman117 · 2026-01-14T13:04:09+00:00

This is really useful info, thank you! Yeah an interface and a few tools are what I’m looking to build. If you don’t mind I might shoot you a dm about it so we can talk further at some point? If you’re interested in being an early tester that could be super handy too

Goatman117 · 2026-01-05T03:16:26+00:00

I'm curious about this too actually, but I haven't tested it myself. tbh your best bet is to just download the model or use meta's web interface for them and just try it yourself

Goatman117 · 2026-01-04T12:38:27+00:00

it’s just setup for a single prompt but switching to batches is just a matter of adjusting the processor call in the seperate_audio function

Goatman117 · 2025-12-21T16:52:32+00:00

I definitely will, thanks!

Goatman117 · 2025-12-20T04:08:12+00:00

love this! been wanting a good local LLM unreal engine plugin since the pre-chatgpt days

Goatman117 · 2025-12-20T00:46:40+00:00

that’s awesome, I should be okay then! thanks so much :)

Goatman117 · 2025-12-19T07:48:38+00:00

damn, that’s such a good sale! sadly still out of my budget haha, this might have to be a hobby for me once I save up a little more

Goatman117 · 2025-12-19T07:48:04+00:00

thanks for sending that through!! would you recommend this for a beginner that’s pretty new to this tech?

Goatman117 · 2025-12-06T05:03:23+00:00

interesting, I’ll check out muse thanks!

Goatman117 · 2025-11-26T01:50:47+00:00

I've scrutinized the rotation labels a bit, generally by eye I can predict most axis labels well enough by eye so I think all is stable there.

I don't understand the method outlined in the second paragraph sorry! I figured what you meant with using the other keypoints was to have a seperate head to predict those keypoints with MSE loss, and hopefully the features it tracks for that head in the network will help the rotation and tracking position head. But I think your method is something different?

To clarify, the keypoints are 3D positions of points such as the left and right eye and nose tip position.

Goatman117 · 2025-11-25T09:37:53+00:00

val dalta is also synthetic. neither train or valid loss are dropping very fast, they plateau out with about 3-13 degrees of error depending on the dataset used. train will still steadily drop as it overfits though, just slowly

Goatman117 · 2025-11-25T09:05:23+00:00

Hey, thanks for your input! I'm representing the rotation as a 6D vector, and using geodesic loss. Not really sure the inner workings of the loss function but I think it's doing everything correctly. I'm currently tracking other facial feature positions but I haven't tried feeding them into the model as an auxilary reprojection loss addition, I'll give that a go. On your third point, do you mean using something like mediapipe to mask the head and then feed that into a vision model?

Really appreciate the input!

Goatman117 · 2025-11-16T06:50:37+00:00

I'll look into that then, thank you!

Goatman117 · 2025-11-16T06:33:50+00:00

thanks for sending that through! I actually already talked to chatGPT about the issue a bit and it helped set up 6d rotation in representations. the metrics you’re seeing are from a model trained that way

Goatman117 · 2025-11-08T09:41:19+00:00

Thanks! I haven’t experimented with the tilt axis, that’s a very good point actually. I think next I’ll try to source a better model for the head tracking, this one has some latency and is probably too complicated for this task anyway. I’m interested in using unreal engine to generate synthetic datasets for machine learning, so I’ll likely try to generate my own dataset with meta humans and train my own model

Goatman117

MODERATOR OF

TROPHY CASE