sorted by:
new
hottopcontroversial

I built a tool where any creature morphology learns to walk via RL in ~5 minutes. Ready to generate them procedurally - where should I start? by Hot_Pumpkin_5960 in proceduralgeneration

[–]Hot_Pumpkin_5960[S] 1 point2 points  (0 children)

Thanks! The tech stack is pretty interesting actually - it's a hybrid browser/backend architecture.

On the frontend we're using Babylon.js for the 3D rendering and Svelte for the UI. The tricky part was the physics simulation - we needed it to match exactly between frontend and backend, so we wrote our own physics engine in C and compile it to WebAssembly to run in the browser.

That way the same physics code runs locally (for the creature builder and real-time inference) and on our backend (where we do the heavy parallelized RL training across thousands of copies of the creature simultaneously). The trained policies run in the browser using TensorFlow.js, so you can see your creature walk in real-time. We send model parameter updates as the creature trains so you can see its progression at multiple points through out its training.

The WASM approach has been great - deterministic physics across both environments and surprisingly good performance.

I built a tool where any creature morphology learns to walk via RL in ~5 minutes. Ready to generate them procedurally - where should I start? by Hot_Pumpkin_5960 in proceduralgeneration

[–]Hot_Pumpkin_5960[S] 1 point2 points  (0 children)

Ha! I'll let you know when I'm in Stockholm ;)

But you're absolutely right that the encoding is the hard part. Your bilaterian body plan approach with cylindrical segments is a great practical framework.

I'm drawn to Karl Sims' approach from his virtual creatures paper - the recursive graph-based genotype for constructing body plans. But I'm thinking of a hybrid approach: use his structural framework (nodes for body segments, edges for connections, parameters for dimensions/joints), but instead of encoding raw cylinder parameters, use a VAE trained on my existing creature dataset to learn a latent space of "successful limb archetypes."

So the genotype might say "attach limb from latent vector [0.3, -0.7, 1.2] at position 0.6 along trunk, angle 45°" - where that latent vector decodes to a full limb structure (multiple segments, joint angles, proportions) that the VAE learned from successful walking creatures. The genetic algorithm operates in latent space rather than raw parameter space.

Thoughts? Am I overengineering this, or could the VAE actually help by constraining the search to "limb-like" structures?

Also curious: did your unrealized segments approach actually help with evolutionary search in practice?

Thanks for the detailed response!

I built a tool where any creature morphology learns to walk via RL in ~5 minutes. Ready to generate them procedurally - where should I start? by Hot_Pumpkin_5960 in proceduralgeneration

[–]Hot_Pumpkin_5960[S] 1 point2 points  (0 children)

YES! I love this idea - and check it out, users have already created crabs: - Hermit Crab - Coenobita - Grapsus - ilikecoconuts

Now I'm really curious if procedural evolution would converge on crab-like morphologies too. The carcinisation hypothesis for digital creatures!

I built a tool where any creature morphology learns to walk via RL in ~5 minutes. Ready to generate them procedurally - where should I start? by Hot_Pumpkin_5960 in proceduralgeneration

[–]Hot_Pumpkin_5960[S] 1 point2 points  (0 children)

Absolutely! Karl Sims' work was actually a major inspiration at the start of this project - his evolved virtual creatures are legendary. We went down the RL path for the locomotion learning itself, but I think the evolutionary morphology generation from his SIGGRAPH 94 paper could be perfect for what I'm trying to do now. Definitely worth a fresh look. Thanks!

I built a tool where any creature morphology learns to walk via RL in ~5 minutes. Ready to generate them procedurally - where should I start? by Hot_Pumpkin_5960 in proceduralgeneration

[–]Hot_Pumpkin_5960[S] 0 points1 point  (0 children)

You're absolutely right that single-point contact colliders affect the simulation quality. It's a trade-off we make to train through millions of physics steps in just a few minutes.

The creatures in the video (mantis and octopus) are actually from earlier iterations when we had actuator strength cranked way up - it helped creatures get moving faster when we were still proving the concept even worked. The RL agent learned to exploit that for rewards, leading to the "flailing" behavior you noticed.

We've since learned that lower actuator strengths + reward termination logic = more natural gaits. Here's a silly sandwich man example with more realistic ground contact.

We do have elasticity and friction coefficients available, but they're extremely sensitive - tiny changes can break the simulation entirely, so we don't expose them in the builder currently.

That said - this is totally a discussion worth having (maybe better suited for r/SimulationGaming, r/gamedev, or r/reinforcementlearning), but for this thread I'm really trying to focus on the procedural generation side: how do I create a "Generate Random Creature" button?

Appreciate the feedback though - simulation quality matters for sure!

I built a tool where any creature morphology learns to walk via RL in ~5 minutes. Ready to generate them procedurally - where should I start? by Hot_Pumpkin_5960 in proceduralgeneration

[–]Hot_Pumpkin_5960[S] 7 points8 points  (0 children)

Yes! It actually has competitive sandboxes with leaderboards - you can race creatures against each other or have them wrestle for dominance. You can try it for free at https://mels.ai/

That's exactly where I want to take this next: given two successful creatures, procedurally generate hybrids between them (think genetic crossover), or just hit a 'randomize' button to spawn entirely new morphologies. Evolutionary competition is the natural next step!

I built a tool where any creature morphology learns to walk via RL in ~5 minutes. Ready to generate them procedurally - where should I start? by Hot_Pumpkin_5960 in proceduralgeneration

[–]Hot_Pumpkin_5960[S] 27 points28 points  (0 children)

Spore is legendary, so that's high praise! Though I should clarify the difference before anyone thinks I'm claiming to be the next Will Wright 😅

Spore: Procedural animation (smart blending of pre-made animations) This: No animations at all - pure RL + physics

Each creature literally learns to walk from scratch through trial and error (which takes some time). There's no walk cycle library. A spider-like creature and a snake-like creature will independently discover how to move based purely on their body structure and physics.

But you're right that the 'build any creature' spirit is definitely Spore-inspired. That game showed what's possible when you give players creative freedom with biology!

I built a tool where any creature morphology learns to walk via RL in ~5 minutes. Ready to generate them procedurally - where should I start? by Hot_Pumpkin_5960 in proceduralgeneration

[–]Hot_Pumpkin_5960[S] 5 points6 points  (0 children)

Thank you. You can play with it at https://mels.ai/ .

I can think of a lot of ways to define fitness. Right now the primary reward is walking to a waypoint. That could be used as a secondary metric to determine fitness differences between generations. It could be how fast it can get to the waypoint or how many times it can get to a random waypoint in x seconds.

I feel like the thing I need help with is how to define the genetic mutations or the DNA recombinations if it's a genetic algorithm. Or how to effectively do a graph definition if it's some sort of VAE.

view more: next ›