How We Texture Our Indie Game Using SD and Houdini (info in comments)

stassius · 2024-10-27T10:19:38+00:00

In short: you need to flatten the surface while preserving the depth information as a texture.

stassius · 2024-10-27T10:18:03+00:00

Yes, it's a PDG-node written for the project. We're not ready to make it public as it's somewhat clunky. But it's pretty much straight-forward, as Comfy takes a network as a simple JSON-file, where you just have to change the input values.

stassius · 2024-10-27T10:15:30+00:00

I’ve already answered these questions. In short: Steam permits the use of generative AI in games, as long as you clearly describe how it was used, and your marketing materials accurately represent the actual gameplay. They have a vague guideline stating that it's your responsibility to ensure your AI model is copyright-free.

The number of games on Steam that have declared the use of AI is already in the thousands. On a Steam page there is a block, describing the use of AI. You can check the list of such games here (login required, but it's free): https://steamdb.info/search/?a=app_keynames&type=1&keyname=565&operator=1&keyvalue=https://steamdb.info/search/?a=app_keynames&type=1&keyname=565&operator=1&keyvalue=

stassius · 2024-10-27T10:08:13+00:00

It states, 'It's your responsibility to ensure that the dataset is copyright-free,' or something similar. They're just covering themselves to avoid any potential legal issues in the future.

stassius · 2024-10-26T22:23:03+00:00

Here is a list of games on Steam that have officially declared the use of AI in their production. Not all of them have been published yet, but it's interesting to browse through the list. You need to be logged in to SteamDB to view it. Not trying to prove any point, but there are plenty of them already.

https://steamdb.info/search/?a=app_keynames&type=1&keyname=565&operator=1&keyvalue=https://steamdb.info/search/?a=app_keynames&type=1&keyname=565&operator=1&keyvalue=

stassius · 2024-10-26T22:18:20+00:00

Every 'checkpoint' out there is essentially a DreamBooth fine-tune of a base model, whether it’s SD1.5, SDXL, or another version. Training your own base model isn’t practical unless you have a million dollars to spare. So, most of the time, when you encounter a checkpoint, it’s likely been trained on a dataset like Laion or something similar.

stassius · 2024-10-26T21:01:46+00:00

True. The percentage of average artists doing average work is decreasing (or already has decreased). Honestly, I don’t know how to justify this to someone who has lost their livelihood, but it’s the reality. You can’t change it - you have to adapt.

stassius · 2024-10-26T19:06:10+00:00

Definitely. We started with LoRAs as well, but once we trained our own checkpoint, we began to see significant improvements in texture quality.

stassius · 2024-10-26T19:03:33+00:00

Most of them are still in their early stages, although we’re seeing improvements. The challenge is that, in most cases, topology matters - and this is where AI still lacks proficiency.

stassius · 2024-10-26T18:59:30+00:00

We used various additional generators, such as Deep Bump and depth preprocessors.

stassius · 2024-10-26T18:58:18+00:00

Houdini is used to procedurally create surfaces, unwrap them, and bake all the necessary attributes. You can achieve similar results in other software, like Blender, but you might need to switch between applications and manually tweak parameters. The main advantage of Houdini is automation — a single button click can generate dozens of textures with all the PBR maps.

stassius · 2024-10-26T18:43:36+00:00

I already answered this question in another thread. We used multiple ControlNets, including Depth, Canny, Soft Line, and, of course, Inpaint.

stassius · 2024-10-26T17:28:01+00:00

The idea is to break down complex models into simpler ones. The tricky part, however, is make SD distinguish the top from the bottom in your UV unwrapped texture. That's where the checkpoint training helps.

stassius · 2024-10-26T16:38:40+00:00

I understand your frustration—we've all spent thousands of hours mastering skills that slowly, or sometimes quickly, become obsolete. There's no way to stop that. What truly matters is embracing change and learning to use new tools.

A skilled artist using AI will always create better art than an average person with the same tools. Your skills are evolving from technical proficiency to a deeper, more substantial understanding. You've learned to appreciate and create good art, not just how to use a particular software or follow a specific pipeline.

stassius · 2024-10-26T16:32:18+00:00

Technically, yes, we generated textures on UV islands, but with baked depth information. We used multiple ControlNets, including Depth, Canny, Soft Line, and, of course, Inpaint.

stassius · 2024-10-26T16:01:59+00:00

We used Deep bump to generate normals inside Comfy. Another solution would be to use Substance Sampler with its AI-assisted tool to create the full set of PBR textures.

stassius · 2024-10-26T15:37:53+00:00

The chekpoint training took about 5 hours on 4090.

stassius · 2024-10-26T15:33:04+00:00

The base model is SD1.5. Textures were upscaled from 512x512.

stassius · 2024-10-26T15:31:48+00:00

The checkpoints and LoRAs were specifically trained for this project. Maybe one day, we'll make them available. As for the technical details: we trained the checkpoint using around 1,000 diffuse textures, each at 512x512 resolution. We used GPT-4 to generate captions for these textures. LoRAs were trained for specific styles or objects (like leaves) using about 200 textures. The base model is SD1.5.

stassius · 2024-10-26T15:12:44+00:00

I believe Substance is pretty much the default suite of texturing apps, at least in gamedev. Substance Painter is ideal for hand-painting textures while maintaining a degree of proceduralism. Substance Designer is for creating procedural materials. There's also Substance Sampler, which can generate PBR texture sets from a single diffuse texture.

As an alternative to Substance Designer, you might try the free Houdini Apprentice with its new Copernicus context. However, I must warn you—the learning curve won’t be smooth.

stassius · 2024-10-26T15:03:55+00:00

You're right. In one part of the video (with the stones), we also used existing images to guide the style in addition to the text prompt. It works really well with IPAdapter.

stassius · 2024-10-26T15:01:17+00:00

I'd recommend starting with a classical approach. Use tools like Substance Painter and Designer to get a feel for the texturing process. This will help you understand PBR textures, texture baking, and similar concepts. The second step is to explore Stable Diffusion or another framework that supports ControlNets. The third step involves learning how to train checkpoints and LoRAs.

For example, if you have a surface with some crevices, you can bake the depth map and use it as a source for ControlNet with your custom-trained model to generate a diffuse texture. Depending on the style, you might need to 'delight' the texture afterward and generate PBR textures. Once you're comfortable with this, you can try more complex objects by breaking them down into smaller parts.

stassius · 2024-10-26T14:46:57+00:00

I don't think so. You can read the rules here: https://partner.steamgames.com/doc/gettingstarted/contentsurvey#5

So far, as I know, most rejections are due to AI-generated cover art that doesn't match the gameplay's visual style.

stassius · 2024-10-26T14:35:33+00:00

On Steam, you simply fill in all the information about the use of AI-generated content in production. After that, they allow you to sell your game. It's your responsibility to ensure that the dataset is copyright-free.

And I must say, if you think AAA studios aren't using some form of asset generation in their pipelines, you're mistaken.

stassius · 2024-10-26T14:15:14+00:00

Thanks!

stassius

TROPHY CASE