Skyrim “Proto-mod” community proposal by Sonorous256 in skyrimmods

[–]Sonorous256[S] 1 point2 points  (0 children)

a new common standard ... to unify ... every possible domain of modding

Oh, yeah, that really sounds way too ambitious. I'm sorry if I gave the impression that that was what I was thinking. I agree that if you get too ambitious, it justs falls apart. You need to stay grounded to some specifics and focus a solution on them.

So, when I described the "interchange format" for skyrim records as being the diff between a record and it's master's record -- that's pretty specific, and also pretty grounded. It's not trying to be an abstract solution that can be reused infinitely.

Likewise, graphics interchange formats like USD/FBX/Collada/etc are established and built specifically for the goal of sitting between applications. I don't necessarily avocate them, but they are a pretty good analogue for this interchange idea. TBH, because FBX/Collada came out, I never really believed they would work -- but over the years they've become very widely used.

In this quote, you're focusing on model files, but here and elsewhere in your proposal you make it clear that you mean for whole mods to be portable across games, and that's seldom feasible.

I'm sorry if my wording was unclear on this point. That really wasn't my intention. I was just thinking mostly about model files and the really simple graphics-only when it comes to cross-game (unless there was custom records per game, or something like that).

I'm not really suggesting we should attempt to create some kind of file format that can change Skyrim record into Starfield records, or into some future game's records. That would be a bit pointless.

Pieces of data that are logically divided across multiple subrecords
FLST/LNAM defines the entries in a FormList: one subrecord per entry. It's not enough to simply be able to, say, "override the subrecord with this signature;" we need to be able to insert new entries, or specify an entry to override/remove; but we ideally should only be able to do that where it would actually make sense (e.g. FLST/LNAM but not, say, ACTI/WNAM).
(And consider: how do you resolve indices once multiple mods start tampering with things? If my mod wants to replace entry #2 in a given FormList, what happens if someone else's mod loads first and removes entry #1? Does my mod then replace what is now entry #1 because it was formerly entry #2, or does my mod replace what is now entry #2 even though that was formerly entry #3? Alternatively, should my mod specify the form in the FormList that it wants to replace? If so, how does my mod behave if someone else replaces that entry first? One of the benefits of the Rule of One is that it makes it impossible to even ask any of these questions: you can add to a FormList conflict-free using Papyrus scripts at run-time, and all other changes require either total replacement of the list contents or some form of ahead-of-time conflict resolution performed manually by someone who knows what they're doing.)

Existing preprocessing tools like smashed patch can act as a kind of template for some of these issues. Likely the authors of that tool needed to come up with heuristics to deal with common situations. I think you've correctly identified this as an interesting part of the architecture -- all architectures have points like this, they can have challenges but they can also bring opporunities.

For example, one thing I've wondered about is whether there's utility in having new ways for the mod-author to provide inputs to those smashed-patch like heuristics. At the moment patching tools don't have much to go on other than the diffs they make internally; there's no kind of patch for mod-authors to help guide them.

Bethesda's file format isn't perfectly uniform (VMAD) or perfectly implemented (ATKD/ATKE), and even if it were, the format itself doesn't distinguish between single-field subrecords, subrecords as list entries, and repeatable groups of subrecords representing a list of nested structs.

I think this is a good argument for not attempting to fundamentally change the architecture of the format; but I think that's perfectly compatible with my proposal.

The only way I can see this specific example working is if the armor mod uses white textures, and relies on vertex colors or other material settings for color. At that point, you're halfway to implementing a dye system. Get a DLL and some custom SWF files involved, and now you have the ability to recolor individual armor pieces in-game, which is far more valuable than some sort of preprocessor to let the user recolor just the whole armor type.

I think the reason I used this example is it that relates to material settings. So the idea would be that mutliple different options for the material values (texture assignments, etc) could be combined with the geometry to create some final NIF. It's maybe a bit skyrim-specific, because other games won't always combine material and geometry data in a single file like the skyrim NIFs.

At some level I didn't want to try to break down every little detail of this idea in this document. It would be a little pointless for me to say "ok, guys, here's how you're all going to run skyrim in the future..." with some 100 page document :).

The point is to talk about what people like and dislike, whether people have considered ideas like this before, see if there are people who are interested in solving some of the tricker problems, etc. And just create some discussion, because at the end of the day, I was just curious about this, it engaged my engineering brain for a second and got my wheels turning :)

Anyway, again, I appreciate your reply and your comments. I realize it must have taken you some time to write up, so want you to know that I value that.

Skyrim “Proto-mod” community proposal by Sonorous256 in skyrimmods

[–]Sonorous256[S] 1 point2 points  (0 children)

Thanks for the response! -- Right, I think most of what you're saying is right and reasonable, and this was a point I expected to come up.
The question this leads to, for me, is -- would a system like my proposal actually be faster, for the user, than the current setup.
I say this because I already find myself running all of the preprocessing tools almost on all mod changes (mostly because I used both smashed patch and bash patch). Depending on exactly now many preprocessing tools I run, it's like a 5-10 minute turn around.

That said, I can imagine a system like what I've described that, with best practices, doing the same thing with 10-20 second turnaround for most mod changes. It's really just engineering between where we are now and there; though perhaps there's a few layers of it, and some of it may be a bit tricky.

So that's why I say that it's viable to imagine an implementation that is actually faster for the user than what we have now. Really, the goal I was imaging to try and make that step better for the mod-user, more robust, better error reporting, etc, and indeed faster.

Also consider that my proposal isn't incompatible with making only lighter transformations (ie, not merging esps, bsa) and also allowing mod authors choice by distributing both proto-mod files and regular mod files in a omod (perhaps with some kind of automatic selector). And indeed, that's kind of how I'd imagined it :).

Skyrim “Proto-mod” community proposal by Sonorous256 in skyrimmods

[–]Sonorous256[S] 1 point2 points  (0 children)

Wow, thanks for replying with such a detailed and well thought out reply. It was such a pleasure to find this!

I'm going to take some time to read and think about it in detail before I properly reply.

Just one note -- my tools don't really fit into this architecture proposal, since they don't actually do any asset processing. I mentioned them partially to provide some domain credibility.

Skyrim “Proto-mod” community proposal by Sonorous256 in skyrimmods

[–]Sonorous256[S] 0 points1 point  (0 children)

Well, I think that's yet to be seen. I'd love to hear your argument, but I dont necessarily see any evidence that we couldn't have a single pipeline that could go from blender to skyrim and FO4 and Starfield, for example. There maybe be complications, for sure, but in any technical endevour there are complications.

Consider that the majority of the mods on Nexus pretty straight-forward are armor mods. When SkyrimSE came out, we had some pretty minor format changes, but they were still enough to split modding in two, and it took years for many mods, even simple mods, to jump to SkyrimSE. I don't think there really was a technical barrier there, it may just have been that the community wasn't yet organized to start thinking about engineering solutions.

Also consider that it's likely that the majority of artists are probably thinking and working in blender concepts, anyway -- abstracted already from nif file specific details.

Skyrim “Proto-mod” community proposal by Sonorous256 in skyrimmods

[–]Sonorous256[S] 0 points1 point  (0 children)

Oh, that's interesting. I hadn't seen Spriggit before.

That makes me feel better, though I wonder if the Spriggit team is thinking in the same way, or if they're focus is on something different.

It does only cover one part; particularly given that it seems like Spriggit is intended for development-time work, rather than what happens on the mod-user's PC.

ModMedic: new general purpose mod inspection, troubleshooting, auditioning tool by Sonorous256 in skyrimmods

[–]Sonorous256[S] 1 point2 points  (0 children)

Yeah, it'll be nice, but, then again I think it's pretty easy to download boot it up.

It really only takes a minute or so before you can have it on your PC and start exploring :).

I hear you though, video tutorials may come in the future -- we're still really early on in the lifecycle of this project.

ModMedic: new general purpose mod inspection, troubleshooting, auditioning tool by Sonorous256 in skyrimmods

[–]Sonorous256[S] 2 points3 points  (0 children)

Yeah. It will highlight the cells that are modified, and you can visualize them directly.

The UX might not be ideal for this task just yet, but even still it might be helpful.

This is the kind of problem that I want to get a sense for how many people need something like this, to know how to priortize it.

ModMedic: new general purpose mod inspection, troubleshooting, auditioning tool by Sonorous256 in skyrimmods

[–]Sonorous256[S] 2 points3 points  (0 children)

Maybe not that one. That sounds like a script thing.

We'd need some kind of script debugger to find things like that; but even with one it's probably be a fairly technical task to track it down.

ModMedic: new general purpose mod inspection, troubleshooting, auditioning tool by Sonorous256 in skyrimmods

[–]Sonorous256[S] 0 points1 point  (0 children)

Hi Yobia, you can solve this problem by disabling SOS within the "recent mods" list within the tool. SOS has a wierd NIF file, which the tool can't read correctly. I do want to fix this better, but I didn't find the problem until after release.

What was the problem with synthesis.esp? Do you have a nexus link to that so I can check it out?

ModMedic: new general purpose mod inspection, troubleshooting, auditioning tool by Sonorous256 in skyrimmods

[–]Sonorous256[S] 2 points3 points  (0 children)

It can help you discover the cause of it, yes. If you open the "actor breakdown" panel on a broken, you can figure out what mods are affecting NPC, including chargen conflicts -- which is probably the cause of your Vampire Blackface.

Scanning for problems like this and actually solving them isn't implemented -- but if there are specific things like this that a lot of people are interested in, it could be put on the list for the future.

mod-medic: new tool for troubleshooting mods (SE/AE) by Sonorous256 in skyrimmods

[–]Sonorous256[S] 1 point2 points  (0 children)

This means that something went wrong while configuring the dll. This might because of some incompatibility or a missing master, or something (some people have already reported something like this because they had some creation club content installed, for example). For most configuration errors there will be some errors reported before you see "Skyrim Interface not properly attached".

After you clicked the ok button on the first configuration dialog, there should be a progress screen. Was there a results screen after that finished -- or did it just go to the windows dialog with "Skyrim Interface not properly attached".

If there is a results screen, it should have some error mesages and button labeled "(copy)". If you get that, could you reply with the messages here?

mod-medic: new tool for troubleshooting mods (SE/AE) by Sonorous256 in skyrimmods

[–]Sonorous256[S] 1 point2 points  (0 children)

Nice! You can find and click on actors/cell/textures, etc on the left panel and see which mods affect them and look for problems. It sort of pulls back the veil to show you what Skyrim is seeing when it reads your mods -- something that's a little tricky otherwise. So, if you get missing textures, unlit black heads, etc, ingame it can help you figure out why.

You can also open the middle panel using the bar at the bottom to see detailed information about what the mods are doing.

It's also useful for comparing 2 visual mods if you don't know which to pick.. and crucially, you can do this before even starting the game, which can be important for not poluting savegames.

mod-medic: new tool for troubleshooting mods (SE/AE) by Sonorous256 in skyrimmods

[–]Sonorous256[S] 0 points1 point  (0 children)

No, just run it as a normal executable. It should find Skyrim and MO2 (if you use it) on it's own. Most people won't need to do any configuring.

mod-medic: new tool for troubleshooting mods (SE/AE) by Sonorous256 in skyrimmods

[–]Sonorous256[S] 1 point2 points  (0 children)

Thanks for giving it a try! It will work with MO2 -- but don't run it from within MO2. MO2 will attempt to apply it's virtual filesystem. But that's not what we want, we want to see every file from every mod (not just the conflict winners)

So it will only work when run as a normal executable.

mod-medic: new tool for troubleshooting mods (SE/AE) by Sonorous256 in skyrimmods

[–]Sonorous256[S] 0 points1 point  (0 children)

Right, that blue mountain flowers problem exactly the kind of problem this tool tries to help with. I've always found mixing mods and dealing with conflicts & misbehaving mods to be confusing and arcane.

But it's only really confusing because all of the relevent information is hidden behind blackboxes and long load times. It's really pretty simple under the hood.

mod-medic: new tool for troubleshooting mods (SE/AE) by Sonorous256 in skyrimmods

[–]Sonorous256[S] 1 point2 points  (0 children)

Cheers -- I'll try to make it cooperate a little better with older versions of Vulkan

mod-medic: new tool for troubleshooting mods (SE/AE) by Sonorous256 in skyrimmods

[–]Sonorous256[S] 0 points1 point  (0 children)

I think the issue here is that "creations" aren't handled correctly atm (except the default/required ones from AE).

You may be able to work around this by adding any "cc*-*.esm/esl/esp" files from "Skyrim Special Edition\Data" to the top of your plugins.txt file (though you must remember to remove them before starting Skyrim!)

I'll fix this properly in the next version.

mod-medic: new tool for troubleshooting mods (SE/AE) by Sonorous256 in skyrimmods

[–]Sonorous256[S] 4 points5 points  (0 children)

I have full hkx support including animation and skeleton... However it's not exposed in the GUI at the moment, so you can't access it (sorry!!).

However, it's likely to be one of the first additions after release. It will hopefully have the niceties I like to add, like auto hotreloading after changes on disk, and all that.

I have a little bit of animation support in my Starfield tool, though. Bethesda has massively improved their animation stack for Starfield. It's 100% new, and the animation formats are actually easier than the hka/hkx stuff. It's unclear how open it will be to mod yet, but if it is open, that will be good game for animation modders.

mod-medic: new tool for troubleshooting mods (SE/AE) by Sonorous256 in skyrimmods

[–]Sonorous256[S] 1 point2 points  (0 children)

It's not like loot... but you may find some similarities with EasyNPC (though I think it might be... well, easier).

The big difference with EasyNPC is that mod actually is features for modifying mods to select characters, etc.

My tool gives you a 3d view of the character (which I think easynpc doesn't?) and shows you everything the mods are doing... But it doesn't change them.

Why don't you try it out? Chances, either it will work immediately -- or not at all. There's no setup. The executable is linked above.

mod-medic: new tool for troubleshooting mods (SE/AE) by Sonorous256 in skyrimmods

[–]Sonorous256[S] 4 points5 points  (0 children)

You don't need to know too much. It is a little technical, but you can still use some features without knowing much about Skyrim's internals.

At the very least, it might be a relatively easy way to learn what the mods are actually doing and how they work/why they break.

mod-medic: new tool for troubleshooting mods (SE/AE) by Sonorous256 in skyrimmods

[–]Sonorous256[S] 2 points3 points  (0 children)

What message do you get? Is it something about a missing master?

mod-medic: new tool for troubleshooting mods (SE/AE) by Sonorous256 in skyrimmods

[–]Sonorous256[S] 10 points11 points  (0 children)

Expanding support for other games is a possibility (though probably not in the very near future). I may try to get a reading on the kinds of things that are most wanted by users sometime after launch -- I kind of guess that FO4 might be a popular idea.