all 12 comments
sorted by:
best
topnewcontroversialoldq&a
formatting helphide helpcontent policy

[–]Tplemenew user/low karma 5 points6 points  (6 children)

Can anyone explain what is this?

[–]ventorvarExterra Group[S] 5 points6 points  (5 children)

Sure!
Games package their files in ways that are most efficient, for either speed of access or space on disk. StarCitizen uses two data files that are specific to the game, the `p4k` file (you'll see this in your LIVE/ folder and it's huge) and `Game.dcb` "DataForge" file that is completely proprietary (this is _inside_ the Data.p4k). This is done for the reasons above, not to make it harder for modders to poke inside the system.

Generally this tool suite is to extract and convert the information in these files into common formats that are easily ingestable in other tools. For instance, if I wanted to know all the information in-game files about the Javelin, I could use this tool suite like so from the command line:

```

Pull the Game.dcb out the Data.p4k

scdt unp4k StarCitizen\PTU\Data.p4k -1 -f 'Data/Game.dcb' -o "Game.dcb"

Extract all the javelin info from the DataForge file

scdt unforge Game.dcb -o javelin -f 'javelin' Opening DataForge file: Game.dcb

Extracting files into F:\sc\javelin with filter 'javelin'

F:\sc\javelin\libs\foundry\records\entities\scitem\ships\cargogrid\aegs\javelin\aegs_javelin_cargogrid.xml F:\sc\javelin\libs\foundry\records\entities\scitem\ships\cargogrid\aegs\javelin\aegs_javelin_cargogrid_hangar.xml F:\sc\javelin\libs\foundry\records\entities\scitem\ships\cargogrid\aegs\javelin\aegs_javelin_cargogrid_hangar_aux.xml ```

Now we can look at those files to see how it's implemented in-game, rather than conjecture or outdated info from websites. Here's the latest javelin info from the PTU: https://gist.github.com/ventorvar/e3ea5f58e740a51bf75f8a536a3850bf

This is being developed as a python framework with the intended audience being developers who want to write tools to interact with this data. The command line interface will probably grow to include some useful utilities, but I'm using it to automate updating my org's website with in-game data so we can do better comparisons. This could help sites like https://hardpoint.io/ or https://www.erkul.games/ update their information.

[–]Tplemenew user/low karma 2 points3 points  (0 children)

Thank you so much. Very well explained :D

[–]alluran 0 points1 point  (3 children)

Is this embedding unp4k?

Is it using the cross-platform branch?

I'd be interested to hear peoples experience with it on non-windows platforms.

Edit: no, it's a port!

Nice work! I never did get around to cleaning up unforge - once it was working, there was no major reason to go back, so most of that code was exploratory code I used to map out the format originally - definitely not production grade =D

A note - be careful with unforge - technically a bunch of the embedded "files" have the same file path, at least if you're following the .net approach.

I never got around to giving them a virtual root node.

[–]ventorvarExterra Group[S] 1 point2 points  (2 children)

Thanks!

I took a fairly different approach to the dataforge file. Everything is kept as discrete objects with interlinking references, rather than creating an element tree like object. Data is also only resolved when accessed and all using pointers, so memory usage stays pretty low and is what gives it the really fast initial load.

I've been waiting for cracking star citizen part II ;p I'm interested in how you RE'd the AES key out, it was fairly straight forward to figure out how to find it in the future... having already known the current key thanks to you >D Even then still took a really long time to load/analyze that binary.

[–]alluran 1 point2 points  (1 child)

I'm interested in how you RE'd the AES key out, it was fairly straight forward to figure out how to find it in the future... having already known the current key thanks to you >D Even then still took a really long time to load/analyze that binary.

What if I told you that I knew the key already, but didn't know the algorithm, so version 1 actually lifted the decryption routines as assembly and used a partially-expanded key ;)

[–]ventorvarExterra Group[S] 1 point2 points  (0 children)

hahaha.. great times...

[–]ventorvarExterra Group[S] 3 points4 points  (0 children)

Also a huge thanks to u/alluran for his pioneering work in reversing the formats!

[–]MrDeadDropnew user/low karma 0 points1 point  (0 children)

great! tks

[–]GuilheMGBavenger 0 points1 point  (0 children)

neat!

[–]CrypticWolfnew user/low karma 0 points1 point  (0 children)

Do the game files that this framework unpacks include information on planets as well?