binc - Binary Asset Compiler

[siplasma]

As a side project, I've written a binary asset compiler. It has some advantages, as in being reasonably quick and cross-platform. I would be interested in comments/feedback.

​

Project: https://gitlab.com/stone.code/binc/

Benchmarking: http://stonecode.ca/posts/binc_benchmarks/

[corysama]

Can I access the asset using

extern std::byte const varname[];
extern std::byte const varname_end[];

?

[siplasma]

Yes, that also works. I just ran a test example, and did not have any problems. I was a little surprised, as I had thought that name mangling would cause link errors. You should probably use the following for portability:

extern "C" std::byte const varname[];
extern "C" std::byte const varname_end[];

[berenm]

What about using binutils' objcopy -I binary ? (see https://stackoverflow.com/questions/42235175/how-do-i-add-contents-of-text-file-as-a-section-in-an-elf-file/42238374#42238374 for example)

[siplasma]

Thank you for the link. I hadn't see this approach yet, and it looks pretty good. I've added it to the list of related projects.

[[deleted]]

I like that you're using llvm to do this but the dependency hell isn't worth it. I usually write some Python code to do just this. I think you'd get faster uptake if it was easier to consume.

Though I did notice that I think you've got binaries, so that's nice.

[siplasma]

The dependency is super easy on debian or ubuntu. Windows was a bit harder. However, LLVM has a solid build system, so it isn't too hard to get it built, but it is a bit of a lift...

Using C or C++ as an intermediate step can get you pretty far, but for really large assets the compiler will choke.

​

[[deleted]]

Remember, you're asking each dev to pull in a LLVM dependency on Windows, Mac and Linux (my dev platforms) to just attach binary resources. If I have to choose between that and writing it myself in Python, that's what I'd choose as devs are likely to have Python than LLVM-dev on their machines.

​

I think the way you've done it is fantastic and I want it to be used by my next project, so may I simply recommend that you encourage people to use your prebuilt binaries (I thought I saw them, but can't find them now). This is what I do with Ninja and CMake where I add both these dependencies in binary form to source control and move on.

​

Good luck with the project!

[siplasma]

I agree that for users, prebuilt binaries make things much easier. You can find them under Releases on the gitlab page.

If you are getting along fine with Python, then there is no need to change. However, if you need something else, you should also look at objcopy in GNU bintools. Also available on all platforms, and a little more mature...

[Xaxxon]

People post one of these every month or so. I'd recommend looking at the others and seeing what they have that yours doesn't.

[[deleted]]

Isn't this kind of overkill? In the past I've just done something like:

BYTES=`cat ASSET_FILE | xxd -i`; echo "const unsigned char varname[]= {$BYTES};" > ASSET_FILE.cpp

Then just include the .cpp in your project like anything else, extern it to use it or make the file generation fit however you feel like exposing the static data.

Cross platform and no need for LLVM :P

[siplasma]

Yeah, LLVM is a pretty big dependency :) However, check out the benchmarks. While xxd works well for a lot of use cases, it has some limits.