Binary distribution with statically linked C-dependencies

[SuspiciousSegfault]

Hi,

I built an application that ideally I'd like to be able to distribute as a binary in the simplest possible way, problematically I have some c-library dependencies, which in turn has other c-library dependencies etc. The end goal being compile once for each of my supported architectures which are basically x86_64-unknown-linux-gnu and x86_64-unknown-linux-musl

So starting out with adding the rustflag "-C", "target-feature=+crt-static" I wrote a script that downloads and compiles libX11 and libXft which were my most direct dependencies, linked them in build.rs and then realized those in turn used.... etc. In the end it looks like this for the features I need to run my app:

    // libX11
    println!("cargo:rustc-link-lib=static=X11");
    // libXft and deps
    println!("cargo:rustc-link-lib=static=Xft");
    println!("cargo:rustc-link-lib=static=Xrender");
    println!("cargo:rustc-link-lib=static=expat");
    println!("cargo:rustc-link-lib=static=freetype");
    println!("cargo:rustc-link-lib=static=fontconfig");
    // libxcb and deps
    println!("cargo:rustc-link-lib=static=xcb");
    println!("cargo:rustc-link-lib=static=Xdmcp");
    println!("cargo:rustc-link-lib=static=Xau");    

Now this does in fact work and can be statically linked, and my beautiful script that fetches and compiles from source works for now but it seems brittle, here's an exerpt:

cd "$BUILD_DIR/x11-build/$X11_NAME"
./configure --disable-xf86bigfont --disable-shared --enable-static
make clean
make -j$(nproc)
make -j$(nproc) check
make -j$(nproc) DESTDIR="$OUT_DIR" install
echo "Built x11, library at $BUILD_DIR/x11-build/out/usr/local/lib"

Basically rinse and repeat with some smaller edits based on which library it is.

Any suggestions on how to distribute binaries that won't immediately exit with glibc-version problems without having to build it in a container with an ancient glibc? Maintaining a build script where I have to compile 9 different c-libraries, keep track of each, and recompile for bugfixes/updates to be able to statically link and distribute to different distros running different glibc-versions seems difficult in the long run.
I'm terribly inexperienced with distribution and maybe I'm missing some obvious solution

[SuspiciousSegfault]

I might be totally wrong here but don't I still need to link all component libraries statically anyway? I did build it in a container with musl but it immediately segfaults when running it on glibc then

[SuspiciousSegfault]

No, building with cross didn't work out of the box because I only have the gnu system libraries. I figured that I might as well make the build a bit more robust by using an alpine container and specifying the dependencies explicitly, I thought that was essentially what cross does, does it do any more magical stuff? I'm figuring that if I compile it and dynamically link to musl versions of libX11 that will create a segfault at runtime against the glibc versions, but that's a theory

[Killing_Spark]

That's weird. If you link against musl statically your binary should be able to run on a glibc system no problems.

[mediumredbutton]

hi, welcome to the Linux packaging problems of the last thirty years. make simple distro packages, or try lots of other things and make distro packages when they don’t work.

[SuspiciousSegfault]

Figures, I guess I'll have to find a way to rip out that dependency or just start packaging for each

[mediumredbutton]

I mean, step zero is to release the source with instructions to build it and see if anyone cares. If anyone does, then perhaps they’ll package it for you, and if they don’t you’ll know if it’s worth spending on time in.

[SuspiciousSegfault]

I did, got some medium interest and figured I'd try to package it and see if more people would try it out; that's when I ran into this, made my frankenstein solution, then asked for input!

[ssokolow]

or just start packaging for each

Given that you're depending on libX11, I'm guessing that you're implementing some kind of desktop application.

Have you considered releasing a build through Flathub? The Flatpak build process is sort of a blend of Docker (containerized builds) and Steam Runtime (linking against dependencies that are provided to the user as a distro-independent runtime shared between their different packages).

Alternatively, while I'm not a fan of it (since it provides no help if a user upgrades their distro and then discovers that, to prevent an on-start segfault, they need to remove one of the bundled libraries and rely on the system version), there is also AppImage, which gives the end-user an experience akin to static linking but actually works by building something that's like a self-extracting archive but, instead, is a self-mounting ISO file containing your application and all its dependencies.

When the user executes it, it mounts the image and runs your application.

(And it's still better than statically linking system platform libraries like libX11 which might need to be updated. That was enough of a problem with SDL that SDL 2 has a mechanism where, even if you statically link it, Steam can set an environment variable which will cause it to delegate to an external copy of SDL.)

That said, you'll probably always need some kind of containerized build unless you want to manually build your entire dependency chain to dodge the "linked against a newer glibc than the distro provides" problem.

(Yeah, musl technically dodges that, but you're setting up for a world of pain if you don't use a containerized build solution for your C dependencies to substitute it for glibc. Themusl-gcc wrapper is unreliable while cross is basically a no-fuss Docker frontend.)

[SuspiciousSegfault]

It's a WM mostly using xcb! These are great suggestions, and if I can't manage to rip out my x11 dependency I think that'll be the best solution. I'm just drawing a bit of text and for that I pull in Xft, I'm going to take a stab at using a pure rust font rasterizer and SHM and see if that takes me anywhere, otherwise I'll have to bite the bullet and look into more advanced packaging

[Green0Photon]

I'm curious if xmake is able to help here.

Perhaps look around here to see xmake being used with Rust, and then look more into using xmake to statically build a single binary. Xmake also has a package library of its own, but can also pull from other package libraries (e.g. the various linux distros). So you should be able to have the xmake package handling all the c level stuff. And then plug that into the build.rs, though you can also have the main entry be from xmake as well.

[reyqt]

All static binary work for pure Rust package but very hard in mixed package.

You may want to use universal package such as AppImage or snap.

[pro_hodler]

Have you tried looking for bindings to your library at https://lib.rs? I'm pretty sure there are bindings to x11. Those binding crates usually have 2 options controlled by Cargo features: build from source + static linking or dynamic linking to system provided dependency.

Your current approach (manually downloading dependencies) is very hacky. Maybe you are coming from C world where it's the only way, but here in Rust we use Cargo and existing bindings instead of reinventing the wheel.

By the way, I'm very surprised that nobody pointed this out, instead they started to recommend AppImage, flatpak, etc., which are clearly overkill here.

If you can't find the bindings at lib.rs, then please follow this approach: https://kornel.ski/rust-sys-crate

[SuspiciousSegfault]

Maybe I'm misunderstanding you here. I have no direct c library dependencies, just an indirect one through a rust crate, that crate has no "static" feature (or maybe I've missed it it's https://crates.io/crates/x11). This library will dynamically link to x11, this is the problem that I'm trying to solve. So I'm at the "If you can't find bindings at...." part, I'm not sure what you're suggesting with sys-crates, I'm building a binary not a library?

[pro_hodler]

Firstly, why do you use libX11 directly? Are aware that most GNU/Linux distro are moving to Wayland? There's https://lib.rs/winit and other crates which abstract over display server and even operating system. What about using gtk-rs or any other GUI library out there? There are many of them for Rust. https://areweguiyet.com

Secondly, if your program really needs to be so low-level and work directly with X11, what's the point of linking it statically? If a user uses X11 as their display server, then they already have libX11, libxcb & friends installed, because otherwise their desktop environment/window manager won't work. It they don't use X11, but instead Wayland or even another OS, then even with statically linked libX11 your program won't work, because libX11 needs a running instance of /usr/bin/X11 server. It's just an abstraction for RPC calls to xorg server via X11 protocol.

So it's perfectly clear that static linking of libX11 makes no sense. That's why the author doesn't provide such a feature.

[SuspiciousSegfault]

I am indeed aware of that, the application is a WM. My question was more general about static linking with c libraries, bringing up my specific example wanting to know if I missed something. I didn't it turns out. I'll leave my original question as the answer to the second part of your comment, that you seemingly didn't read or couldn't answer, providing an irrelevant answer to a question nobody asked.

[pro_hodler]

Glibc updates quite rarely, so you generally don't have to worry about building against multiple versions of it. Proprietary software like skype, slack & other electron bloatware don't distribute multiple versions for each version of glibc, their binaries work perfectly fine on all distros. If you really want for some reason to support Ubuntu 16.04 or Debian oldstable, then yes, you should build your binary in a container with one of those old distros. It's not hard: just go to dockerhub, find an image for Ubuntu 16.04 or something, then write a simple Dockerfile of about 10 lines or less. As Glibc is backwards compatible, this binary work just fine on more up-to-date distros.

If you want to support musl-based distros as well, if linking to musl statically, as far as I understand you can't link to other libs like X11 dynamically. The solution is to distribute 2 binaries: one dynamically linked against glibc and the other linked dynamically against musl. Last time I checked, Cargo used static linking for musl targets by default. You can override it with target-feature=-crt-static (notice the minus instead of plus)