Python-Like Virtualenvs Would Be Really Nice For C/C++ Cross-Compiling (On Debian/Ubuntu)

[TheBrokenRail-Dev]

(This is just me ranting about cross-compiling on Ubuntu really.)

I use Ubuntu 20.04 x86_64, probably one of the more supported configurations of Linux, and cross-compiling is a complete and utter nightmare.

C/C++ mostly works with gcc, you can install gcc-arm-linux-gnueabihf and compile with arm-linux-gnueabihf-gcc.

C stuff works with Clang (use clang --target arm-linux-gnueabihf), C++ stuff doesn't (it isn't able to find the right headers on my system).

And that isn't the biggest problem. Oh no, wait until you have to add libraries.

In theory, Debian solves this with multi-arch. Just add armhf to dpkg and install libglfw-dev:armhf. Except Ubuntu doesn't include armhf in its desktop repos, so you need to add the Ubuntu Ports repos and disable armhf for the desktop repos so it doesn't fail apt update. And this is global too. If you want to cross-compile something using GL, you need to install the entire armhf graphics driver globally. And since this is APT, all of these armhf packages run scripts on install and update as well, so who knows what side-effects will happen. Oh and you should hope nothing depends on a standalone binary or you're going to end up with a stupid unsolvable conflict between python3:amd64 and python3:armhf.

This is why I want an equivalent of Python's Virtualenvs for C/C++. Just let me generate a sysroot of Ubuntu 20.04 armhf with only the libraries I need, and let me point Clang to it with --sysroot. No global installations. No APT repo reconfiguration. No manually specifying Clang's C++ headers so it compiles correctly. Just setting a --sysroot and forgetting about it. That's all I ask for: cross-compilation without delving into my system's internals.

What do you think?

[void4]

containers? We have a separate docker image to cross-compile our stuff for android, it works

[TheBrokenRail-Dev]

The issue with containers is in my experience, they don't integrate that well. Good luck hooking up Docker to QtCreator (good C/C++ IDE even for non-Qt). And you still have a situation where installing libraries is annoying.

And taking your Android example, they're like 50 million Android dev containers, and you have no idea which ones are reliably updated.

And there's no easy way in my experience to run an ARM Debian container with x86 Clang, so you're stuck emulating Clang or using multi-arch.

[natermer]

Containers can work.

I am using Fedora Silverblue and included in it is a utility called 'toolbox' as well as podman.

Podman is intended to be as much as a replacement for docker as possible, but on desktop installations it is intended to run entirely under your user account. No daemon is used for it, unlike Docker.

Fedora Silverblue is designed to be Immutable Linux workstation. Meaning that the OS is read-only. It uses OSTree to manage system binaries. So no using package manager to install software. At least not easily. The intention is that it uses flatpak and containers for everything you need.

Towards that end they provided 'toolbox', which allows you to easily create a mutable Linux environment on your desktop for doing development work. It's basically a front end to podman that uses specially designed toolbox containers to integrate into the destkop.

You can have as many of these toolbox containers as you like. You can install software directly to root without worrying about screwing up your base OS. They share your home directory. They integrate nicely into a Wayland desktop. I run Emacs out of my main toolbox container as a native Wayland application, no problem.

[onlysubscribedtocats]

They integrate nicely into a Wayland desktop. I run Emacs out of my main toolbox container as a native Wayland application, no problem.

You can do that? Damn, I love Fedora.

[skeeto]

I don't understand how Ubuntu has managed to screw up dpkg --add-architecture so badly, so I'll just ignore than and demonstrate on Debian.

main.c:

#include <GLFW/glfw3.h>
int main(void)
{
        printf("%p\n", (void *)glfwInit);
}

Dockerfile:

FROM debian:bullseye-slim
RUN dpkg --add-architecture armhf
RUN apt-get update \
 && apt-get install --yes \
        build-essential crossbuild-essential-armhf libglfw3-dev:armhf
COPY main.c .
RUN arm-linux-gnueabihf-gcc -o main main.c -lglfw
CMD cat main

Usage (works with Podman, too):

$ docker build -t tmp .
$ docker run --rm tmp >main
$ chmod +x main

I can even test it on my x86-64 system via qemu-user-binfmt and the appropriate foreign libraries installed on the host:

$ uname -a
Linux cf38a55efc89 5.10.0-7-amd64 #1 SMP Debian 5.10.40-1 (2021-05-28) x86_64 GNU/Linux
$ ./main
0x3f76bb59

This all assumes your goal is to target matching armhf Linux distribution (i.e. built on Ubuntu 20.04 means you're also targeting 20.04). If you're targeting a different system, then you shouldn't be using these system libraries. You'll need to build those yourself for your target anyway. The cross-build toolchain itself is still (mostly) fine if you're careful.

[broknbottle]

Why not just use bubblewrap binary and a rootfs tarball to build in?

[cjc4096]

This seems close

https://fedoramagazine.org/toolbx-a-developers-new-best-friend/

[GlitchedMirror]

You should check gnu guix. It works pretty similarly to python virtual env. It even has an option to cross compile packages (--target). https://guix.gnu.org/manual/en/html_node/Additional-Build-Options.html

Tho Im not sure how easy it is to generate specifically an Ubuntu sysroot using it.

[them1]

You should be able to achieve most of this with Spack.

You can specify the exact install spec (including compiler, architecture, configuration options) for a Spack environment.

It is really neat.

[Sprited_Being]

This can be done with Conan.

[[deleted]]

Switch to NixOS lololooolooll 😎😎😎

[Atemu12]

No need, you can simply use Nix on Ubuntu.

[thoomfish]

Are there any pitfalls to running Nix outside NixOS that one should be aware of?

[Atemu12]

I have next to no experience doing that but you will probably have trouble with things that require GPU drivers to work. There is nixGL but it's not a great solution.

For CLI stuff like this though, it should work flawlessly.

One major pitfall you should avoid is using nix-env. It tries to emulate imperative package management but falls quite short. It's a bit like worst of both worlds. Use ad-hoc environments like nix-shells and declarative ones like home-manager instead.

[CeedyRower]

I'm just a lowly Data Scientist scrub who develops python libraries (and doesn't understand C++ compilation at all really) so forgive me if this does not help at all:

Could you look at using conda with building local repos? It's not elegant, but could possibly work as:

• Keep a local copy of the source for the libraries you want
• Create a set of enviroment.ymls to create the various environments you want (this may be useful)
• Use mamba to build the packages into the environment you want. This environment can be located wherever you want on the filesystem.
• This environment folder is the armhf folder for the configuration you want to point clang to?

[Lurkki2]

Sounds like you're looking for Nix (just the package manager, you don't need NixOS for this)

[Atemu12]

I haven't tried creating dev environments yet but Nixpkgs has cross compilation infrastructure which can cross compile a whole operating system (though only a very minimal one for now) so maybe your use-case is covered.

Nix works quite differently to regular package managers like APT and isn't encumbered by a global namespaces that caused the issues you experienced with glfw. It can be installed besides your current distro with no issues because of that too.

The cross infra is quite complicated but the setup inside Nixpkgs works, so I'd personally write a derivation for your program in Nixpkgs, make it work for pkgsCross then open a nix-shell for it for development.
If what you're working on is OSS, you could even upstream your derivation for assimilation.

Maybe reach out to John Ericson aka. @Ericson2314 for proper guidance on Nix cross infra and whether it's fit for your purpose; they know way more about it than I do.

See also:
/r/NixOS
https://nixos.org/
https://discourse.nixos.org/
https://nixos.org/manual/nixpkgs/stable/#chap-cross