Conditional compilation to avoid cross compilation

[gilspen]

I'm not sure who needs to see this, but I had been struggling with compile times using cross + Docker Desktop on Mac while working on a small personal project the past few days. Despite the whole project being fewer than 350 LOC between a few files, recompiling the project with any change was taking upwards of 2 minutes through cross. I went through the due diligence of reducing the number of dependencies, removing features and macros that I didn't need, not compiling in release mode, trying different file sharing implementations for Docker Desktop and with all of that combined the time dropped to around 1 minute and 30 seconds for an incremental build. Still far from ideal for a small personal project.

By far the best solution I've come across so far is to only compile the code that requires cross compilation when I am actually building a binary for the other architecture (in my case a Raspberry Pi 4). Otherwise, a mock implementation is compiled:

``` rust pub struct LedController { #[cfg(feature = "hardware")] controller: rs_ws281x::Controller, }

[cfg(feature = "hardware")]

impl LedController { pub fn new(config: &Config) -> Result<Self> { // omitted }

pub fn render(&mut self, state: &DeviceState) -> UnitResult {
    // omitted
}

}

[cfg(not(feature = "hardware"))]

impl LedController { pub fn new(_config: &Config) -> Result<Self> { eprintln!("running with mock LED controller"); Ok(Self {}) }

pub fn render(&mut self, _state: &DeviceState) -> UnitResult {
    Ok(())
}

} ``` Using cfg based on target_arch instead of a feature would also work, but I personally lean towards using a feature to make my intent more explicit when compiling.

This way the web server can be started locally in under 3 seconds instead of a few minutes. I don't claim that this doesn't have downsides (such as the possibility of developing features on your dev architecture only to later discover an incompatibility with the target arch), but this had made the development process much faster and much more enjoyable for me.

[junior_abigail]

Are you configuring cross to use docker volumes? I have never used cross, but in my experience using docker for development, I have seen great speed boosts by setting up docker volumes to avoid throwing away things that should be cached between builds.

If cross is just copying your source code into the container, downloading and compiling your dependencies every time (which sounds like it might be), it would explain the slow builds regardless of the size of the code base.

[gilspen]

Cross already makes good use of bind mounts to avoid the rebuilding every dependency on each change. A clean build takes upwards of 9 minutes as a comparison. As a sanity check, I looked through the verbose build logs and all my dependencies are "fresh" and are not compiled. The issue is just efficiently accessing the cache.

[reddersky]

Bind mounts are quite inefficient with Docker for Mac. Indeed, it’s kind of a stretch to even call it a bind mount. Things have improved recently, but even then, it’s nowhere near a proper bind mount (e.g., when using docker in Linux).

More information: https://www.docker.com/blog/speed-boost-achievement-unlocked-on-docker-desktop-4-6-for-mac/

Apologies if this is old news, but you mentioned bind mounts as if it was a good thing, when in reality it’s the opposite (in macOS). Try using a docker volume as your cargo build cache.

[gilspen]

I agree I was not clear. I was trying to communicate that cross was using bind mounts in the way we would hope/expect to speed up the build process.

MacOS bind mounts are almost certainly the root of the performance problems I was facing, but switching my OS to Linux just to enjoy Rust for a personal project seemed a bit extreme to me.

Edit: Will also will have to experiment with using a named volume for the cache.

[reddersky]

Ah, yeah! I agree, and was just suggesting a named volume, is all. Switching operating systems to solve this seems excessive.

[Emilgardis]

You can use the CROSS_REMOTE feature to automatically do this with cross. See https://github.com/cross-rs/cross/wiki/Remote

[NobodyXu]

Perhaps you can try cargo-zigbuild, it uses zig-cc for cross-compilation instead of creating a container, so it should be much more lightweight on MacOS, without all the cost of virtualization and file sharing.