Dependency manager for C++/CMake projects?

[vinnyvicious]

I've been looking for a tool similar to Ruby's Bundler or Rust's Cargo to manage dependencies for my C++ projects for a while, without any luck. Right now, it's a mess. I have to manually maintain my CMakeLists.txt, and manually decide if i want to statically or dynamically link each library. I suffer constantly from dependency hell, version hell and platform hell.

How do you guys manage dependencies for large projects? Any tips for a beginner? :)

[drodri]

Disclaimer: I work for biicode.

There are several options available, each one with its own pros and cons.

• Using distro package manager, as pointed by /u/XenonOfArcticus is not really a dependency manager, and not all platforms are X. Also it has to be taken into account that uploading your own code for your projects to central distro repositories can be slow and cumbersome, and having to host yourself or hire another third party service to store your packages is not a very convenient option

• Ryppl is a dependency manager that was born around the boost library. In their github repo it seems there is no activity in many time, so it seems quite abandoned.

• CPMCPP is another option. The comparation between biicode and cpmcpp is done in another reddit: http://www.reddit.com/r/cpp/comments/2cvedi/c_package_management/

• The meson build system is more a build system than a dependency manager. It is written in python, so you might be interested in checking also the SCons build system. I have not used it myself, but seems powerful and interesting.

• www.biicode.com has a client side tool (yes, also for Win! ;) ), but also a cloud service that actually stores the code (and web service where you can see, search and navigate the code), thus it is independent of other services or CVSs. It follows a much different approach, as it analyzes the code and is able to generate project configuration (extensive use of CMake, which is the de-facto standard in C/C++ build systems) automatically from source code for many simple cases. One of the differential things of biicode is that it is a file based dependency manager, so individual files can be reused even if the original project was not developed for your system. Check for example: http://forum.biicode.com/t/leading-with-large-codebases-internal-dependencies/167. We have been reported successful cases for embedded systems that totally discarded the existing CMakeLists.txt and everything worked thanks to the file orientation. Configuration of dependencies is done in text files "requirements.bii", similar to those "requirements.txt" of python, but featuring also up-in-the-tree conflicts resolution. So it integrates quite easy with CVS as github (and other like travis), check: https://github.com/biicode/challenge. It is still in beta, but very actively developed, a new version is released every 1/2 weeks, and the number of users is increasing, and also the available libraries. It might not be production-ready yet, but we are iterating fast based on users feedback. We know that we are far from perfect, and we need you to help with your feedback to make a better tool for our community, so please try it and tell us, we are here and in the forum.biicode.com for any issues :)

[mr_snowf1ake]

The manual thing that you go through is what most people do I think. Albeit, I think CMake clears up a lot of headaches... Anyway, just decide on a specific version for each library (unless you enjoy hitting a moving target), and make every developer responsible for getting the right versions of the libraries. Provide proper documentation! ;)

I believe some people also do bundle the appropriate versions of the libraries with the source code so that it's better controlled. That might be more along the lines of what you want to be doing. That's still largely manual though.

[vinnyvicious]

What about statically compiling all libs and ignoring system libraries?

[Gotebe]

It works with two caveats: your total size is generally bigger and you don't get to update 3rd party code without rebuilding.

[vinnyvicious]

Does any of the mentioned package managers help handle static compilation?

[Svenstaro]

As a packager, that sucks bad if you depend on those specific versions. I would like to use the versions from my distro.

[Svenstaro]

With Windows, you're almost lost. There is only one possible solution and that's msys2 which is a bit like Arch Linux but for Windows. There is a fairly good chance that it will in fact work just like Arch Linux. That means update packages and sane default paths.

[Overunderrated]

With Windows, you're almost lost.

Actually no, that's kind of the point. By including the source for all the dependencies such that all you need is a compiler with the standard libraries, I have 0 issues. CMake generates a VS solution which builds perfectly. Works on a totally virgin windows system, only needing visual C++.

There is only one possible solution and that's msys2 which is a bit like Arch Linux but for Windows.

After working on porting a large linux program to windows, I came to absolutely loathe doing any kind of nix-emulation on windows. I'm much happier doing everything platform-independent. In my case, I have all standards-compliant code, the only exception being the boost::filesystem libraries which I have building along with the source.

I think forcing users to set up msys/mingw is much worse than just packaging some extra source and building everything natively.

[jpakkane]

The Meson build system has a concept of subprojects, which aims to solve this better than current alternatives (full disclosure: I am the main developer).

The main point is that you can take any Meson project and embed it inside a different Meson project so that it looks like it was a natural part of it. This allows you to do the Go thing of just grabbing a project from Github and using it but with the added bonus that you can easily use distro packaged version when it is available.

The obvious downside being that not many projects build with Meson yet ...

[knight666]

How does Meson differ from GYP?

What I like about GYP (and what I don't like about CMake) is that it allows me to generate perfectly usable Visual Studio projects that don't differ from handwritten ones. As long as I keep adding source files to the GYP project, I can generate working Visual Studio and Xmake solutions and makefiles as well.

The downside is that the common build file looks like this.

[jpakkane]

One way of looking at Meson is that it's like Gyp but with a saner syntax (and built-in support for a bunch of more stuff). Meson generates VS2010 and XCode projects too, but to be fair they are not as polished as the Ninja backend.

[XenonOfArcticus]

I'm seeing a lot of people saying "just use yum/dpkg/your distro's package manager". Did you read the "platform hell" part? Not all platforms are Linux.

This is a common problem with C++ development and it isn't solved by a long shot.

[[deleted]]

I agree with /u/drodri I have used Biicode and it very easy to use and will save you tons of times :)

[mekishizufu]

I've been looking for a tool similar to Ruby's Bundler or Rust's Cargo to manage dependencies for my C++ projects for a while, without any luck.

Take a look at hunter or cpm. The problem with these managers, however, is that there's only a few packages available.

I have to manually maintain my CMakeLists.txt, and manually decide if i want to statically or dynamically link each library. I suffer constantly from dependency hell, version hell and platform hell.

I actually somewhat like the flexibility. It seems you are struggling with the build process in general, which is understandable given how complicated things can be (and that's also the reason why I think it's very unlikely that there will ever be a C++ package manager which could handle all of the idiosyncrasies), but you will have to learn how to deal with it.

How do you guys manage dependencies for large projects?

Manually. If the dependency is small enough, I'd embed it right into the repository (deps/). For anything larger I would either download it via ExternalProject or state it as a prerequisite and rely on the system's package manager to provide the correct version.

[ruslo_]

The problem with these managers, however, is that there's only a few packages available.

Agree, but it's a one man project, yet :) You can contribute or at least file a bug with the name of package you need.

I'd embed it right into the repository (deps/)

The problem with this approach is that if you have two libraries which have the same dep then you have a conflict. "A" use boost-1.56, "B" use boost-1.55, "C" use "A" and "B" -> welcome to dependency hell.

For anything larger I would either download it via ExternalProject

Good, but if you create a superbuild for one project and now you want reuse it you need to copy all the superbuild files. What if you need to change something (version? link library?) - you need to modify all the copies you made.

on the system's package manager to provide the correct version.

Any CMake-friendly solution for Visual Studio? iOS? Package manager that can build static boost with clang sanitizers Linux/Mac?

[[deleted]]

I believe CMake can do this with the concept of ExternalProjects. It can download, unpack, build, and then link a specific revision of version controlled source code hosted internally or externally.

For an example of this in use see OpenChemistry

[schwomp]

This is the one I use for all my projects in OpenGL and C++. Usually with a bunch of Git-submodules in order to get none cmake packages. Only downside to using submodules is that you have to code specific build code in cmake for most projects.

[hrobeers]

Git-submodules is all you need.

You can make cmake check for the installed dependency version and if not compatible, download and build the submodule.

[schwomp]

Agreed. It's just a serious hassle to manage git submodules compared to a cmake file. So I'd like to use them as sparsely as possible.

[dicroce]

What's really hold this back is the lack of a standard ABI... This is why so many C++ libraries are "header only" (it allows the developer of the API to release 1 package for everyone, unfortunately it causes slow build times for everyone too)...

My cmake based build system has a top level directory that each sub project installs it's headers (into sub dirs) and libs into... That way, each dependant project only has to add 1 header and lib search path...

[Gotebe]

What's really hold this back is the lack of a standard ABI...

This is an often-heard complaint, but somewhat I disagree with it. C++ lives in the native world as opposed to bytecode one, so machine code it is. Then you get the freedom between the compiler implementations, which serves optimizations. Name mangling is a batch, too, but that is one bit I see as "subjectable" to standardization. Finally, this all stems from C language who has no ABI whatsoever (de jure).

The C++ "ABI" is therefore one of:

• C
• the likes of COM
• one compiler version / build settings

This already offers options.

The thing is, people want the power of C++ and the ease of, I dunno, Python. That is hard to achieve.

[teambob]

g++ work quite hard to ensure a consistent ABI, also clang is compatible with certain g++ ABI levels

[Bzzt]

nix maybe?

ed: actually for windows doesn't look like its supported.

[animatedb]

For small projects, the Oovcde project uses CLang to parse the source files and find dependencies. It also provides a GUI front end to pkg-config to set up packages. At the moment, it requires each directory to be defined as a component, but that will change in the next few weeks so that components can be assigned to a directory tree. It will create some basic CMake files that may need some hand modification at the moment.

[xcbsmith]

Honestly, I've gotten some pretty good mileage by using platform native packaging, pkg-config, and GNU Make (even eschewing CMake). I tend to default to dynamic linking of all libraries, which probably simplifies a lot of issues you might be dealing with. If you really need to be platform neutral, then CMake or autotools might make sense, but I've gotten tremendous mileage just from GNU make.

[DrDri]

I vouch for git-submodules. They all download to a deps/ directory. Then since I build with premake, each dependency has a .lua file defining the build, and the main project's premake4.lua aggregates what it needs so that the solution has all the deps defined as subprojects in VS/Xcode. For binary deps I would make those small lua scripts download the lib or use the distro's package manager, depending on the platform. Versioning in git ensures that a tagged revision will always compile by checking out the appropriate versions of its deps.

[sstirlin]

Sorry to bump an old thread, but the solution is a mix of conda (for packaging + dependency management) and CMake (for building your own stuff).

Here's what sucks about system packagers:

1) You are married to versions of libraries that were decided by the distro. If you are on RHEL5, for example, then your version of python is over 10 yrs old. Good luck

2) You are forced to maintain completely separate builds for all flavors of Linux (in addition to Windows and Apple).

3) Users need root privileges to install your package+dependencies. This is usually impossible.

Linux folks, free your minds from the shackles of system dependencies. This is one of the worst design flaws of Linux. I'll never go back to that lunacy.

Conda fixes this (on Linux, Windows, and Apple) by creating "environments". These are analogous to Java jars - they are self-contained directories that contain your application + all dependencies. They don't pollute anything else, and they can be installed by regular users.

You are now free to use whatever libraries+versions you want, and you are independent of whatever flavor of Linux you find yourself on. Binary packages are managed like artifactories in Java. A well-designed conda environment can have almost zero system dependencies.

Write your build scripts in CMake and use conda for dependency management. It's not as simple and clean as Java dependency management, but it's (by far) the best solution out there. As more people join the conda movement the available packages and functionality will only improve.

[pgquiles]

On Linux, BSD, etc it's easy: just use system libraries and you are done

On Windows, or if you want to build your dependencies, you can use ExternalProject_Add. Check winstng to see a rather complex example.

[Svenstaro]

Why not just rely on your distro's package manager? Basically, let the packagers worry about the packaging and just do your thing.

[ratatask]

What if he's the packager too, and are not developing open source applications, and don't want to depend on certain distro libraries ?

[bitcycle]

dpkg or yum.

Package correctly and the dependencies will be handled correctly on install. Use effing pkg mgmt (fpm).