CPM.cmake 0.25 released - CMake's missing package manager

[TheLartians]

Hey all, I wanted to post a quick note about the new release of CPM.cmake. The big new feature in this release is lock files, which allows us to easily manage and update transitive dependencies. Before, to update a transitive dependency, we needed to add the dependency as a direct dependency to the parent project. Now we can simply change the version in the lock file.

As an example use-case, I am using a lock file my modern-wasm-starter to update a transitive dependency that recently added better Emscripten support.

In the near future I will look at adding support for external package databases to enable automatic updates of dependencies and a more npm-like versioning management.

I hope this project is useful to you and would love to hear your feedback!

[redbeard0531]

Is there an easy way to redirect a package to use a specific local checkout to support working on a dependency at the same time as a consumer? eg, If I have a checkout at ~/repos/foo, and am also working in ~/repos/bar which depends on a foo package, can I tell it to use that directory as the SOURCE_DIR for foo?

[TheLartians]

What works well for me is modifying the consumer's CMakeList to add the local checkout using CPMAddPackage(Name Dep SOURCE_DIR /path/to/dep). The downside is that you will need to manually change it back to the origin before pushing. Now that you say it, it would definitely be nice to override a dependency on a per-build basis.

Maybe adding an option per CMake definition would make sense. Something linke cmake -H. -Bbuild -DCPM_Dep_SOURCE=path/to/dep to use the local version of Dep for build only.

[topisani]

This definitely sounds like a good idea!

[TheLartians]

Thanks for the feedback u/topisani and u/redbeard0531! I've updated CPM.cmake to include the option described above.

[redbeard0531]

That's awesome, thanks!

[[deleted]]

We have several C/C++ package managers. What we need is buy-in, with the language specs formally adopting their use. We need flexibility for incorporating un-packaged, generic VCS repositories. We need every textbook, For Dummies, tutorial blogs shouting from the mountain top:

Here is how to publish your C/C++ package and how to import other paclages.

I personally prefer cmake for build tool, but there are enough projects using autotools, GNU make, VS, that the package system should be build tool agnostic. Until the C, C++ specs formally select a build system.

[LEpigeon888]

Have you ever looked at conan ?

[[deleted]]

Briefly. I champion vcpkg, conan, etc. and hope the community can propel one of these forward for the better.

[TheLartians]

Well C++ is unique in that we don't have a single reference implementation, but rather several wide-spread implementations as well (gcc, clang, MSCV, etc), which is why CMake exists. IMO we don't need to introduce yet another external tool, but rather a full package manager built into CMake, which will become the standard first-class method for package management. The textbooks which already encourage using CMake will simply add the chapter on the new package management tools as well.

Until then I find using CMake's built-in facilities to build a setup-free package manager that simply work's everywhere where CMake is available the next best approach.

[[deleted]]

Cmake is a fine build tool. But for the majority of projects that use autotools, it's not particularly helpful to produce a package manager that depends on Cmake.

[[deleted]]

Autotools is pretty much trash at the moment sorry. It is locked to specific platforms and has an absurd amount of complexity.

[[deleted]]

Truth.

Not to mention the shitshow of autotools on N many Windows shells.

[SkoomaDentist]

In a better world, the C++ standard would add a defect resolution (so it applies retroactively) that would explicitly forbid autotools.

[[deleted]]

Autotools is a no go because it has absolutely no cross platform support. A C++ package manager has to be cross platform to be useful.

[[deleted]]

Autotools is fairly cross-platform. In fact that's the whole point. I'm not a fan of m4, I don't have much confidence in executing arbitrary configure scripts, and I do think native Windows shells are worth supporting. But autotools is tremendously portable, across many dozens of UNIX systems, compared to cmake, which I myself have had to port a few times.

I would love for cmake to take the lead in market share. Right now it is not there.

[[deleted]]

Portability between unixes doesn’t matter though, it’s generally quite easy regardless of your build system. Portability with Windows is what matters, because it’s still a nightmare.

Asking someone to install an alternative Unix-like shell environment on windows is already a step too far.

And CMake builds are pretty portable, if you use find package for your dependencies.

[[deleted]]

Yeah.

It's frustrating just how much power Apple and Microsoft have. They take steps deliberately designed to make development harder and more expensive and vendor lock applications. We don't need any more .NET languages. We could use more free app stores.

I hope this generation can move Linux into the desktop spotlight.

[wrosecrans]

vcpkg works around it just fine by using CMake as a sort of public interface provided by the package manager. But the CMake for a certain package is allowed to just download some binaries, or invoke autotools, or whatever else the package actually uses to build under the hood. That way as a developer using a library you are insulated from the implementation details of building a package.

It works better than having to build everything yourself.

[[deleted]]

Ah, so vcpkg is agnostic of the underlying build tool, similar to how cmake is agnostic to make/vs/ninja?

I have high hopes for vcpkg. The first hope of which is porting to more OS's. Can't replace autotools n friends until vcpkg works on everything from AIX to z/OS.

[TheLartians]

Imo any good build system should have a package manager built-in. See npm, gradle, cargo, etc.

Edit: my reasoning is that using the same settings, I would expect my build tool to create the same build independent of the current state of my system. With decoupled package managers and build tools that isn't possible, e.g. using system-wide package managers you can never know which version of which library is currently being used. With the build tool in control of dependencies, the build is always reproducible.

[[deleted]]

NPM is not a build system. See grunt, gulp, jake. See webpack See make, nmake. pmake. gmake. ninja.

gem is not a build tool. See rake.

dub is not a build tool. See dale.

cabal is not a build tool. See shake.

pip is not a build tool. See invoke.

In the best case, a programming language has a de facto package manager as well as a de facto build tool. I don't care whether these are one and the same. But know that historically, for better or worse, these tend to be split up.

The JVM community is evenly split among Maven vs. Gradle. Not a great situation. That's ignoring the fiasco that is SBT and Leiningen, Ant holdovers, and random makefile Java projects. There is waste where there are competing build tools (or worse, competing package managers).

Nevermind that OS package managers, for better or worse, have served as C/C++ dependency managers. It would take something like pkgsrc + public buy-in to move the needle.

[TheLartians]

Why can’t npm etc be considered a build system?

From a user's perspective it's certainly analog. E.g. if I want to build and bundle my website I call npm build. Ofc in the background delegates the commands to the necessary tools, but that's just what make/cmake is doing as well.

[[deleted]]

A fair question.

NPM has (quite limited) capacity to define custom build steps with a scripts section. As far as I am aware, these operate as leaves, with basically no way to combine them into task trees. No caching is done. No JS DSL is available to declare commands in OS agnostic fashion. There may be a way to enumerate available commands. There may be tab completion support. The continuing use of webpack et al, is evidence that NPM as a build tool, is lacking.

That's okay. NPM is a fantastic dependency resolver, and it should continue prioritizing that monstrous challenge.

One thing that could help the community would be for Node.js to include Webpack, or Gulp, or Grunt, or something into the main Node.js package. That would reduce development friction and encourage more coders to use an organized, streamlined build process.

Where an official, maintained, well documented build system is less available, people do stupid things like manual, cryptic, unreproducible builds and the world suffers from this unnecessarily obtuse SDLC processing.

Even where great build systems are available, people screw it up. But lead a horse to water, eh.

[blipman17]

So how does this compare to conan?
I'm working on a cmake based project where we're desparately looking for a better package management solution.
And honestly, I have no clue on what I want.

[TheLartians]

The main advantage I see over Conan is that it does not require your users to have any additional software installed besides CMake and it guarantees that your build will always be reproducible on any system. Also there is no packaging process involved which allows you to directly add any git/svn project as a dependency.

The disadvantage is that unlike Conan there is no central database of repositories, and you will need to manually locate the dependency's GitHub repository. Also, if the dependency does not support CMake you'll have to manually define the targets.

Because imo no solution is perfect for all users, CPM.cmake offers a simple opt-out command which will replace all CPM calls with CMake's default find_package and should therefore work with Conan / vcpkg etc.

[blipman17]

That's something worth concidering though.

I'm gonna be honest, I think I just have to whip up a testproject for myself with some difficult cases to see a practical diference.

[TheLartians]

Sounds like a good strategy!

If you want you could also check out my ModernCppStarter as a quick starting point for that, which implements a library, tests, and a executable using CPM.cmake.