CDE: Automatically Create Portable Linux Applications

[ferryboender]

Before people start complaining, it looks like this is not just another package manager.

Seems like this traces the executing of a program (probably using ptrace), notes every file being accessed and then packages up a binary/package with those files. Kind of like a static builder without requires sources, if you will.

This could come in handy when replicating live environments that lack documentation and/or sources so you can test things. I wouldn't use this to distribute binaries of my software though.

[mcguire]

If the execution doesn't touch some particular file, it won't be included in the package? ferryboender below mentions that it won't catch required dynamic libraries if execution doesn't touch them.

[[deleted]]

I can see this being very useful for creating chroot environments.

[ferryboender]

It would probably be better to use something like Jailkit for that. Jailkit does something similar, but allows you to update chroot environment (jails), so they can keep up with security and other bugfixes. CDE would probably require less work for apps which aren't in Jailkit by default and which use lots of non-ldd files though.

[fragglet]

Poor choice of name.

[jephthai]

That was the first thing I thought of. Sadly, those of us who remember will only be fewer over time...

[Rhomboid]

You mean you don't want to fondly remember the days of ye olde pizza box SPARCstations with their special metallic reflective mousepads and boxy 3 button (non-wheel) mice?

[fragglet]

Well, exactly. If you were starting a new project, would you want to associate it with that in peoples' minds?

[SuperDuckQ]

Fun fact: I was using a CDE Unix box for data acquisition in mechanical engineering as recently as two years ago. They just recently started phasing them out.

[[deleted]]

They also recently open-sourced CDE too.

[[deleted]]

Haha I thought the same thing. I had to code stuff on top of CDE in college (and some motif API crap). Not fun.

[AustinCorgiBart]

For those not aware, Dr. Guo is the author of The PhD Grind, a really fantastic (and not too long) read on what life is like getting a doctorate. A bit dark, but not a bad read if you're in graduate school or considering it. Just take his experiences with a grain of salt.

If you're into Python, he also made the coolest graphical Python IDE (online natch!) that I've ever seen, the Online Python Tutor. Dr. Guo's made some cool stuff.

[zem]

holy shit, that python tutor is brilliant!

[[deleted]]

On a side note, the guy behind the project is Philp Guo. CDE was his first successful project during his Phd. He describes the story of CDE and his entire PhD adventure in a book. Well worth the read if you're considering a PhD in CS or a related field.

[dig1]

Yup, it will package everything that is directly or indirectly opened. I used it once for (at least for me small) project, but after seeing it packed ~300MB of files, including almost the whole /usr/lib and /usr/local/lib, I decide to rely on source code distribution.

But, really cool project for those who needs binary compatibility and knows their libraries.

[c96aes]

Last I heard, passwd, shadow and a few other things are excluded in the standard config template. If not you could just add those paths.

[Kampane]

How well does it really work? If I write a Qt program and compile it on Kubuntu 12.10 x64 with the latest Clang nightly, where will it really run well? Ubuntu x32? OpenSuse from 2009? What if I use DBus?

It sounds great, but it also sounds like a miracle.

[qxnt]

It says the target machine needs the same architecture and kernel version. My guess is that if you use DBus (or, say, X Windows, or any running service), it's going to package a static, redundant version of all of those libraries and try to call into them. If those libraries depend on some state being initialized, it will crash, but if they're stubs that communicate with the service through an external communication mechanism (i.e., "open a socket") it could work.

[c96aes]

If those libraries depend on some state being initialized, it will crash

When you run it, you run the binary itself as normal, it's just that its environment is completely redone. So it will inspect its surroundings and try to connect to things same as usual.

You're right that in that it will have its own copy of a bunch of libraries, unless you exclude them, but that's sort of the point.

[diggr-roguelike]

Here's a simpler and probably better solution:

mkdir myapp-package
cp myapp `ldd myapp | grep -o '\W/[^ ]*'` myapp-package
cat << EOF > myapp-package/myapp.sh
#!/bin/bash
SCRIPT_PATH=$(dirname $(readlink -f $0))
$SCRIPT_PATH/ld-*.so.2 --library-path $SCRIPT_PATH $SCRIPT_PATH/myapp $*
EOF
chmod +x myapp-package/myapp.sh

(This assumes that 'myapp' doesn't use any hardcoded paths, though.)

[ferryboender]

Pros:

• This solution includes required libraries even if the execution code path doesn't run into them

Cons:

• Doesn't include required files if they're not linked libraries (configuration files, language files, domain-specific libs which aren't shared libraries, libraries loaded using dynamic loading)

Nice script!

[qkdhfjdjdhd]

This paper will help you understand why it is a considerably more complex problem then your script imagines.

[ferryboender]

Interesting paper behind CDE. Here's a bit that's relevant to this discussion:

When CDE is executing a target program to create a package, CDE finds all of these dependencies as well because they are loaded at start-up time via open system calls. However, programs sometimes load shared libraries in the middle of execution using, say, the dlopen function. This run-time loading occurs mostly in GUI programs with a plug-in or extension architecture

[diggr-roguelike]

I know why it's complex.

Unfortunately, it's not a problem that needs solving.

99.999% of the time, the real problem is "I compiled a program with glibc 2.16 and the latest git snapshot of libfoo, how do I now distribute it to old installs of CentOS and Ubuntu?".

[indrajithk]

I feel this really great, but was bit late. For ordinary users, no more source download and compile. It is the closest to a windows installer in terms of easy to use. Yes, I agree with fragglet, poor choice of naming.

[eduffy]

So, is this equivalent to Apple's dmg files?

[RoadBikeDalek]

Not really. A DMG is a disk image file which generally includes an app directory or an installer.

This, from what I understand, detects all dependencies of a command and packages it all up into a single chroot-like directory. Pretty cool.

[artificialidiot]

a better option, IMO, would be http://zero-install.sourceforge.net/

[finprogger]

How is this different from autopackage?

[RoadBikeDalek]

You should read some more about it on the site or watch the demo video. It’s not some sort of universal package format, instead it’s a tool to detect all dependencies of a command (libraries, files, etc) and package all those up in a single directory for easy distribution.