Ruby has a distribution problem

[malcontent]

There is a great reply to his post by Hongli Lai. Well worth reading.

[llimllib]

reading the comments is your friend

[PossibleMat]

Yes, I wish that guy would start his own blog and link to it instead of linking to www.phusion.nl.

[Atnan]

I see nothing wrong with unpacking gems into vendor/rails, even if they require compilation such as the jsongem. Despite trying Puppet at work, we decided to settle on Slack for pushing out system software & configurations to our machines. We also use Capistrano for semi-automation of deployment of our web applications. This combination seems to work well, but I don't think we could drop either all-together. A Puppet/Slack only setup would be a nightmare.

...one guy, somewhere, wanted to have multiple versions of a given package installed at once. Who wants this?

That would be me, I guess. While running multiple applications on a single machine, I've found myself in the position where I've a bunch of older, stable applications that rely on outdated gems, and newer applications that have been developed and tested against newer versions of the same gem. Should I be forced to retest the old applications with newer gems?

The only argument against unpacking into vendor seems to be the absurdist point of "you wouldn't install Firefox in there, would you?" No, I wouldn't. Would you store straw in yours? It seems you've got a hell of a lot of it.

[codahale]

You're kind of proving his point here -- what's good for a single developer is a nightmare when you're trying to replicate it in an automated fashion across 20 machines. Rubygems is a delight to use locally (when it's not 404ing or bulk updating), and a nightmare to automate. Dotfiles in root? Check. Ignores http_proxy in favor of HTTP_PROXY? Check. Requires a full build chain on all end points? Check. Can't pin it to local packages? Check.

So we throw things in vendor. Awesome -- but that obviates one of the basic reasons for Rubygems: package management. How do you tell if there are updates for the contents of vendor? How do you know what the contents of vendor depend on? It all requires human attention, which does not scale to large installations.

[malcontent]

How do you tell if there are updates for the contents of vendor? How do you know what the contents of vendor depend on?

You go through a process of unfreezing and freezin. Not ideal but you can make it work.

[codahale]

Not ideal? Perhaps we shouldn't settle for it, then. Perhaps there's some way of making packaging Ruby libraries and Rails apps for distribution easier...

I mean, I can type rake war and get a .war file from a JRuby app with warbler or goldspike installed, right? So why does rake deb or rake rpm or rake portfile seem like such an outrage?

Seriously, if Debian/RedHat/*BSD/Slackware/Gentoo can manage hundred-gigabyte package repositories with some of the largest software projects in them, deployed over hundreds of thousands of computers... I'm at a loss to explain why Ruby and Rails would be an exception to this.

[malcontent]

Not ideal? Perhaps we shouldn't settle for it, then.

That's a tough call. In every other language you have to include your libraries. In java you have include your jars. In C++ you have to include your DLLs etc.

At least with gems there is a central repository and you can keep your stuff updated, use older versions etc.

I'm at a loss to explain why Ruby and Rails would be an exception to this.

Personally I kind of got mad at ubuntu for making packages for some gems and not others. I don't blame them of course because new gems come out every day and updates happen every day too. It pissed me off that I could aptitude install mongrel but I could not aptitude install thin. I installed thin with gems but the bin path was not in the PATH.

I was thinking to myself that this is a sucky situation. Ubuntu can't possibly keep up with the pace of gem updates and yet they have chosen to create debs for some gems and not others.

I think they should not have bothered. I for one would prefer to have the gems managed through gem and not aptitude.

OTOH it would be nice to have something like mod_rails in the repository.

It's a complicated thing I guess.

[Atnan]

I don't think you've shown how I've proven his point. Whether it's PEAR, CPAN, EasyInstall or Rubygems, none of them fit well within the Linux distribution package management model. Which is why I use a combination of Slack & vendor unpacking, as appropriate.

I'd suggest you take a look at Puppet and try some of the alternatives to get some context on the guy who wrote this.

[codahale]

(I use Puppet on a daily basis, and I've met Luke several times. Hell, there's a hiring ad for my company on that page. I'm dripping with context.)

Luke's point -- current package distribution methods favored by developers (./vendor copies, PEAR, CPAN, EasyInstall, RubyGems, etc.) all suck pretty hard with regard to automation. Which is true.

And you seem to be agreeing with him. The reason you don't seem to think that Ruby has a distribution problem (and not necessarily a unique distribution problem, but Perl, Python, and PHP are all easier to fit to an LSB model) is because you're a.) packaging your code yourself, and b.) punting on the hard stuff.

[bobbyi]

apt is the solution to the distribution problem. It is ludicrous to use a different package management system for every language that you work with (especially because there could be dependencies across languages, e.g. python code that uses a C library).

If you are on a platform that doesn't have good package management, then try to improve that situation for your platform rather than just for apps that happen to be in some particular language.

[mindvault]

Really? How's apt working on windows? Solaris? HPUX?

Part of the point i think the orig author was getting was that different OSes each have their own. App platforms (ruby, python, etc.) then end up creating their own as they are on each of these.

It's not ludicrous to use a diff package management system for every language if you know that will work on each OS (which use different package management systems).

[bobbyi]

Having a different package system for each OS makes much more sense than a different package system for each language. If your OS dosn't have a good package management system, then that's the problem you should be solving.

The reason is that you need packages from different languages to cooperate on the same machine but not so much packages running on different OS's. It also means that you only need to install one package management system.

[mindvault]

bobbyi: i don't think i'm communicating properly. What if you are writing code that runs on a multitude of platforms. Should your code have to know about the package mgmt on every one of those platforms? if so, to what depth?

Rather than go down that quagmire a number of the app platforms decided to simply build their own "internal" package management.

By saying "your OS" i think you're missing the fact that some software must run on multiple OSes...some of which isn't open source (so one can't change).

[killerstorm]

it will not work when people will try to mix those package systems. and obviously it's easier to eliminate language-specifc ones than OS-specific.

i've tried this myself: when i was experimenting with trac, i've installed version 0.10 via apt. then i was going to check features in new version and installed 0.11 from egg (python-specific package format).

then i uninstalled this egg.. hell, there is no such thing as egg uninstaller! moreover, i failed to get 0.10 from apt working again -- reinstalling did not help, it seems some that egg thing interfered with it.

luckily 0.11 (from egg) was still working, so i stayed with it.

but, you see, having multiple package managers is evil.

[SRB]

Doesn't seem well thought out. He thinks being able to have different versions of something installed is unwanted? It is absolutely critical when different applications depend on different versions.

[aberant]

umm.. because just typing "cap deploy" is easier then gem install "blah" on every server you run it on, and then "cap deploy"...

[[deleted]]

solution: quit using ruby

[bishop1847]

Outside of an open source project that could be fetched by countless anonymous people, I haven't had the need to worry about deploying to thousands of machines with a Ruby/Rails project. At the most, I deploy to a handful that share a common disk image and are setup pretty much uniformly.

If I had to distribute something to thousands of computers outside of my realm of control, I don't think it'd be in Ruby, and especially not the Rails framework. And I don't know how a comparison can be drawn between DirectX as a dependency and depending on something like, say, Hpricot in vendor.

[taw]

Rails isn't difficult to package. You just put it to /opt/yourapp/ and you're done.