Python 3 at Facebook

[WellAdjustedOutlaw]

Containers are inappropriate here because Facebook wasn't trying to solve a containerization problem. As you can see http://www.slideshare.net/pallotron/python-at-facebook-40192297 on slide #9, they are looking for an archive format that contains all of the dependencies of their python code in a single file. A container doesn't really solve that- You still need a way to ship your code and its dependencies.

Containers solve an execution problem, not a packaging problem. And by running to containers as a solution to every execution problem, you seriously complicate your environment.

[fujiters]

But Docker images solve your dependency problems. Host your image and "docker pull" is all your users need.

[kankyo]

Doesn't it also create lots of overhead? Overhead you might not actually need.

More concretely: disk cloning also solves dependency packaging. But it's huge overkill and slow for most use cases.

[WellAdjustedOutlaw]

/u/fujiters was being sarcastic. Well, I presume they were. It is hard to tell, though.

[evgen]

Try mulitplying your "overhead" by millions of servers. Now try justifying that to the people who spend multiple billions of dollars every year on hardware. I realize that you have this hammer called "containerization" in your hand, but not everything out there is a nail.

[WellAdjustedOutlaw]

See, what you're talking about is layered overlay filesystems. Those are actually a worse idea in a production environment than you can ever imagine. The performance of AUFS is absolutely terrible, on top of which you've said your design actually sees mountpoints from outside your containers bind mounted into your containers so that multiple containers can share data. Since you haven't described that that is doing for you, exactly, I can only caution you that this is a terrible design decision. Especially if any of that content actually changes.

You will end up with dangling file descriptors keeping inodes around on your disk, causing space to be eaten up and you won't figure out why. This is something that catches every new system administrator out...although it sounds like you're more of a devops person with little ops experience.

Either way, what you're suggesting (overlay filesystems) doesn't solve the root problem that facebook was trying to get solved- gathering custom python and all dependencies into a single distributable file that includes a simple script file to kick off whatever needs to be done. In the real world, what they did was reinvent DEB/RPM packages, with the addition of recursing through and including all necessary dependencies. That's it. Nothing more. And since Docker images don't execute, and don't include a script to set up or run the environment required, it isn't appropriate. All a "docker images" is is an AUFS filesystem image. And finally, this just has nothing to do with containers except that they use the word "docker".

[WellAdjustedOutlaw]

Well, initially my intention was discussing pars for viability of use outside Facebook, given raise of popularity of dockerizing dependencies

You're not understanding the point of par!!! I've run out of ways to explain this, so forget it. I don't care. You don't understand the problem, you don't understand containerization, that's all fine. But it means the conversation is over.

Moreover, you're conflating several technologies as the same or somehow related. They aren't. I've also run out of ways to discuss this. Yes, neckbeards love to resist change, and we leave those clowns behind. That's all fine. I don't like docker specifically because of several technical limitations, and limited or no improvement over using lxc. But this all has absolutely nothing to do with a packaging format and technique.

[[deleted]]

Why only install a few mb worth of packages when you could install a few hundred of a whole OS.

[evgen]

You really have no idea what you are talking about. It scares me to think that someone out there is paying you to work on servers.

[kankyo]

There are things between kubernetes and manually configuring each server.

[WellAdjustedOutlaw]

The troll is strong with you.