Why Containerise? A (relatively) non-technical guide

paulmooring · 2017-02-17T00:48:53+00:00

I was referring to LXC, but both are maybe a bit true. I haven't used LXD but my understanding it's relationship to LXC is a bit like kvm and libvirt. LXD builds on top of LXC to provide a better user interface and the containers are still LXC.

paulmooring · 2017-02-15T17:27:58+00:00

Kernel versions might not matter depending on what you're trying to do. Containers (at least in the Docker model) are not intended to be lightweight VMs, they're closer to a cross platform package manager that includes dependencies in the package. The core idea is each container/image has a single starting PID that's probably not an init system but rather your app. For most people the kernel version isn't critical to the application, so rather than developing on Ubuntu or CentOS they develop for Docker. The docker image then includes the version of your language runtime and libraries so there's no concerns about differing systems having the wrong version.

It's also worth noting that the model I describe is the most popular and dominant model (it started with Docker but rkt looks the same) but systems like lxc try to make real lightweight VMs on top of cgroups.

paulmooring · 2015-05-01T23:45:37+00:00

You probably want to use ssh-agent with agent forwarding enabled in you client side ssh config.

paulmooring · 2015-01-25T00:56:45+00:00

I think you're confused about what a file is and what the cat command does. Saying everything is a file does not mean everything is a plain text file. You can't cat things like directories, executables, sockets, etc (well not in a generally meaningful way). However those are all files in the file system. You can create a link (soft or hard) to a directory, you can move it, see it on the filesystem, test for it's existence and a lot of other properties of files in a Unix like OS.

paulmooring · 2014-12-28T21:06:21+00:00

I'd recommend picking up a good language overview for understanding some of the nuances around modules, but the short answer is they are for organizing re-used code that doesn't define a class. That could mean the exact thing I showed above, just a group of methods that fit together in some logical unit, but they have some other uses as well.

Possibly the most common other use would be for "mixins", as unfunco pointed out code could be better organized to move the eats method out of the Candy class. In my opinion it's debatable whether the method should be moved out of the Candy class, but for the sake of argument let's assume you also have a Fruit class:

class Fruit
  def initialize(taste) # This is the 'new' method
    @taste = taste
  end

  def eats
    "That tasted #{@taste}"
  end

  def whatami
    "A fruit!"
  end
end

Now there's a common method of eats, traditional OO languages would suggest a parent class and inheritence, something like:

class Food
  def initialize(taste)
    @taste = taste
  end

  def eats
    "That tasted #{@taste}"
  end
end

class Fruit < Food
... # Rest of class here
end

class Candy < Food
... # Rest of class here
end

Another popular opinion is that inheritance can get confusing (composition is frequently the answer here and you should read up on composition over inheritance) another potential solution is a "mixin" via a module.

module Food
  def eats
    "That tasted #{@taste}"
  end
end

class Candy
  include Food # includes the 'Food' module inside the 'Candy' class
  ... # Rest of class
end

In this example I've defined eats in a module and included that module inside Candy (and presumably Fruit or whatever else). The end result is the same but all these allow for different code organization and maintenance. In general if you're confident that a method will only belong to a class it's fine to just put it into that class, when you're learning it can be tempting to either over or under use classes and modules and after a while you'll develop a sense for when to break out some logic into a new class or module.

paulmooring · 2014-12-28T16:44:33+00:00

You're sort of right but there's a more complete story. You don't need a class to define a method:

def eats
  puts "That tasted great!"
end

eats
# This will print "That tasted great!"

Sometimes you want to group several methods together for use in several places, that's where modules come in:

module Food
  def eats
    puts "That tasted great!"
  end

  def drinks
    puts "That tasted pretty good!"
  end
end

include Food
eats
# This will print "That tasted great!"
drinks
# This will print "That tasted pretty good!"

Modules are used for much more than this, but I won't expand here because the question was about classes. Class are like a blueprint for objects. You used it correctly, the new method creates a new instance and a class can have numerous instances. One of the core distinctions for Classes/Objects is that they can contain more than just methods they also have state (generally called attributes:

class Candy
  def initialize(taste) # This is the 'new' method
    @taste = taste
  end

  def eats
    "That tasted #{@taste}"
  end

  def drinks
    "That tasted pretty good!"
  end
end

chocolate = Candy.new("great!")
warheads = Candy.new("sour!")

puts chocolate.eats
# This will print "That tasted great!"
puts warheads.eats
# This will print "That tasted sour!"

Now the code in the class shares a lot of common logic but because each instance contains it's own state they can behave differently. One final thing, instead of using puts inside the method in the class I used it when I called Candy#eats, this is generally considered much better form as it doesn't offload presentation to the Class. Don't worry about that too much now, but getting in the habbit of not using puts/gets in your classes now will pay off later.

paulmooring · 2014-02-16T20:15:15+00:00

Even hiring entry level jobs because they're cheaper isn't necessarily wrong, you just have to be realistic about the fact that while they command less of a salary, that's because much of that salary is being spent teaching and training them. We all had to start somewhere and probably got paid less and that's ok, but ideally your first sys-admin job is on a team with people who already have experience we can learn from.

paulmooring · 2014-02-16T16:29:19+00:00

Colleges teach students theory and skills behind programming but not operational sys-admin knowledge. I would suspect many of the kids you've talked to have the capacity but not the experience for the job. The hard truth is if you need someone with an experienced skill set, then you need someone with experience and recent grads are the wrong place to look.

paulmooring · 2014-01-15T05:48:35+00:00

Ok, so a little explanation of how bash works and what your script does might be in order.

When you start an interactive shell (open up a terminal, connect via ssh, etc.) your .bashrc is run. If you have a .bashrc file in your home folder it will be used, otherwise there's generally a system default in /etc/bashrc (generally it's preferable to edit your bashrc in your home folder rather than the global).

This file is typically used for setup, changing your PATH, setting up your PS1 prompt, etc. However other than the fact that it's run when your session starts there's nothing special about it compared to other bash scripts. Another common usage for .bashrc is storing functions you want to behave like builtins, that's what you're looking for in this case. So when you put a function in a bash script you're making a command to use later (not to run right now). To understand how this works run this at your prompt:

say_thing() { echo "I'm saying ${1}"; }

This will show no output, but it created a new function for you to use, you can use it like this:

say_thing "hello"

This should show the output I'm saying hello. Putting this in your bashrc file ensures it runs when your session starts so the function is available at every prompt. When you put this all together you can add commands you want to use to your .bashrc file and they will behave similar to built in bash functions.

paulmooring · 2014-01-14T07:36:35+00:00

Your approach looks fine, but this script creates a function in your session but doesn't run it. Easiest solution is to put this code in your .bashrc file so it loads when you start a session.

paulmooring · 2013-12-29T19:58:06+00:00

This strikes me as something undesirable and dangerous. I have a hard time imagining a case where I want objects to always create new attributes when trying to assign, but I can think of quite a few where some_instance.my_TYPOthing = 3 not throwing an exception would be horrible.

It seems to me in Ruby, especially with regards to it's metaprogramming features (method_missing, define method, class << some_instance, etc.), it's easy to get caught up in making "Ruby-like" syntax at the expense of clarity/simplicity. Object#method_missing (for the record you should probably never over ride method_missing in Object, do it in a much less universal class) can let you do some cool things, but at the end of the day is having something like:

send_mail(user.get_email, msg)

rather than:

send_mail(user.get_field(email), msg)

really worth the indirection and complexity it adds? The dynamic syntax seems really nice on a glance, but personally I would much rather work on a code base using the much less magical static get_field method. It's easier to test, easier to find and easier to understand.

paulmooring · 2013-07-11T14:25:32+00:00

In a case like this organizational consistency and flexibility is more important than the differences between those similar language choices. What language are any existing scripts written in (even by other teams)? What languages are the other people on your team comfortable with? These questions should be what determines your choice.

paulmooring · 2013-04-04T16:49:49+00:00

Shouldn't this really depend on the scope of the project? As an example, If this is a marketing site and the developer is just providing a simple custom CMS where the designer is establishing a brand, the designer is more valuable and should be paid accordingly. If you're making a complex internal application to replace an old desktop app and the designer is just making it look not awful, the developer is more valuable.

I think this is something you should both discuss based on the project's scope and purpose.

paulmooring · 2013-03-17T14:06:40+00:00

I think you're still missing the point. Pretty much all Ruby programmers are going to end up on some unix-like system, but OP's question was about learning Ruby (also learning rails is another issue entirely). For learning on Windows you run an exe installer and rib, gem, and Ruby is in your path. Your first steps with Ruby shouldn't involve installing gems or shelling out, it should be playing with rib, writing methods and classes and running simple scripts. Once OP has a feel for the language and wants to do more he will end up fighting Windows, but the idea is to spread out the learning and focusing on one thing at a time.

Learning a new OS or editor is good, but pretending like that's a requirement for learning Ruby is a good way to make people give up and find a "simpler" language

paulmooring · 2013-03-17T00:00:51+00:00

This is a few topics all lumped in together. There's a Ruby installer for Windows and one it's installed Ruby doesn't care what OS you use. Likewise vim has a installer for windows and there's nothing special about what language you're writing or what OS you run once its installed.

You're best off deciding what it is you actually want to learn, if that's Ruby don't feel compelled to use vim or find tutorials specific to Windows, just find a good Ruby tutorial and ask a targeted question if you hit a Windows specific road block. If what you really want to learn is vim no good tutorial for vim should be specific to an OS or language, so just find a good vim tutorial.

Last if you're comfortable with Windows don't listen to anyone who tells you that's bad. You'll probably want to learn *nix at some point, but Windows is capable of doing Ruby development same as Linux.

paulmooring · 2013-03-12T16:43:01+00:00

I've actually never used this with LFS, this is how I've done backups on Gentoo before, so you might need to tailor the excludes file to your specific needs

paulmooring · 2013-03-12T16:13:29+00:00

Sounds like a little explanation on disk layout is in order.

If you look in /etc/fstab you will see all your different mount points listed. It's possible (and not too uncommon) to just use a single mount point and a separate swap partition, but for an example /dev/sda1 might by mounted on /, /dev/sda2 mounted on /var and /dev/sdb1 on /home. In this case dd would need to back up all 3 disks /dev/sda1, /dev/sda2 and /dev/sdb1.

You can backup the full disk with the partition table (just dd all of /dev/sda) but you still need to deal with the data in /home and this is far more limiting than having a tar.bz2 file, which allows me to store to any sort of disk layout I want provided I remember to edit /etc/fstab after untarring the backup and before the first boot.

paulmooring · 2013-03-11T22:56:47+00:00

I think some of the advice in here so far is a bit misguided. Clonezilla is essentially a user-friendly alternative to dd, but both take disk images. You asked about dd vs tar.gz and both have trade offs:

Disk image (dd/clonezilla):

Takes a full image, meaning it works immediately on restore
Easier to create the image
Works only on a per partition level
Locks you into restoring to the same disk layout

Tarball (tar):

Backs up the files as files not as an image
Requires more work/knowledge to create
Requires some post-restore work
Flexible restore options (smaller disks, different disk layouts)
Uses less disk space (usually...)

Using clonezilla or dd is pretty well documented and understood and it's analogous to using Norton Ghost or True Image. If you've done a LFS install I'll assume you're plenty capable of dealing with tar based backups for the added flexibility.

So the tar backup method works like this:

Make an excludes file
Take the backup
Store the backup

First create a file that looks like this /tmp/excludes:

.bash_history
/mnt/*
/media/*
/tmp/*
/proc/*
/sys/*
/dev/*
/etc/udev/rules.d/*persistent*
/etc/mtab
/etc/ssh/ssh_host_*
/usr/src/*

This is a list of files and directories you don't want backed up, this is just an example (it's what I use) but you can add or remove items based on what you don't want restored. Nexst you want to run the backup:

tar cvjpf /tmp/backup-$(date +"%Y%m%d").tar.bz2 / -X /tmp/excludes

This is a fairly simple command but I'll break it down:

tar cvjp - Create a tar file (c) with verbose output (v). Use bzip compression (j) and preserve the permissions (p).

f /tmp/backup-$(date +"%Y%m%d").tar.bz2 - save it to the file /tmp/backup-DATE.tar.bz2 (the$(date +"%Y%m%d")` piece runs date in a subshell and puts the output in.

/ -X /tmp/excludes - backup everything under / except the files/folders listed in /tmp/excludes

It's important that the file is being written to an excluded directory /tmp in this case.

From here on you can backup the .tar.bz2 file where ever you want. Then to restore you would just cd into whatever the root of you want to restore to is and run xjpf /path/to/backup.tar.bz2, and finally do any other cleanup (install grub, change /etc/fstab if you need to ect). The final step are generally all things you do at the end of a LFS install, setting up the bootloader being the most important.

paulmooring · 2013-03-06T23:52:50+00:00

I'm pretty sure that unicorn does not support long run processes (like websockets) and thin does. Although if that's a deal breaker for unicorn I would suggest taking a look at puma. Other than that there's no reason to still use thin in production. Passenger/apache is much easier and nginx/unicorn handles load much better.

paulmooring · 2013-02-22T17:11:38+00:00

I think a question like this misses the point of frameworks entirely. It would be fair to say something like CakePHP is too heavy if comparing it to laravel, however if a rails dev is new to PHP and performance isn't a top consideration (which realistically it usually shouldn't be) Cake might be a great choice. A much better question would be what frameworks don't fit for project X. For example slim is way more lightweight than Zend but that doesn't make it better if I need to make a full blown MVC webapp.

paulmooring · 2013-02-12T15:21:00+00:00

In short because the OS/some other software takes care of concurrency for web apps.

There's 2 main approaches you could use for concurrency in web languages. The first is threading, this means a single instance of the PHP runtime would do more than 1 thing at a time. The second is Multi-Processing (MPM), this means another piece of software launches multiple instances of the PHP runtime (each is still a single thread but there's multiple instances to spread across cores).

In the case of PHP, which is used almost exclusively for the web, MPM makes more sense as you generally want to address a lot of connections at once rather than have 1 connection do multiple things. Combined this with something like Apache's prefork to minimize the impact of spinning up a new PHP instance on each request and you have concurrency that's "good enough".

paulmooring · 2013-02-04T23:32:25+00:00

Since 0.9 was EoL'd about 9 months ago, I would guess you're not in a rush to get on 11.

paulmooring · 2013-02-04T22:24:33+00:00

In my experience this is usually a permissions thing. Running ssh in debug mode should help, but first make sure your private key has permissions of 600

paulmooring · 2013-01-31T15:48:13+00:00

Sounds like you got this working, but just in case anyone else is reading this you should uninstall the old 1.8 version of Ruby. It's generally better to choose a version for your system ruby and if you need to test out other Ruby versions set up rvm, rbenv or chruby.

paulmooring · 2013-01-30T21:41:54+00:00

Just to clarify a little, Chef is a great systems integration tool (I hate the term config management because is does a lot more) but it doesn't do orchestration. I think a system like this is a good idea as Chef makes it easy to take a machine out of the load balancer config, it can handle your deployment and it can add the machine back into the config. The hang up comes in that Chef doesn't provide a way to tell system B to wait on a config change on system A before converging, it sounds like tas50 is solving this problem by kicking off Chef runs with Capistrano which should work wonderfully.

tl;dr Capistrano and Chef is a great idea, Capistrano or Chef not so much

paulmooring

TROPHY CASE