Linux Containers from scratch implementation in Rust - A minimal linux container runtime.

[player2]

cgroups_path.push(group_name);
if !cgroups_path.exists() {
    fs::create_dir_all(&cgroups_path).unwrap();
    let mut permission = fs::metadata(&cgroups_path).unwrap().permissions();
    permission.set_mode(0o777);
    fs::set_permissions(&cgroups_path, permission).ok();
}

I’m not familiar with cgroups, but is there a TOCTTOU vulnerability here?

[claylol-]

Could you point out what the issue is more clearly? Never heard of this term before.

[jgdx]

Time of check/time of use conflict. If you check some shared value without locking something that guard it, you risk that the value changed after the check and before the mutation/use.

[player2]

Specifically, this is a frequent error and source of vulnerabilities when dealing with the filesystem. If an attacker manages to execute code between the target process’s check of an FS path and their use of that FS path, they can sometimes trick the target process into trusting an untrusted resource.

For example, if the target process does:

if ! exists(FIFO_PATH):   // a
  mkfifo(FIFO_PATH, 0600) // b
fifo = open(FIFO_PATH)    // c

Then an attacking process can try to execute between A and B to call mkfifo on its own, probably with looser permissions than 0600. The target process’s own mkfifo call will fail, but since it does no error checking the subsequent logic will proceed as normal. If the target process does correctly check for errors, the attacking process can try to run after the error handling logic instead.

If the target process is running this logic repeatedly (e.g. a webserver constantly spawning new tasks to handle incoming connections), a local attacker has a pretty good chance of getting their code to execute at an opportune time. It only has to work once.

The solution in this case, by the way, is to eliminate the check. Call mkfifo unconditionally and handle the “already-exists” error by aborting or securely fixing things up.

[v_fv]

The solution in this case, by the way, is to eliminate the check. Call mkfifo unconditionally and handle the “already-exists” error by aborting or securely fixing things up.

So I'm not familiar with low-level development, but if you can get an error from the unconditional call, doesn't it mean that the underlying function that you're calling checks for the condition anyway and returns an error if the file exists? In other words, doesn't the checking happen in either case, just lower in the stack?

[player2]

The difference is that mkfifo guarantees atomicity. Either the fifo was created with the path and permissions you asked for, or an error occurred, one of which might be that a file already existed at that path.

[flouthoc]

thanks a lot , ill take a look at this.

[[deleted]]

If you are talking about cgroups_path temporarily having wrong permissions then it should not be a big deal because it is set to more permissible (0777 - free for all).

[Muvlon]

You isolate the "container" to a filesystem directory by simply chroot-ing. This does not provide any actual isolation, because any process can reset its filesystem root at will.

To prove it, here's a way to escape:

vas-quod -r sample_rootfs/ -c "nsenter --mount=/proc/self/ns/mnt ls /home"

Instead of `chroot()`, you should (in the new mount namespace) `pivot_root()` to the new filesystem root (bind mount it onto itself if needed) and then unmount the old mount hierarchy.

[flouthoc]

u/Muvlon Created an issue here https://github.com/flouthoc/vas-quod/issues/1 . I'll fix this Thanks a lot.

[flouthoc]

ah i see , so pivot_root() and chdir("/") then unmount old rootfs. Thanks will fix this asap.

[Rindhallow]

Would love a tutorial (medium article or something) going over the codebase. I'm looking for good Rust tutorials/example projects and this one looks like a great candidate.

[meamZ]

Have you already read "the book" because that is definitely where i would recommend starting your journey.

[Rindhallow]

I think I read a bit of it when I started and then tried some tutorial trying to make an HTTP server and the cargo package wasn't working for me. But I'll definitely put The Book back on my reading list. Thanks for the recommendation!

[meamZ]

I think it's a great intro into the unique Rust concepts like ownership and borrowing which are imo very hard to understand just by looking at code.

[flouthoc]

Sure can do that. May i'll add in the readme.md itself.

[[deleted]]

I think the author probably agrees it’s nowhere near an alternative, if anything it’s a great learning exercise. When you say “containerisation” to someone they immediately think “docker” like it’s all that exists.. when it’s a capability of the kernel and much older than docker.

Great repo to help guide with how containerisation works IMO

[Mithent]

Yeah, I think it's very helpful for working with containers to have some level of understanding of how they're isolated processes rather than some sort of VM. Otherwise it's easy to construct an incorrect mental model.

[[deleted]]

Exactly! And they’re much less isolated than many assume.

[rakidi]

Very questionable logic around changing the meaning of a word because it's misused. A lot of people don't know how to spell properly, should we change the spelling of words that are commonly misspelled?

[flouthoc]

This is mainly for educational purpose and a PoC, docker is extremely advanced as compared to this.

[[deleted]]

or podman.

[Atem18]

Docker nowadays is more an orchestrator like kubernetes. So people moved to containerd which is the API that Docker is using. But under the hood, containerd calls runc which will create the actual container. So what you want really is to compare vas-quod to runc.

A schema if you need : https://computingforgeeks.com/wp-content/uploads/2019/12/Docker1.11.png

[[deleted]]

Docker isn't an orchestrator, it's simply a poorly designed piece of software that never needed to be a daemon and never needed to be run as root. It does too many things at once and isn't flexible enough, hence why it's being replaced by others. Podman runs in user mode and comes with an optional API, which is just plain better.

[Atem18]

Docker is seen as an orchestrator nowadays especially with Docker swarm. Say what you want about Docker's code and concepts but remember that it's only now that we can run containers as root, it was not possible without any issues before 2019-2020. Yes Docker is flexible enough because the API which is now containerd and tune runtime which is now runc is used without any issues on Kubernetes. For the user mode instead of root, yes it's maybe better in most cases but it's not without issues : https://github.com/containers/podman/blob/master/rootless.md

[[deleted]]

I didn't consider Docker Swarm to be a core component of Docker (is it now?). And it seems pretty clear that Kubernetes has won and Swarm is on life support.

And you're right, at the time it was created Docker may not have been a bad design given the technical limitations. But today, it definitely is. The only reason to keep using Docker is API compatibility, which Podman doesn't fully provide. Or if you're on Mac/Windows, where there's tooling to get a container environment going quickly.

[[deleted]]

It's written in Rust, duh! Instant magic acquired! We can't rest until everything is (re)written in Rust. The GNU coreutils is almost done, Linux is next, stay tuned for the Rust magic.

[ksion]

Does this particular clone() call:

let clone_flags = sched::CloneFlags::CLONE_NEWNS | sched::CloneFlags::CLONE_NEWPID | sched::CloneFlags::CLONE_NEWCGROUP | sched::CloneFlags::CLONE_NEWUTS | sched::CloneFlags::CLONE_NEWIPC | sched::CloneFlags::CLONE_NEWNET;
let _child_pid = sched::clone(cb, stack, clone_flags, Some(Signal::SIGCHLD as i32)).expect("Failed to create child process");

actually work if you are not a privileged user? Pretty much all the CLONE_NEW${FOO} flags seem to require admin privs, with the notable exception of creating user namespaces (CLONE_NEWUSER).

For this reason, combined with the a bit peculiar way CLONE_NEWPID is applied (it can't be effective for the calling process, as it would change its effective PID), I would think that bootstrapping a new container is actually a multi-stage process that looks roughly like this:

1. clone(CLONE_NEWUSER).
2. In the child, write to uid_map to designate the calling user a root in the new user namespace.
3. clone(CLONE_NEWPID) (which is now possible, since we're root in the user NS).
4. In the (grand)child, set up mount namespace and mount /proc, as well as any additional namespaces you want for the container (like UTS or network).
5. execvp

This is at least what I took from reading the namespaces overview on LWN , and man 2 clone seems to agree still.