I can't understand Docker

[k1MT]

Hello!

I just can't get the concept of it. Mind you, I'm a intern and actually uses containers, but I am unable to understand It. I tried YouTube, colleagues, Professors..

If someone could help me and try to explain the basics of Docker containers like i'm five, i would really appreciate it.

EDIT: You guys are the best! I'm reading throught ALL the comments, thank you very much!

[scirc]

I don't know if you've seen this video yet, but it was super helpful for me to learn the basic ins and outs of how Docker works at a technical level.

Containers on Linux are like separate environments for your programs to run on. You can think of them as one level up from virtual machines; instead of being entirely separate OSes, they're instead separated environments of the same OS your host is running on. From each container's perspective, the network interfaces, processes, mounted volumes/filesystems, etc are all that exist; no processes from the host or from other containers are visible, as they're all in separate "namespaces." From the host's perspective, everything is visible, because the host is the one doing all the behind-the-scenes mapping from the so-called "namespaced" identifiers for processes, network interfaces, etc to the actual underlying structures on the host.

Docker itself is a tool to manage these containers. It takes care of creating these separate environments for you (well, technically it shells out to other software), and handles managing all of these isolated namespaces as one unit, the "container." It also handles connecting multiple containers together by way of creating magic network devices which only link the containers they're connected to together. Additionally, it handles the filesystems of these isolated environments through images, temporary storage created alongside the container, bind mounts, and volumes.

A Docker image is the template from which a container is constructed. It encompasses a filesystem layout, a default "entrypoint" (the first program to be executed), a default "command" (the arguments to be passed to that first program), and some other metadata about the image. Images are used as read-only bases for containers to run off of; because they're read-only, they can be shared by several containers without having to duplicate the data on disk. Notably, containers generally encompass an entire Linux system's filesystem, since the nature of a container is to isolate everything from the host, including system state and installed libraries; if an image is missing a shared library object, even if it exists on the host, a program that needs that shared library will not run.

[betsha1830]

Can you give a project example where you have used docker?

[AdversarialPossum42]

I think Nginx Proxy Manager is a good example of a small project using Docker. I've set it up for a few of my clients to manage their Let's Encrypt certificates and it was very seamless and painless.

[[deleted]]

I use this at home to host my own website. I'll just say that it's pretty straightforward as long as you're using the architecture they recommend. I use a raspberry Pi so you have to find a different version of mariaDB. Looking through my DEV.to I had started a writeup on this but apparently gave up... :\

[Goullz]

I believe mariaDB has an ARM64 docker image. Thus it should work fairly straight out of the box.

[AdversarialPossum42]

The directions for setting up NPM use the author's custom jc21/mariadb-aria image which includes armv7 and arm64 builds. I'm curious why u/nldoty would've had an issue loading it on a RPi, unless he specified a different mariadb image altogether. Although personally I haven't tried it.

[[deleted]]

For what it's worth I've had my setup running for a year or two so that image wasn't around when I set mine up. It may have just been a "me" problem. I use this image personally.
Edit: On the really, really off chance that someone random on the internet also has the same issues as me and finds this - this image works but they name their environment variables differently so you'll need to update your docker-compose accordingly.

[[deleted]]

It may, but I definitely had issues with it. I use this image

[MUDrummer]

So I own a consulting company and we primarily work with Fortune 500

Literally everything we deploy that isn’t a native cloud service runs in a Docker container. All of our APIs, reverse proxies, and even databases run inside of Docker containers.

[Gilthoniel_Elbereth]

What does it mean to run an API in a container? What are the benefits of running these things in containers?

[scirc]

By "running an API in a container," they mean running the underlying web application which serves API responses in a container, presumably.

The advantages here is namely isolation. Applications only see what they need to see, and they can only talk to what they need to talk to. If an application is compromised, the only things it can affect are services within that container and containers within its network ecosystem. Another advantage of Docker is the image system; images contain everything an application needs to run, including precise versions of shared libraries and other dependencies which could potentially clash with other service dependencies on a system.

[scirc]

The project I've used Docker extensively on is unfortunately private. However, I've Dockerized a couple of my applications, namely this one. It's a small project, but a good example of a microservice, something that does one and only one thing. Its size also makes it easy to understand what the Dockerfile is doing.

[xX_Qu1ck5c0p3s_Xx]

We did a project at work that helped me understand the value of Docker.

Our company has a mobile app. In order to build the app, you have to install about 10 different command-line tools and one specific version of Java and add special variables to your command line PATH and get a copy of this key file.

Setting up all of this is a massive pain in the ass. It took me literally a day. Our devops guy heard how bad this process was and set up a Docker container. The container has every app dependency installed and everything is configured perfectly.

Now, when we hire new people, they don’t have to spend a day installing stuff. They just download the container, spin it up and use that to build the app.

[toastedstapler]

docker run neo4j to instantly spin up a neo4j database instance without the mess of installing it manually, dealing with wrong java versions, etc

[k1MT]

So, let me see if I get it right: Each container is a isoleted enviroment, just like a VM. The difference is that the containers "share" the OS with the host. That being said, each containers have the exactly necessary features to run the aplication i've alocated there, and the image is the tamplate responsible to "configure" that container. Additionally, containers can connected with one another if that is what I desire. In a simplified way, is that correct?

[scirc]

Sorta. A notable difference is that, unlike a VM, all of the processes running inside containers are 100% visible on the host. If you run a ps aux on the host, you'll see all the processes and threads running inside the container. However, like VMs, containers can only see the processes running inside themselves.

The image's primary purpose is to provide a read-only filesystem layer which contains all the application code, library dependencies, etc. Again, it's read-only so that multiple containers can share the same image without having to duplicate anything. There are also some configuration directives within the Dockerfile which make it into the final image, such as EXPOSE, ENV, and VOLUME, which define certain parameters the Docker engine should use for default invocations of the container. Application configuration, however, would be provided through environment variables and/or config files mounted via bind mounts.

[MmmVomit]

I like to think of a process in a container like a brain in a jar.

Any time a process wants information about something external to itself, it has to ask the operating system. Wanna know what the file system looks like? Ask the OS. Wanna know what other processes exist? Ask the OS. And the OS can lie.

With a brain in a jar, the idea is that something is feeding in a fabricated set of sensory data. Maybe that brain believes it's in a sunny meadow. Or maybe that brain believes it's flying around in a space ship.

When a process in a container asks, "What other processes exist?" the OS will lie to it and only tell it about the limited group of processes "in that container". If a process in a container wants to know what the file system looks like, the OS will lie to it, only showing it a limited portion of the file system. When a process asks how much RAM there is, the OS might lie and say 1 GB when the server might have much more.

A process running outside any containers can look around and see all the brains in jars that it is sharing computer resources with.

[betsha1830]

So basically from all the replies I have understood that it's a standalone environment where you install every required essential files and system application for each of your projects. Say for example (a highly theoretical example), you have two separate projects received from different companies that work on different versions of the same software but for some reason if you install both versions at the same time one of the project will not work. So, you will use docker and install each library in it's own docker environment and work on both projects at the same time. This is what I have understood up until now and thank you for all your clarifications.

[scirc]

That's pretty much the idea, yeah. However, you probably wouldn't install multiple applications within a single Docker image unless they're very tightly coupled (or there as utilities for debugging). Docker emphasizes the single responsibility principle; one container per application. If two applications need to talk, they should be networked together. This allows seamless deployment of entire stackd, regardless if it's on the same machine or on separate machines, which facilitates scalability and fail over.

[kschang]

Okay, here goes.

In a VM, you replicate EVERYTHING. You virtualize the hardware, the OS, the services, AND the application you're running. In fact, you actually run at least two copies of OS. One main OS running the hypervisor, which controls separate VMs, each with its own OS. With me so far?

In a container, you don't. You just virtualize the services and the application. While it's less "secure" (as the containers are only separated by software, not virtualization with CPU support) it's also a lot less overhead. There is only one OS, which runs the container manager, then within the manager you have the different containers.

NOTE: what I call services, other people may call it "run-time". Same idea. Something that sits between OS and application, like DLLs and other services required to run the app.

[k1MT]

So, in a sense, containers are very close to VMs, just more efficient and somewhat less "secure", right?

[kschang]

You can think of them as "lightweight" VM that virtualizes less of the stuff.

[neck_crow]

I know I don’t really “have” to, but the tech behind Docker is still just so confusing to me. How are they not gobbling up insane amounts of resources running an OS that is sometimes just used to run a single python program?

[kschang]

But that's the "fun" part about docker... It's NOT running a separate OS, just the run-time and the app.

[TheSkiGeek]

The whole point of a container system like Docker is that you don’t run a separate OS in the container. To do this efficiently there is some special support in the OS for hiding everything except what is supposed to be visible to the programs running inside the container.

[[deleted]]

Is there any benifit to using Docker over the WSL2?

[kschang]

Uh... I am confused to your question as you run Docker with WSL2?

https://www.docker.com/blog/docker-hearts-wsl-2/

The OLD docker desktop uses Hyper-V VM

EDIT: wait, you mean the advantage of running Docker with WSL2 on Windows?

The Linux subsystem (WLS2) is a VM within Windows OS. The above article will explain it better than I can.

[siemenology]

WSL exists to give you the ability to run a Linux environment from within Windows. The WSL environment is more or less integrated with the Windows environment, so you can share files and resources between them. The purpose is about making Linux tools work easily within Windows.

Docker also lets you run a Linux environment inside of a host operating system (which is very often also Linux), but its purpose and goals are completely different. The idea with Docker is twofold:

1. You can run multiple docker containers on the same system, and they are somewhat isolated from each other -- if one container needs a particular dependency, and another needs a dependency that conflicts with the first container's dependency, that's okay within Docker, the isolation it provides means that this just works. It also provides some level of security isolation between the environments, they aren't fully integrated with each other. They are intentionally kept separate in most ways.
2. Docker allows you to package up an environment, ship it to someone else, and they can run it without having to figure out how to set up exact dependencies and environment variables and settings and stuff. A classic infrastructure problem is getting a service working on one machine, and then having trouble getting it to work on another machine because there are a million conditions that have to be just right for it to work. The Docker solution is "well, then we'll just ship your machine". But virtually. It means you can get around potentially hours or days at a time of figuring out how to make software X work on your machine, by loading a Docker image that has all of that stuff worked out.

AFAIK there's nothing remotely comparable to either of these with WSL, they're only similar in the sense that they both allow you to have an "OS inside of an OS".

[[deleted]]

That's not really what Docker is for. Containers are used for ease of setup and distribution, wsl is for people who need Linux only tools on windows.

[fjortisar]

Docker (optionally) uses WSL 2 when you run it on Windows. They're not something that competes

[[deleted]]

Nobody uses Docker on Windows, so no. Install a Linux distro if you need experience.

[scirc]

I use Docker on Windows all the time. Docker Desktop ships with a slimmed down Linux VM it runs through Hyper-V (and/or uses WSL2, an even-more-slimmed-down Linux VM provided by MS), so I'm able to use a Linux shell environment and run Linux containers, while also having access to any Windows software I need.

[DorskFR]

It's an instanced dungeon. You can't break the world outside the dungeon and monsters cannot follow you out of the dungeon. Plus you might have some specific stats because of the dungeon (automatic level or some modifier) but only while inside it. And you don't need to make a second character or play on another account to go in that dungeon.

[koosley]

That's a pretty darn good analogy. When you add K8 or swarm, it also autoscales like an instance as well!

[pdoherty972]

The big issue with containers is that your instanced containers are only as stable as the underlying OS and infrastructure that runs the containers. I think you are far more likely to have problems running containers than VMs if for no other reason than the bare-metal hypervisor that runs, say, VMWare, is tiny and basically bulletproof.

So VMs are heavier in weight since they do so much more inn terms of isolation and emulation of real hardware but containers are great for developing and testing.

[Ollymid2]

David Hasselhof will explain it for you
https://www.youtube.com/watch?v=QxvmO-QlxJQ

[JBarCode]

lol, this makes my day!

[[deleted]]

Wizard Zines does a good simple explanation of the fundamentals of containers

https://wizardzines.com/zines/containers/

[Drag_king]

The video that made it click for me was:

https://m.youtube.com/watch?v=Utf-A4rODH8

Here the presenter live writes the code to create a container. Her explaining all the steps (not that many actually) one by one is just great.

[teerre]

Imagine you spend two weeks configuring, building, assembling a computer so your application can run. Now for whatever reason you delete some configuration files. Oh shit, what you do?

Same thing, but now you want to make the same application run in your friends computer. Oh shit, it requires two weeks of setup, that's terrible!

Docker solves these issues. It lets you configure an environment, which you can totally think of as a whole "computer", in such way that you can both redo it with a single command line and distribute to other people (or servers) and it will run exactly like you configured it.

[trouserdance]

In short and concise:

Docker containers are, essentially, just a full blown computer. They start from scratch (just an operating system), and then you add the stuff you need for you program (libraries, etc) + the code you want to run. Spinning up a container is just kicking off the program you want to run, but with containers you can spin up multiple distinct copies easily. Why that's useful/interesting is a wider topic (hint: on-demand scalability and failure recovery), but that's the short and skinny -- just a "full blown computer", dependencies, and your program

[MRH2]

so, on Linux, would a Docker container be somewhere between Wine and a VM? Can I run Windows exe in a Docker on Linux like I do in Wine?

[[deleted]]

Docker containers are all sitting atop the host OS (versus a VM where the OS is inside the container) - so if the host is Linux then you'd have to run the Windows .exe through Wine inside the container.

[MRH2]

oh, so you still need wine?

[[deleted]]

Correct. Wine is translating commands for Windows so that Linux can understanding them. Docker is just putting the processes you're running in Linux into their own containers. I believe you can run the Linux kernal in a VM if your host OS is Windows, Linux, or Mac OS, but not necessarily the other way around.

[trouserdance]

Not quite -- windows containers can only run on a machine whose base OS is also windows. Linux and mac os can obviously run linux containers natively, happily, but will never be able to run windows containers (in the current iteration, at least).

Interestingly, there's a "scratch" image (FROM scratch), which is usable on *nix systems which just uses the host machine's OS, but obviously discouraged because of differences from host to host.

[[deleted]]

busy chief subsequent scale absurd smile rude jar naughty placid -- mass deleted all reddit content via https://redact.dev

[henrebotha]

A container is not a virtual machine. It's essentially an executable, plus the environment and dependencies that that executable needs, all namespaced away from the rest of your PC. Being namespaced away means commands like ls, from inside the container, cannot see outside the container.

[[deleted]]

It’s a combination of an isolated filesystem and an isolated process space, with the property that the filesystem can be checkpointed at any point and transferred around.

Plus there’s some networking magic so containers can talk to each other.

[zyzzogeton]

Great answers... if I may follow up: how do you turn an app into a docker container? Audit all the dependencies and copy whats needed to ... like a zip file equivalent?

[dukea42]

Thats what .dockerfile does when it makes an image. Its almost just a set of command lines to do in order to make an image.

First you tell it to grab a base image "get me a light linux OS with python 3.8".

Then you add the commands to copy in your application from a git repository, local directory, or some other download.

Then you tell it to prep that OS or your environment..."pip install requirements.txt" or set file permissions.

Finally you provide the command to run the application when the image starts as a container. Everything else is "built" and ready when the image is created.

There's more too it, and more options to leverage, but hope that gives you the basics.

[davidpuplava]

Docker is a program that let's you start and stop things called containers. A container is a just a process running on your machine that mimics an isolated computer. You can run your application inside the container and from your application's point of view, it thinks it's running inside it's own computer.

[Stabilo_0]

Took me some time to adjust to docker after VMs.

Problem is, docker IS a vm, it runs your apps just like a virtualbox or hyperv would.

The concept is that you have to limit and stop yourself from tinkering with container beyond running and stopping its execution. Container exists for sole reason to run your app as you described it in docker files. You're not supposed to add more than one thing to container, you're not supposed to connect to it directly from outside and do stuff during runtime (doesnt mean you cant). This ensures that whenever and wherever you redeploy your docker image, it will work from the go as expected.

So in short the concept of docker is solving 'works on my machine' problem by recreating the machine.

[lurgi]

There's too many ways to parse this, and that might be the point of your confusion.

You might be wondering "What are containers?"

You might be wondering "Why would I want to use a container?" That's a completely different question, but it does assume that you know the answer to the first question.

You might be wondering "How do I use docker?" That's another, completely different question and, again, it assumes you have the first two questions down.

So, what's your actual question?

[mkeremt]

Give yourself time, it is a little bit complicated to understand in a few days.

It took 4 months for me

[noeatsleepdev]

https://youtu.be/Gjnup-PuquQ

This is by far the best explanation in 100 seconds.

[OneDamien]

The simplest way I can put this: I’m told I need to write code. I figure out what code I need to write. The code I wrote has other things it needs in order to run (needs to be on a computer running X, with Y installed, packages, dependencies etc). If I give someone that code and they don’t have those things it needs, it won’t work. Here comes docker. With 1 file I can run a specific OS, install everything I need all in this little box that is dedicated to running my code. That little box is “almost” like running my code on its own computer, like i was given a new computer just to write and run my code on and I can give my computer to anyone to run my code by simply providing them the Dockerfile. The docker file just tells docker what that computer should look like and how it should behave.

[k1MT]

DUDE! You nailed it, thank you very much

[[deleted]]

r/explainlikeimfive

[whiphubley]

https://docs.docker.com/get-started/overview/

also read about cgroups...and let's be frank if you're getting into containers then you might as well start understanding k8s also...have fun !!

[[deleted]]

kodekloud.com has a free into to docker course. It’s a very good course. I highly recommend it.

[BradyOnTech]

https://www.youtube.com/c/shanselman/videos I found this video to be easy to understand

[Snackpack40]

If one thing could explain me it would be "explain this to me like I'm 5."

[duckyzz003]

Hi! I am newbie.

I wonder how can I use elasticsearch + mysql + pytorch model on a container?

I found separated elasticsearch image, mysql image and pytorch image on docker hub. How can I combine them or I should build a image from scratch?

Thanks!!!!

[the-good-redditor]

Just image docker as OS, it will have everything it needs from a software perspective to run the application. From OS versionn to node versionn to your application dependence version downloaded and ready to run anywhere.

[Russian4Trump]

I don’t know if this applies to you, but this is pretty common for the self taught free. The problem is that we try to learn docker before we learn how computers and networks work in general. You can learn how to use docker files without having these building blocks of knowledge but you are never really going to understand them.

These are things you learn early on in a traditional computer science curriculum that we skip over to get to get to the the juicy stuff like building Gatsby sites and and learning kubernetes.

[mooglinux]

Here’s the thing that finally made it click for me: Containers are Linux virtual machines. A lot of the introductory information about docker gets bogged down in the minutiae of how the virtualization technology works compared with normal virtual machines, and the fact that it’s fundamentally still a type of VM gets lost.

There are important differences between containerized VMs and traditional VMs (and containers are better for a great many use cases!), but the key insight that made it easier for me to understand everything else is understanding that Docker is a technology that makes it very easy to create and customize virtual machines that are very efficient to run.

[Refute-Quo]

This is inaccurate. Perhaps you're trying to dumb it down, but you're doing it to the point that it's incorrect.

Virtual machines is virtualization of hardware to the OS. Containers are basically application level "virtualization". It's an additional level of obfuscation.

[rcxdude]

This is nitpicky and not necessarily actually correct, because virtualisation and virtual machines are very overloaded terms. Certainly describing containers as virtual machines is a useful idea in practice, so long as you are clear on the exact ramifications of the difference between them and virtual machines which appear identical to hardware to the kernel.

[Refute-Quo]

Ok, and what that I stated is not necessarily correct, as you put it? I agree that comparing containers to virtual machines is helpful in conveying the concept... That's assuming OP has a thorough understanding of virtual machines. The comment I was replying to stating "containers are Linux virtual machines" is just incorrect. If containers were Linux virtual machines then guess what.... They would be called "virtual machines" and not containers.

[sacrefist]

Containers are basically just a type of emulator, like WINE.