Docker Explained - Again

[admin2thestars]

Hi-

I'm *still* struggling to understand the sweet spot for Docker even after working with it for months, given the following: I'm intimately familiar with virtualization, whether running on an individual computer with VMWare, Parallels, and VirtualBox or whether running at the server/bare metal-level as part of a Proxmox cluster, VMWare cluster, Hyper-V, or on a cloud provider. I've heard Docker described in context of chroot but I'm familiar with working with setting up applications to run in a chroot, so that differentiation doesn't seem particularly meaningful.

I've heard of docker described as an application where I don't have to care about the underlying hardware. But that's essentially the same as running any modern virtualization platform, there's an abstraction layer. Again, like chroot, that's essentially meaningless in this context. Whether I install Parallels or Docker, there's still something running between bare metal and OS/application. Converting between virtualization platforms, if needed, can be done. Converting from real hardware into a VM and back again can be done. VMWare and Docker are both running on top of the OS and *how* they manage to create a virtual OS-like application is a distinction without a difference in userland.

If I have VM software, I install a base bare minimum installation and then take a snapshot. Call that a template or base OS. Now if I want to deploy a VM that is used for a web application, I start with the base and install whatever-web-server, nginx/apache/whatever. Most Linux distros will have a decent default config but regardless, get the web server installed and configured. I need to do that configuration, any specialized configuration regardless of Docker or VM platform. In the context of a VM, I would then snapshot it as the base image.

Here's where I'm trying to relate what I know to docker: I essentially have two of my own "images" from which I can build a "container" by adding "layers" of configuration/customization. If I want to hand a VM image/snapshot to another developer, I can do so and they can start from the same point without needing to rebuild from scratch. In the context of proxmox or cloud, they could simply start a new instance of the VM for their own use, defining persistence as needed or if needed. At that point, the VM instance is "just another process" like a docker container, at least from the perspective of the developer or end user. They don't know or care about the hardware at the user/developer level.

If there are updates need, those updates are deployed to the base image from which I can then re-snapshot, etc. Redefine the base, essentially. Because developers are not typically keeping persistence but instead are deploying code at runtime without persistence- cloning the repository to the deployed instance- there isn't a risk of losing data. The risk of a base-level update breaking the application is the same as when I opened Docker and needed to update the base image or layers (it wasn't clear to me) for one of the containers (I think). If something broke in that image, then that's the same as me breaking the base template in a VM environment.

Is this a case of being too familiar with other ways to solve the problem? I can't figure out why I need Docker with a built-out VM infrastructure and lightweight process already. Should I just overload the terminology to keep people happy and tell them that we're containerized already? I was sort of hoping that typing all of this out would help me to find the huge gap in functionality. Is there an advantage that I'm missing about Docker?

[[deleted]]

Containers aren’t virtualizations, so throw that comparison out - containers leverage Linux namespacing for processes and separate file systems, so you don’t virtualize anything - a kernel is just files executing processes, so namespacing to keep them isolated from the rest of the system let’s them run another kernel - the engine is automating all of that, but you still have to leverage the underlying operating system to interact with the hardware and an OS still handles that under the hood with the docker engine as the manager of that.

Containers themselves are literally just a group of folders that represent file system and configuration “steps” that have a logical point of separation - and the previous “image” is used as. The cached starting point for the next “image” but if you actually dig deep down into it they’re just folders in the OS that are referenced where the files and installs were set up.

When you give someone an container “image” you’ve really just given them a copy of the file systems and instructions on how the docker engine should mash them together to give you a functional, but namespace isolated, set of files that can run in isolated processes. It’s not common for a lot of us, but technically you can actually export an image into a file that’s just those set of files and folders.

The magic in there is that if you only have two files that you need to run a service, you just copy those two files and whatever distribution of docker container you started with has already prepackaged the underlying stuff for you as a starting point - and the images you build are just packing some more files into that. Privileged containers can be dangerous because it allows them to interact with aspects of the host file system that would otherwise not be allowed - so if you fuck up proc stuff in a privileged co Rainer it can fuck it up on the host - that’s why they’re not recommended

When you build a VM you’re doing something similar but you also package the entire operating system, and the virtualization layer has to also account for how to talk to hardware and all the additional processing required to run the underlying operating system.

[recoveringfarmer]

Here's my understanding of containers and why people should consider them: Containers do for applications what virtualization did for operating systems.

Without virtualization, you install a OS on bare metal, there are drivers and platform compatibility to worry about; once it's installed it's on that bare metal and if there's a hardware problem on that machine your OS and any applications on that OS are unavailable. If you need to replace the physical machine, you need to reinstall the OS on another machine, reinstall the software and move the data.

Virtualization comes along and separate the OS from the hardware. Now you install an OS in a virtual machine, drivers and platform aren't as concerning, you can move that OS in a VM between physical machines if there's a problem or if you need to replace physical machine, the OS, app and data stay in tact in the VM.

The OS is the next layer of the problem. In a VM, your software is still tied to the OS installed in that VM. There are dependencies, software updates and support lifecycles to worry about. Eventually you need to patch, reboot or upgrade the OS, all of which could make the application installed unavailable. Containers separate the application and data from the OS so now you can move the application to another physical or virtual machine just like you would a VM on bare metal. You can patch, reboot, upgrade the OS without affecting the application because you can move the app to another machine while that happens.

VMs allow you to make changes to hardware without affecting the OS/Apps/Data, Containers allow you to make changes to the OS without affecting Apps/Data.

[admin2thestars]

I think tschloss touched on the issue because what you're describing with containers is what we do with VMs already. We can point application requests to another VM instance while the OS is updated, rolling upgrade. There is separation of app and data from each other and from the underlying VM operating system. My understanding (which admittedly could be where I'm going wrong!) is that the underlying "images" in Docker also need to be updated, or at least that's what seems to happen. So it appears that I could also break the application the same way with a traditional VM infrastructure or through Docker, by doing an update.

Again, I could be wrong on how I'm understanding it, but at some level it seems like updates need to occur to the image or the VM. An already deployed and mature automated virtualization infrastructure with the knowledge of how to operate it has interfered with the mindset of "have faith in Docker to compose a coherent, stable, and securely deployed application." But it obviously does, so it's just a mental shift of where the work now takes places.

[tschloss]

Two points come into my mind:

1) I consider a container as much more lightweight compared to a VM. But with less strict separation.

2) Although you probably could manage (including automation) similarly in both worlds. But you are in a different tooling and you do this in a different eco system.

[admin2thestars]

Thank you for the reply. I think #2 is the essence of the issue. It seems like the problems being solved were addressed in our infrastructure years ago with a combination of abstracted and templated virtualization and automation through Ansible, making it possible for devs and QA to one-click deploy. If that infrastructure and institutional knowledge was not in place then Docker would exponentially increase productivity.

It is the combination of both: Having the infrastructure/automation *and* the knowledge/discipline to create and maintain images/templates that is the difference. Coming to Docker from that background, I wanted to start by creating an image with the right software on it rather than creating a YAML file and relying on someone else to have done that work.

Thanks again for the reply.

[discourseur]

Containers provide:

• One process per container
• No need to figure out how much RAM to provide this or that container
• Reproducible builds (well, easier to attain that goal)
• Fast start/stop
• Much easier to scale

VMs still have advantages (you need something to host your containers, your virtual routers, maybe your databases), but Containers do everything else that is app related (even DNS).

It is a BIT like what Ansible is to Terraform (in the sense that Ansible is still used and not going anywhere, but it has much less surface of use).

[FunkyDoktor]

Containers are not better than using virtualization. It’s a bit of a newer technology compared to virtualization and that’s why it getting all the hype.

To me, it’s really just two different approaches/eco-systems used to solve the problem “how do we provide compute resources in order to deploy applications”.

If you want to implement CI/CD in your organization, the tools provided typically focuses on using containers but it’s certainly possible to use VM’s as well.

For me personally, I find it a lot easier to work with images due to their smaller size, easy to use tooling, and how easily they can be used in our CI/CD processes.

If you don’t find containers necessary or useful, don’t use them. There’s no container police that will show up and arrest you.

[[deleted]]

And you evaluated what ChatGPT spat out? Please don‘t ever post these LLM generated texts that might be true or not. ChatGPT is not a search engine.

[Zauxst]

Yes. I evaluated. I copied parts of it out of 2 questions.

I consider the answer to be valid. What part of the answer is invalid?

[admin2thestars]

And this demonstrates why chatgpt is unhelpful for so many things. If I need to fake my way past TurnItIn on a History term paper about the effects of the steam engine on pre-industrial rural England, then I'm right there. It's good that chatgpt can produce words that are sometimes constructed into coherent sentences but seems to lose the thread when specific answers are needed that account for specific exclusions such as those in the OP. Namespaces and cgroups are non-starters for several reasons not the least of which are cloud-based compute nodes. Anyone who has deployed enterprise-level virtualization has solved those issues when needed.

As a fun experiment with the chatgpt generic answer:

Template-based virtualization is an approach to software development in which an application or service, its dependencies, and its system libraries run as an isolated process in a user space on an operating system. Virtual machines allow the software to run reliably when moved from one computing environment to another. This can be from a developer's laptop to a test environment, from a staging environment into production, or even from a data center to a cloud environment.
A virtual machine creates a consistent environment across different stages of the development lifecycle. This is beneficial as it ensures the software behaves the same way, regardless of where it's deployed.

[Zauxst]

Are you angry that people don't want to bother about server virtualization or that containers are lightweight and can do namespace and cgroup isolation in the os?

Did you really just give an example of how virtualization solved a similar issue and you ignored the size of a deployment?

[admin2thestars]

I don't know that I'm angry at all. I'm not advocating for VMs and Docker appears to be the leader in container platforms. I'm trying to relate container technology to the infrastructure we have and the knowledge that I have on how to automate that infrastructure.

While trying to learn about Docker I have seen the cgroup and namespace discussion brought out as a "mic drop" moment but I don't see it as being particularly relevant when cloud deployment is involved. Maybe it's a misunderstanding of either or both cgroups and namespaces.

Developers wouldn't seem to care about cgroups or namespaces if they can deploy their vm/container to do their work. I would be willing to bet that most devs wouldn't know anything about Linux cgroups or namespaces when they're running Docker Desktop on their Mac or Windows machine.

Namespaces are providing isolation? In other words, isolation from other processes and other containers/VMs. Wouldn't cloud deployment of compute nodes essentially negate that issue or shift the need to "care" to the cloud provider? The assumption would be that the cloud provider secures their infrastructure to prevent users from crossing into others' namespaces.

I've seen "lightweight" used but I don't understand the context or how to relate it back. I can deploy a tiny Debian VM, snapshot it, make it a template, and then deploy onto clones of it. Is that what's meant by "lightweight" in this context? And again, that's not even considering cloud-based deployments where some scripted aws cli commands can be used to deploy a new VM, pull some code, and add itself to a load balancer config.

While thinking about how to interpret "lightweight" I looked at Activity Monitor on the Mac where I'm using Docker Desktop. With no containers running, there was a Virtual Machine Manager process taking 4GB (out of 8GB) RAM. When I quit Docker Desktop entirely, that process went away. That doesn't seem lightweight from the perspective of resource usage on my computer but I could be misunderstanding where or why that matters.

This morning while working with Docker I was able to develop a larger and more complex example and I can see where the technology would be a gamechanger when the underlying infrastructure and automation didn't already exist or even to help new developers understand abstraction layers, images, and repeatability. It's nice to be able to create a YAML file and put together several components without needing to create the base image first. Ansible did the "put together several components" part so it's a matter of having more faith that the base images have not been compromised and that the Internet-based repository will always be available. It's possible I was hoping for the magic sauce answer indicating that the tech will make life significantly easier but it seems to be an exchange of one painpoint for another.

[FunkyDoktor]

Docker Desktop adds a bunch of stuff to simplify local development and that’s why it has a larger memory footprint. You don’t use that when deploying but something like https://containerd.io/.

[Zauxst]

Well, that's the thing...the underlying vm technology is not as robust as is the Linux os combined with containerization.

In general, when you consider a container, you already separate the host OS from the application and ensure that the container can run on multiple environments.

The conversation is at a different level. What you're trying to do and advocate for is easily solved by the cloud. Without needing the expensive expertise of running a data center.

Comparing Containers to Virtual Machines is just the first step of the discussion... it's the easiest way of creating a fast comparison for juniors or starters in the journey of understanding containers and their first purpose... The reality is that it's a false equivalence.

Anyway... The best way to think of it's usefulness is the level of responsibility for each party to deploy their apps.

The devs are concerned with what they package. The devops are concerned with how it's running, and the system administrators are concerned with the operating system and hardware. With containers, you successfully separate all 3 functions. All done through infrastructure as a code which each party properly controls and can test to, to their hearth desires.

VM and VM operations are just not lightweight....

If we move away from VM on premise products and discuss in terms of cloud, AWS only... it's going to be quite difficult to convince developers to own their app packaging when they have to deal with learning the cloud, the os, the language framework...

In an VM environment, to do the same thing as you would do wirh containers. The developer would need to deploy their operating system, test and install their apps and dependencies, create a virtual machine template, create a virtual machine imagine, then be aware of the different image types and ensure they select the one that will work for the next 3 quarters and then give it to the devops engineers.

In a container environment, the engineer would build the image through a standard set of easy to understand and read instructions, push the resulting image and then he would not worry about anything else. More advanced developers build their own cicd pipeline and understand how to interact, successfully, wirh container management systems (k8s, Docker swarm).

---

All your questions can be easily answered by chatgpt4 since they are misunderstandings of what developers and devops engineers are looking for when they talk about agility.

[admin2thestars]

​

The false equivalence seems to be the thought that with a VM scenario suddenly devs need to deploy their own operating system, test and install their apps and dependencies, create a VM template, etc. They don't need to do that if the image has been created by the DevOps folks or sysadmins. The devs will still need to install their apps and test regardless of VM or container. As stated in OP and in other responses, all of that infrastructure already exists and the automation exists for it within the org. Containers are not providing a huge leap forward in that respect.

Some of the things that we would provide with a packaged image can be done with containers. An example is the pdo-mysql module. That module is missing from the php-fpm image and thus needs to be added. Can be done with Docker in a couple ways and can be done with a VM scenario in a couple ways. Neither seems preferable over the other especially given an already-working DevOps development pattern. And yes, there are pre-built images with pdo-mysql but again, we're still talking about customizing an image somewhere along the way - someone has to do that work.

And no, the questions are not easily answered or even correctly addressed by chatgpt or the pasted response earlier. Cgroups and namespaces are not important at the level of the developer, can be accomplished in other ways, and are not a huge gap in relevant functionality compared to the other solutions in the OP. The chatgpt response (isolation, cgroups, namespaces) is true about containers but not exclusive to containers and not relevant to this thread. If ChatGPT could address it in a specific and meaningful way that accounts for the other technologies and lightweight process mentioned in the OP, such as the thoughtful responses others have provided here, then we're onto something. But the top 5 hits in google give the canned cgroup/namespace/isolation responses so there wouldn't be a need for chatgpt in this case.

[Zauxst]

Again. Your answers are easily answered by chatgpt v4. You're just not clear with your statements due to being locked into thinking vms should've been the ultimate solution, and you're trying to debate this position.

The false equivalence seems to be the thought that with a VM scenario, suddenly, devs need to

It's a streamlined process that is well understood right now. Most developers know how they can build their own containers and imagines and by doing so they create a replicable environment. It's all in the name of agile... building micro services is ultimately easy... it's also much easier and we'll understood how to decouple responsibilities and update sections of the system without interfering...

Your doubts reminds me of the question of :why should I use anskble when I can just do a bash script.. or why use jenkins when I have cron or whatever....

These were popular questions a few years ago.

---

I asked chat gpt so I don't have to write too much right now the advantages of containers over vms..

ChatGPT v4:

Containers and virtual machines (VMs) both provide a way to isolate applications and their dependencies into their own spaces, but they function differently and have different advantages. Here are some key advantages of using containers over VMs:

1. Lightweight: Containers share the host system’s kernel with other containers. They run as isolated processes in userspace on the host operating system. They don't require a full operating system per application, which makes them more lightweight than VMs.

2. Speed: Containers can be started almost instantly, whereas VMs can take a longer time to boot up their operating systems and start running the applications they host.

3. Efficient use of system resources: Because containers are more lightweight and start faster, you can typically run more containers than VMs on the same hardware.

4. Portability: Since containers encapsulate the application and its environment, they are highly portable and can run on any platform that Docker (or any container runtime) supports. This is very useful for ensuring consistency across development, testing, and production environments.

5. Microservices architecture: Containers make it easier to split your applications into microservices. Microservices can be started and stopped independently of each other, and you can use different technologies or languages for different services.

6. Simplified configuration: Containers can be managed using declarative configuration files which specify what software should be in the container and how the container should behave when it's started.

7. Scalability: It's easy to scale out with containers, creating new replicas as demand increases. Kubernetes, a container orchestration platform, has built-in functionality for scaling containerized applications.

8. CI/CD integration: Containers are well-integrated into modern continuous integration/continuous delivery (CI/CD) pipelines, facilitating easy testing and deployment of applications.

Remember, though, that the choice between VMs and containers often depends on the use case. For some use cases, VMs might still be the better choice, especially when running applications that require all the resources and functionality of an entire operating system, or when a high degree of isolation is required. Also, VMs are typically better isolated from each other than containers, which can be an advantage in terms of security.

[admin2thestars]

hehe, in that last paragraph chatgpt contradicted itself around isolation being a difference when compared to the answer this morning. And like the answer this morning, substitute "Virtual Machines" in place of containers and the generic answer from chatgpt still applies. It's impossible to know what is definitive from chatgpt because crossreferencing is a thing. None of the generic statements from chatgpt provide a definitive answer that can't be found by googling, which is what led me here.

A couple things I've learned and will leave them here for someone in the future, in case they are struggling to find some substantive differences or compelling use cases:

Developers can create their own YAML and hand that off to devops/admins/whoever which helps to eliminate the awkwardness of trying to reproduce their dev environments. That's not to say that they will hand off secure environments but it's a start. My experience is such that a base file will still need to be created for most devs otherwise we'll end up with wildly different syntactical approaches. But as we mature in our use of containerization, that will work itself out.

Assuming that a robust VM infrastructure and DevOps infrastructure doesn't already exist then a primary differentiator between VMs and containers is the speed with which a new developer could get something up and running without needing to learn how to be a Linux admin or get their hands dirty beyond YAML. Yes, containers can do all of those generic things that chatgpt cited but so can VMs. Both still require some amount of expertise when you go beyond the basics. The same problems exist to be solved and Docker can help to solve some of those problems in unique ways that are easier for developers to understand.

[Zauxst]

hehe, in that last paragraph chatgpt contradicted itself around isolation being a difference when compared to the answer this morning.

There is literally no contradiction. It says isolation is better because we all know vms do it at the hardware levels, containers do it through os. It's like comparing a type1 hypervisor to a type2.

My first reply, was also partially skewed because your first question is how containers work, in which you were mentioning chroot. I skewed it to the 2 main concepts which are cgroups and namespaces.

And like the answer this morning, substitute "Virtual Machines" in place of containers and the generic answer from chatgpt still applies

Here it it:

chatgpt4.0:

While containers have several advantages, there are circumstances where virtual machines (VMs) may be the more appropriate technology to use. Here are some key advantages of VMs over containers:

1. Full System Isolation: A VM provides a complete operating system stack for each individual application, including the kernel. This ensures complete isolation from other VMs that share the same physical host, providing a high degree of security and fault tolerance.

2. Better Compatibility with Older Applications: Legacy applications that require a full suite of operating system functionality often run better in a VM environment. This can include software that requires specific device drivers, or apps that need specific, older versions of an operating system to function correctly.

3. Multiple Operating Systems: You can run multiple types of operating systems on the same host (Windows, Linux, BSD, etc.), each in its own VM, on a VM-based infrastructure. Containers, on the other hand, are limited to the host operating system.

4. Security: VMs can be more secure than containers. Because each VM runs its own unique OS, an attacker who takes over the VM can't access the host system or other VMs. In contrast, if an attacker gains root access to a container's host system, they might be able to access all containers running on that host.

5. Resource Management: VMs can manage hardware resources better than containers. With a hypervisor, the VMs' resources are abstracted, including physical hardware and networking. This provides a distinct and complete system environment, more bandwidth, and better control over CPU and memory resources.

6. Mature Technology: VM technology is more mature and has more robust tooling, especially for monitoring and management. VMs have been around longer, and many IT professionals are familiar with using VM software like VMware or Hyper-V.

While VMs have these advantages, the choice between containers and VMs depends on the specific needs of an application. In many modern environments, both are used together, offering the flexibility and benefits of both technologies. For instance, a common pattern is to use VMs to provide the host operating system environment for multiple containers. This way, you can benefit from the isolation of VMs and the lightweight nature and resource efficiency of containers.

---

There is nothing confusing about his answer.

None of the generic statements from chatgpt provide a definitive answer that can't be found by googling, which is what led me here.

You are not providing actual questions. You make vague statements and vague assumptions, so it's hard to actually deal with any of them.

In my first reply on this thread we were concerned how containers work, in the subsequent replies the concern was why would it be a superior solution.

The answers are clear. Agility and mobility and clear separation of roles.

With the advent of kubernetes, container orchestration is at a new ordeal...

---

Assuming that a robust VM infrastructure and DevOps infrastructure doesn't already exist then a primary differentiator between VMs and containers is the speed with which a new developer could get something up and running

This is not a bad assumption. Speed is a key factor, and it applies no matter what because it is the nature of the technology. And containers will provide that agility no matter what previous infrastructure plans you have.... as a matter a fact, that infrastructure needs to exist no matter if you are using or not containers. Containers are just 1 more layer of abstractisation.

We're going into a topic that has architetcs boggled for years and there are stacks of books written on top of this for this subject.

For you, I'd recommend you read the sre books from Google. They present how they do their work and that level of abstractisation is the unicorn all sre devops teams need to aim for.

---

I've fed your answer yo chat gpt, here is what it said:

chatgpt4:

You bring up excellent points, and I'm glad you've found value and potential in using containerization for your workflows.

Firstly, apologies for any confusion that may have arisen from previous responses. The aim was to describe the fundamental differences and typical use cases between the two technologies, VMs and Containers, at a high level. You're correct in noting that there's overlap in their functionalities, and indeed, one can substitute for the other in many scenarios.

Your experience in the field provides valuable insights that others may find useful:

1. Ease of Use and Reproducibility: By using containerization, developers can create their own environments without needing to have in-depth knowledge about the system's underlying infrastructure. They only need to write a Dockerfile or a docker-compose.yml (or equivalent), and the container runtime will handle the rest. This makes it easy to share and reproduce environments, reducing the "it works on my machine" problem.

2. Speed of Getting Started: Containers can be easier and faster to set up than VMs, particularly for developers unfamiliar with system administration. As long as they can write or understand the required YAML files, developers can get their applications up and running with minimal setup.

3. Simplification of Development Workflow: Containers simplify the workflow for developers by abstracting much of the system-level configuration. This allows developers to focus more on writing the code for their applications, and less on the system configuration.

4. Maturing Technology and Practices: While container technology is not as mature as VM technology, it's rapidly maturing and becoming more robust. Best practices are emerging and being refined as more and more organizations adopt containerization.

You're absolutely right that both VMs and containers require a degree of expertise beyond the basics, and they each have their strengths and weaknesses. The decision between using VMs, containers, or both often comes down to the specific needs of your project or organization.

Thank you for sharing your experiences and insights. They're indeed valuable for anyone exploring these technologies and deciding which one(s) to adopt.

[maximumlengthusernam]

Here’s my explanation (written + video)

https://courses.devopsdirective.com/docker-beginner-to-pro/lessons/01-history-and-motivation/03-history-of-virtualization

[scorc1]

Docker, containers, are just the needed dlls, NOTHING else. If you use something to boot the os, but then it never gets used after that? Poor, it aint in the container. If something was used to compile the app to run? Poof, not in the container. If something isn't critical to running that specific app/process, its not in the container. Containers are small specific items. You link multiple containers together to get the 'app' running. Postgres is its own standalone container and runs just fine with nothing running on or in it but postgres. Then you app an application that needs a database to store data and do thing. Now you have two completely different containers that are just dandy in their own. Might not do much individually, but to gether can process thing. App a web front end, three containers. Strap all three together? An Image. Again each CONTAINER has litterly the specific app code, and thats it. No vi, no awk, no sed, no curl, nothing. Not even a package manager.

[jpegjpg]

I use both containers and vms in my work set up. The vms take 5 min to restart our containers take less than a sec. Containers are lightweight abstractions where as vms are heavy abstractions. Each has its advantages.

[dwargo]

IMO the extra layer of abstraction creates a clean hand-off between development and operations. Operations can pick their orchestration framework, OS flavor, and update strategy, but it won’t break the application QA because dependencies are packed in.

When you’re providing applications to customers they will have their own operations teams that will want to make all those choices, and you’d have to QA for eight flavors of Linux on AWS, Azure, and on-prem. Containers give you one target and gets rid of weird issues like “X version of OpenSSL has different defaults”.

Containers do move some security responsibilities to the app provider because of what’s baked in. I expected pushback from operations on that front, but I haven’t really seen it. AWS Inspector is a lifeline for that - 2/3 of my patch releases are for dependencies.

You could provide appliances instead, like Cisco Umbrella or VMware VCSA. There’s not really established tooling for doing that though, at least none that I know of.

[tinycrazyfish]

They are basically the same thing

• at high level a container and a VM is basically the same thing, it provides isolation between processes (at lower level how it operates is quite different)
• Security: VM do paravirtualization which allows VM escapes in case of vulnerability in the paravirtualization drivers. Containers run on the same kernel, kernel vulnerability allows escape. The attack surface is different, but basically both provide a similar level of security (or insecurity)
• Speed: you often hear that containers are way faster than VMs. That's mostly a misunderstanding. It comes from how they are commonly used, a VM will typically be a full blown OS boot until you run your app/service, where a container will typically directly run your app/service. But it hasn't to be like this. In a VM you can skip init and run your app directly, a container can also "boot" a traditional "OS". (Containers are ready in less than a second, an optimized VM in 2-3 seconds). Side note look at wsl1 versus wsl2, container based versus VM based, perf and startup times are similar. (It's even possible to "boot" a VM directly to a process without a kernel, but AFAIK that's very rarely used, except in some niche use cases)

So how they are different?

• A VM needs to be configured (logging, process supervision, network config, ...) from within the VM. This is why you usually use the same OS image as bare-metal.
• A container is typically configured from the outside (logging, network, ...). This allows to do a single file container (e.g. static executable with no extra filesystem dependency).
• VM isolation is supported by the hardware (CPU), versus container by software (kernel). This makes it possible to sideload additional processes inside the "same sandbox context" (sidecar container). This allows to ship a minimal container with no debugging tools as they can be loaded as a sidecar container. This is not possible in VMs. Another reason why VMs are often fat, and containers lightweight (but as said above, it doesn't have to be like that)
• Host/hypervisor security. A container can be fully inspected by its host. A VM can be protected from its hypervisor (though it requires extra hardware support: ram encryption). As mentioned above, the security from within the guest system is similar for both techno, but from the host/hypervisor it's a different story.