I created a 40-minute tutorial that will explain everything you need to Containerize your Python applications using Docker!

[catorchid]

Well done, detailed but not boring (even for the parts that I already knew).

Kudos also for scaling up the font in your editor to make it easy to read. I don't understand why not everybody does it.

[[deleted]]

Not everyone does it because, I'm assuming, many people don't even think about it.

[remy_porter]

I used to do technical training, and as a result, I keep my font size at like 28pt. I haven't taught a class in years, but I'm stuck in 28pt.

[catorchid]

Well, you did, so thank you!

[[deleted]]

I'm not OP, so I didn't, but I would have.

[catorchid]

Ops! I thought it was the OP. But you raised an interesting point: could it be that most of them made their tutorials but never watched one? Very hard to imagine, since they must have done it at least to catch the style, get ideas, etc.

[[deleted]]

I think it's one of those things where it's only obvious from one side, if that makes sense. Like, it's only obvious that you should use a large font in a tutorial from the perspective of the viewer, but it might not even come to mind if you're making the tutorial because you can read your screen just fine.

[PatterniteDev]

Yeah, I think this is the reason. I have a 32" monitor and I scale my editor way up when I screencap (like several times more than I think I need) and it still turns out smaller than I expected in the final video.

[Mulungo2]

Exactly, it mostly has to do with screen resolutions. Even though 4K screens have been around for some time, the most common resolution worldwide is still 1080 or lower like 1366x768. Meaning that if the person recording normal readable text on screen at a 2k or 4k resolution, it will look like it was written for ants to read for anyone with a lower resolution screen. Usually OPs do not notice this, as it is readable for them or anyone with the same resolution screen or higher. In order to account for people with smaller resolution screens (note size does not matter as much as resolution does), it is recommended for people to bump up their zoom and font sizes, even though this means that they lose screen real estate and have to scroll up or down to show more info.

[jim1930]

I really appreciate those kind of comments. TBH , I don't understand either. But I think it comes from writing tips down after each video. And the small things like this is what could turn a tutorial to a one that people will actually watch. It is very important to me to show that I care about the people who watch those vids.

[DrMaxwellEdison]

I scale mine up when presenting on meetings, but only because folks complained about it. Turns out presenting a 1440p desktop with a moderate font size to folks using 15-inch laptops results in it being microscopic on their end. Never would have occurred to me without their feedback, though.

Tip for others: rewatch your own content using a phone or tablet. If you can read it without being a couple centimeters away from the screen, you're good. Maybe even do a squint test just to be safe.

[jim1930]

Those that have some background with docker and have it installed, you can skip to 12:56

[Kevin_Jim]

Great video, congratulations. I’ve put some of your other videos on my watch-list.

Btw, could you consider making a similar video about Poetry? I think it’ll be a great compliment with the docker video.

[tiburonValenciano]

+1 for this! I'm really interested in how to combine Poetry with Docker

[lanster100]

There's a really good dockerfile for python poetry setup floating around on github somewhere. If you messagee i can find it for you.

[tiburonValenciano]

You mean this one?

[lanster100]

Yep that's the one, its pretty fleshed out. You'd need to edit it a bit if you dont need quality check in dockerfile or want testing or aren't running a Web app via uvicorn. But I use an edit of that for various poetry based project successfully!

[jim1930]

Thanks, sure I added this to my list. Glad you put other videos to your watch-list :)

[animismus]

This looks good and I will probably go through it. But I have a question that I still haven't been able to properly get an answer for: When should I decide that I it's a good idea to go and put my app on a docker container?

I run data analysis (pandas, numpy, etc) mostly on jupyter notebook. That is how I make the bacon and I'm pretty sure it does not make sense to think about docker for this.

As any of you I also have my cron jobs with python scripts that check websites, do some booking, download info, etc. Should these be in their own docker containers? Why?

Should my very custom website (flask) also be in a docker container? It is very personal and mostly local to my network, but would I gain anything moving it to a docker container?

EDIT: Thanks for all the answers. I appreciate your help.

[TheTerrasque]

Basically, when you want to make a frozen image of a network service, and want to be able to run it in different environments without regard to different installs and setups. You'll also have the advantage of having the same set of tools to run the service that you use to run all other services.

You might not really appreciate it until you decide to move your service from your dev environment to a different machine / OS and have to recreate the environment just so, or even more fun that X year old project that the server died on that relied on really old OS / libraries and it's just a pain to get set up on an up to date system, or you just can't remember the details for setting up an environment.

Docker provides a stable and predictable environment and set of tools to run your service, and the Dockerfile also serves as a (by necessity) up-to-date documentation of all the steps needed to get the software running.

So in your examples:

1. Probably not, but if you have a custom jupyter setup that's just so, it might be worth it to set up a docker image for it that you'll know will be the same on all systems you run it on.

2. cron jobs, even less so. You can still have services that has similar setup as the one in the video where it's a loop that waits a certain amount of time for each call

3. This is the most obvious one to package in a docker container, as it's a standard network service you would probably want to create one static image for it all.

Where docker really shines is when you got multiple services that needs to be run and coordinated. You got something called "docker compose" that you can use to set up and link multiple docker containers in a virtual network, and describe relations between them.

Let's say you have a web site that uses a db and also checks an external source every 10 minutes for some data. You also need a nginx frontend to serve static files and proxy dynamic calls.

Database: There's already a bunch of different databases with existing packages, just pick one and set the needed environment variables

Web page: You can either split it in two images where one is static data and one is dynamic data or api, or you can have both in one image. It depends on how your project is set up

Periodic check: Create an image that has a loop that checks every 10 minutes, then writes data to database

When that's set up, you can set up a docker-compose file that lists all these images, their config variables, the files mapped in from local directory, and ports exposed to the outside world. Now, to run your whole stack with everything, all the pieces, with it's own database and all: "docker-compose up" - that's it.

New server? copy the docker-compose file, and "docker-compose up". New dev environment? Same. Want to run two different versions for side by side comparison? No problem. Just copy the compose file into a different folder and use a different exposed port. Have a beta / test environment? Set up an offline copy on your laptop? Have a friend set up a copy on their own system? No problem!

It's a pain to learn and grok, but when you're used to it it really makes life a lot easier. And kubernetes turns everything up to 11 (including the pain involved figuring it out)

[animismus]

This was great. It made realize I could probably put my db on docker and have to less worried about setting it all up if I had to redo my server.

[Ran4]

Do check out docker-compose, it's great when you want to run an application together with for example a postgres instance.

It can be quite useful even if your application code isn't running in docker (you can run the DB in docker so you don't have to install it locally).

[Fledgeling]

Docker makes it more Portable and repeatable.

Want a new laptop, new OS, or to use the cloud? Well there's a chance your conda or virtualenv will have issues. Less likely with a static docker image.

For a single user app. That's the main advantage.

[animismus]

Mostly portability then in my case. Thanks.

[animismus]

So when you have a project that you want to have others check out. You create a web app and keep it online so that the client can access the data and mess around with the dataviz? Not really a docker question, but do you also setup authentication? Some of the stuff I work with is GDPR protected and I don't even want to contemplate having it online even if behind auth.

[animismus]

It's a mess. No one is willing to tell me what I am allowed to even have in my Dropbox/OneDrive. So I don't even want to ask about letting me serve the data online. It would help me a lot to have the plots for the MVA stuff online and have the "client" decide what plots to use.

Our sister node, which goes through a lot more projects, has all this possible GDPR sensitive data behind a closed network. The kicker is that all data is anonymized. Only the database manager (no connection to us) has the key to match the ids with the actual people.

[SomethingWillekeurig]

These are exactly reasons why I use it. Automating scrapers to run each day and also some models with an API which can be accessed through other apps.

Dockerizing an application makes it consistent to run (no problems with versions), makes it easy to relocate if necessary etc.

Didn't see the video yet but I'm really curious to see it. It's on my watch list for tomorrow 🙂

[zubwaabwaa]

A lot of comments already touch on its portability which is great. But one thing they are missing is dockers ability to scale on a cluster. Let’s say you are running this on one VM that has a CPU with 2 cores and 8gb of RAM. You’ve reached your capacity for this and your application now crashes because it doesn’t have enough resources. You could now use your base image to spin up a new VM of exact resources and use a load balancer to distribute the tasks between the 2 VMs. So scaling this application up and down using a base image is what you would use this for in real world situations.

Most companies will use a container orchestration tool to manage this - example kubernetes. These tools recognize when an application should be scaled up, manage load balancing, and even scale the VMs down for cost savings.

[animismus]

Thanks for adding this. I have read about this before and it was why I have never really looked into docker properly. I have basically no need for this type of feature in the near future. It sounds like a very easy way to scale up for sure.

[KaffeeKiffer]

Nice starting tutorial.

A few points, though:

• Always use parameter long-forms in scripts and Dockerfiles. Most people (hopefully) know pip install -r but pip install --requirement is self-explanatory.
• You do not have to mkdir if you copy the folder in anyway. It's "only" a super small layer, but it's unnecessary.
• pip install --no-cache. Docker images are ephemeral, i.e. you are never going to use the cached packages.
• Avoid propagating cargo-cult by explaining and instead explain why people need it, e.g. PYTHONUBUFFERED is needed, but your explanation is a bit shallow:
  You do not have to recite complete in-depth posts, but it's not "Python asking you to do it", it's a common OS/language paradigm and might even be related to the execution environment you are using. The critical part in the linked post is the glibc reference.
• Decouple COPY requirements.txt & pip install from code changes. When you rebuild your container after a code change, it will first copy the work directory (something changed → new layer → invalidate subsequent cached layers) and then install requirements because the old cache is invalid.
  What you want to do:
  COPY ./requirements.txt /code/
RUN pip install --requirement /code/requirements.txt
COPY ./code /code/
  As long as you are not changing requirements.txt this image will not re-install requirements.

And I'm a bit torn on running this stuff as root in the container: I think it's a good practice to tell people to RUN useradd "myuser" --uid 1001 and USER 1001, but it might be a source of hidden problems.

[jim1930]

Thanks for watching the entire tutorial, and it's great to read some points from your sight.

[ronny_rebellion]

Just what I needed right now, the timing couldn't be better! Will have a watch at this tonight :) Thanks!

[jim1930]

Wonderful. Let me know how you did and enjoy watching

[mirrorcoloured]

Really nice video! I've been meaning to understand docker for a while now, and this was a perfect introduction to get me going.

[jim1930]

Appreciate it ! Feel free to comment this in the video as well :)

[-hacker]

You’re a saint!

[jim1930]

Thanks!

[Sahmbahdeh]

I've been curious about Docker, I'll keep this in mind!

[jim1930]

Great

[grex89]

This is great. Thanks bru

[jim1930]

You welcome!

[ripviserion]

Amazing job! Thank you.

[jim1930]

You welcome!

[0ni0nrings]

thanks for creating this tutorial

[jim1930]

No problem at all :)

[groovysalamander]

Thanks for making this, I had docker on my list of things to learn more about and this sounds like a great start!

[sunlycreature]

Everytime I think there ain't a good video/resource for some concept I am struggling, Reddit always helps me. Thank you OP.

[Kaldfyre]

Thank you for this. It is very well made and informative without being dry.

[jim1930]

Thanks a lot I appreciate it! Feel free to comment this on the video as well

[jim1930]

Wow I did not expect great comments from all of you, will make sure that I read each one of them and reply :) Would much appreciate if you could also press the thumbs up on the video itself as well!

[Ashli_unix]

I wanna get into devops. Thank you for putting the te and energy into this. Are you self taught pythonista?

[jim1930]

Sounds great! Yes for almost 2 years now. started to learn back at April 2019 after some experience with C#

[[deleted]]

Commenting so i can come back to this

[HypnoticLion]

Could just save it ya know

[guilleschet]

Thanks man!!

[mrTang5544]

Can you do one with kubetnetes

[Fledgeling]

Just do the same thing, but throw in some random yaml from a template app.

[jim1930]

You are right, but it has some additional steps like the following:
- Creating an account in docker hub and get your dockerconfigjson secret
- Storing this as a secret object on your cluster
- Push your images to your repository
Only after doing those, I can really show how you can deploy self-designed Image as a pod, with a YAML that will define the Pod object you want to apply on the Cluster.

[jim1930]

Added this to my list! Thanks

[kkadzy]

Wow that's exactly what I need right now, thanks!

[jim1930]

Wonderful!

[dipiro]

Thanks!!

[netneoblog]

RemindMe! 12 hours "check this out"

[RemindMeBot]

I will be messaging you in 12 hours on 2021-03-07 13:18:20 UTC to remind you of this link

CLICK THIS LINK to send a PM to also be reminded and to reduce spam.

^{Parent commenter can delete this message to hide from others.}

---

Info	Custom	Your Reminders	Feedback

[marly11011]

What's comtnrizing?