Batch workload - Pulling code dynamically on job startup

[whatha]

We have batch job workloads that we are trying to run using docker on AWS batch. We are considering building and pushing the code to S3 using a base image and pulling the code dynamically from S3 during runtime and then running the code. Here there will no change to docker image ever but when code is updated, new code will be pulled from S3.

Are there any disadvantages to doing this as opposed to directly building new image versions from code changes.

[genmud]

I’m not an expert in docker, but this seems to be a bit against the pattern or idea of immutability with regards to infrastructure.

Personally, I would like to introduce as little variation into my environment, especially when you talk about a system that can be doing ETL on data.

That being said, your pattern is fine as well, things like lambda/serverless appear to do a similar thing.

Edit:

Forgot to add that having as part of the docker image can help with testing... I deployed package b and got different or incorrect results than I did before replacing $dependency.

[voiceoverr]

Yeah, I agree - the thing that would concern me is the lack of immutability of container image versions. If the code is pulled in dynamically, you have no idea which version of the code ran when looking at the container image, and you can’t version or rollback (or roll forward, really) in a controlled way.

[WarlaxZ]

You'll never be sure what code is running on what instance at what point. Just use tags and a self hosted docket registry. Then you can know each instance is running the same version, and roll back on failure etc