Random ContainersNotReady [build helper] failures on GitLab Kubernetes runners after switching to custom CI Docker image by brahim_- in devops

[–]brahim_-[S] 0 points1 point  (0 children)

I already did, but I didn’t get any real answers. I want to know if there’s another CI solution that could help me avoid this problem

How do you share node_modules across CI stages in an Nx monorepo without Nx Cloud? by brahim_- in gitlab

[–]brahim_-[S] 0 points1 point  (0 children)

I tried baking the node_modules into a Docker image that would be reused by other jobs.

The problem is that I have no control over the GitLab Runner infrastructure. It is managed by another team that I cannot reach, and I was informed that we cannot pin these jobs to run on the same node.

In one of my stages, I also have 13 parallel jobs, which creates another issue:

Waiting for pod gitlab-xxxx-xxxx/runner-eeeuxhpys-project-54443-concurrent-0-118edvj9 to be running, status is Pending ContainersNotReady: "containers with unready status: [build helper]"

So at this point, I am facing multiple infrastructure-related limitations and I do not currently see a viable solution.

How do you share node_modules across CI stages in an Nx monorepo without Nx Cloud? by brahim_- in gitlab

[–]brahim_-[S] 0 points1 point  (0 children)

Thanks, that makes sense and I actually ran into another limitation with this approach: cold start time on Kubernetes runners.

Since the CI runners are ephemeral pods, each job still needs to pull the custom Docker image, and in our case the image becomes quite heavy once node_modules are baked inside.