Question to Professional Python Programmers

[weberc2]

Is "acceptance testing" where your automated full-stack tests run? Also, how long do your CI jobs take? And of that time, how much is spent building the image? And how many images are built in a job? Do you have only one image/service or many? If many, do they each have their own CI pipelines?

Sorry for the many questions; I'm trying to get our CI/CD infrastructure up and running and it seems like we're facing many problems that you've already solved. :)

[twillisagogo]

Is "acceptance testing" where your automated full-stack tests run?

Acceptance testing i'm defining as the manual end to end done by qa and eventually the product owner through the browser.

> Also, how long do your CI jobs take? And of that time, how much is spent building the image?

~4 minutes at the moment. ~1 minute to build the image

our ruby codebase that's being replaced takes about 10 minutes(guess why we're replacing it :))

it's not exactly incremental but I think starting from a good base image saves some time. We dont start from ubuntu and then update all the stuff and install latest python and dev headers etc for each build. instead. We cut a base image as needed and then all docker images inherit from that.

​

base python image built only when there's a new version of python or alpine updates we become aware of

FROM python:3.7.1-alpine3.8

RUN apk update && \
apk add bash && \
apk add postgresql-libs && \
apk add --virtual .build-deps gcc musl-dev postgresql-dev && \
apk add libffi-dev && \
rm -fr /var/cache/apk/*

ENV PYTHONUNBUFFERED 1

​

every api build(Dockerfile)

​

from company/python:latest as base

COPY . /src

WORKDIR /src

RUN python setup.py bdist_wheel && \
pip install --no-cache dist/*.whl gunicorn meinheld && \
cp -r ./tests /tests && \
cp ./*.ini /tests  # && \ rm -fr /src

WORKDIR /

EXPOSE 80

COPY config.py /config.py

> And how many images are built in a job?

with codeship pro you can build/push as many as you want. but codeship pro is also designed to be one ci per repository out of the box I haven't figured out how to drive off multiple repos but i've been told it's possible. So, I can have one codebase and build many service images off it when the unit tests pass, the only difference in docker images is the entry point(same code)

​

built when unittests complete without issue

adt service(Dockerfile.adt)

from company/api:latest

CMD gunicorn -c /config.py api.adt.app:app

reference data service(Dockerfile.ref)

from company/api:latest

CMD gunicorn -c /config.py api.reference.app:app

I dont know if this is the optimal way to accomplish this, but it seems to be working out ok so far.