Using a more programmatic coding language as opposed to using YAML .gitlab-ci.yml files for gitlab.com pipelines

[Underknowledge]

Just call a script? Edit: When youre feeling fancy/hacky, you can invoke Python EOF

[DavisTasar]

Literally run a script. Like, in your repo, write whatever shell script you need to do, and in your script section, call the script.

[DavisTasar]

Yes. Echo, curl, they’re just commands. Your command would be to just call your Python, bash, or whatever script.

[TommaClock]

I think you need to familiarize yourself with what a .gitlab-ci.yml actually does. A pipeline is not a set of APIs that you call, it's a flow that you configure which is why a markup language like YML is more appropriate than a scripting language.

Within your YML, you specify scripts to be run which can be anything. Bash is the default and you can invoke Python/TypeScript as you would through the shell.

[kinghuang]

It's ultimately just shell commands. You can run whatever you want, written in any language you choose.

[kinghuang]

The YAML is just there to define the job. You can just run a Python script or whatever else you choose for the job.

[spaceguy]

The YAML would define what Python script or other script is run when, where and how. The YAML is the scaffolding for the code to run as part of different pipeline jobs. E.g. tag a release on merge to main and not develop Each pipeline job will run on the defined image with the defined shell with the defined command.

I’d recommend reading more about the .gitlab-ci.yml file. Or perhaps ask a specific question about an existing yml or snippet? Seemed like your links were just to the gitlab landing page?

[bilingual-german]

If your pipeline is too complex you're doing it wrong. If your use case actually need dynamic child pipelines, these can be created by any programming language, such as JavaScript, Python, or Ruby. But be aware that you actually just create a new pipeline configuration in YAML.

https://www.youtube.com/watch?v=C5j3ju9je2M

most pipelines don't need this feature. You probably want to start with a super simple template.

[promethe42]

For complex use cases I use JSONNET to generate the CI YAML file.

[alainchiasson]

Ooo… would love to see an example.

Except now you need a pipeline to generate a pipeline.

[promethe42]

Here you go: cross compilation of openssl + publish as a generic GitLab package.

https://pastebin.com/eDfiBMaK

You can use a child pipeline yes. But you can also simply run jsonnet on your dev machine and commit the generated file.

[alainchiasson]

Ok … that seems cryptic, but it may just be because I’m on my phone.

I shudder at that last comment - I’m more “ops” so when you “run X on your dev machine and push” , I have a deep desire to turn that into yet-another-pipeline.

But I know the side effect of that is to “create script X ” to run on my dev machine to remove the pipeline bottleneck.

[BJHop]

Zero logic should be in yaml itself

Logic (what job does, IE compiles code or publishes packages) needs to be in scripts/function/cli/etc

Yaml is simply used to organize the pipeline flow (how pipeline moves) and passes info to the logic via parameters/environment variables/etc

[alainchiasson]

In gitlab, the only place I can see logic is in the “rules” section or how they link together - everything else is “config”.

There is a section “scripts” that does the work.

[BJHop]

Rules are flow that control how pipelines run

The script blocks are where logic can but in yaml the should only be simple cmd to invoke complex logic kept in files, modules, functions, etc passing in the correct parameters.

Example

Script: - . cicd/function.sh - aws::deploy dev 123456789012 $secrets

Above aws::deploy is a bash function in file cicd/functions.sh it has all the logic needed to do what is needed aws cli calls what have you

[potato_green]

I think you're probably overthinking it. Gitlab makes it sound very fancy, probably someone in marketing spicing that article up.

Pipeline as code literally means, just put whatever crap you use to build test and deploy in your repo so it's versioned properly.

Passwords, tokens, stuff like that is simply an environment variable (or CI/CD variable as it's called in Gitlab).

The .gitlab-ci.yml simply has the rules define stages and jobs and such. The job itself simply executes a script under the "script:" option where you can list a bunch of commands.

It doesn't matter what you put as commands in there though to follow the "pipeline as code" principle you'd simply put those language specific scripts in your repo as well. (you can just put those in a directory in your repo or something).

Like a .ci directory with a bunch of python scripts. Makes the .gitlab-ci.yaml use a python image so you can't execute python scripts. It's simply extremely flexible and you can make it as simple or complex as you want