Printing environment variables in GitLab CI/CD pipelines
We had the need to print the environment variables used as parameters of GitLab CI/CD pipelines. Right now there is no official straightforward way to do it, so I resorted to a tiny custom way to do it.
For both debugging and “explanability” reasons it is very handy to know which parameters were used to start GitLab CI/CD pipelines.
Jenkins, for examples, does exactly this out of the box. GitLab CI/CD is super nice but doesn’t have this feature at the moment.
So I eventually crafted a solution for that.
First of all, the head of our
.gitlab-ci.yml is the following:
variables: VARIABLE_1: value: "default value" description: "This variable is used to do x" VARIABLE_2: value: "" description: "This variable is used to do y" ...
Hence the following rules:
- Each pipeline has a configuration of environment variables, defined in the
- At the moment we aren’t interested in other non-defined variables.
The problem is that a simple job that print the
env also reveals other environment variables that are not interesting. So we need something more complex.
I explored these ideas:
Idea 1: the “diff”
inherit:variables option it is possible to get the difference between all the environment variables and only the ones that are injected by GitLab.
In other (totally not needed) words, let’s define:
- $A$: the set of all environment variables of the job.
- $D$: the set of the default injected variables.
Then, $X = A - D$ is the set of the defined variables (what we want).
This idea looks perfect on paper, but doesn’t work as expected, because you actually need two jobs to achieve it:
- Job 1: calculate $A$ with
- Job 2: calculate $D$ with
And in each job you get different values for some default variables that GitLab injects, so a simple
diff wouldn’t work.
(yes, you could work more on that path, for example by having a predefined list of unwanted default variables to filter out, but I didn’t want to hardcode anything)
Idea 2: YAML tricks
The idea is to fully exploit the characteristics of the YAML language to print the very
One way is to define an anchor to
variables and somehow
echo it, but I didn’t find a way to do it.
Idea 3: little custom solution
The images that we use to run the jobs contain some utilities, including
yq, which allows to parse and transform YAML.
So the idea is:
- Parse the very
.gitlab-ci.ymlfile to get a list of all the defined variable names.
- Flatten the result into a string representing a series of
- Use that expression to
grepthe result of
You can do this with a one-liner in a single job.
print_variables: stage: .pre script: - env | egrep $(yq '.variables | keys | ... comments = "" | map("^" + . + "=") | @csv' .gitlab-ci.yml | tr "," "|")
Let’s unpack it and explain it:
- First get the
- Pipe it to the result of a
- Parse the
variablessection of the YAML file.
- Just keep the keys.
- Remove comments.
- Map every
^VAR=, to create a regex to match any string that starts with
VARand ends with
- Transform all to a CSV string (
- Replace commas with pipes, to complete the regex (
- Parse the
The result is just the portion of
env which keys are defined in the
Do I like this solution? Yes and no. Is it perfect? Nope. I would have preferred something given by GitLab. Maybe there is a more elegant solution but I don’t know it. In any case, this solution works good, is fast, compact and yes, also elegant.