Gitlab - templates

Welcome

Working as a consultant is hard. You have so many tasks to do. The day is a bit too short, but the opportunities to learn are awesome. Today I have time to compile my notes about GitLab, mostly CI part. However, as a code repository for organizations, GitLab is my favorite solution. Fast, easy to use, permission matrix, and deployment keys management. All this stuff is awesome. It's not a marketing post, but could be :)

GitLabCI

It's a very popular solution because it's easy to use and fast to set up, fortunately, it's free - if you don't cross the limit (400 CI/CD minutes per month). The main thing is setup here. It's just one file called .gitlab-ci.yml, placed in the repository root.

Problem

There are no problems in this article. That will be text about templating, and why they are helpful. Let's assume that we have a simple pipeline for deploying on VM. Also, take a look at code comments.

 1image: ubuntu:latest
 2
 3before_script:
 4    - apt update
 5    - apt install openssh-client -y
 6    - eval $(ssh-agent -s)
 7    # base64 is very usefull in case of id_rsa formating
 8    - ssh-add <(echo "$DEPLOYER_PRIVATE_KEY" | base64 -d)
 9    - mkdir -p ~/.ssh
10    - 'echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'
11
12stages:
13    # order of stages is important and is configure here
14    - create_env
15    - build
16    - deploy
17
18# now we need to build .env file for compilation time
19# that's tag driven deployment, so we deploy to particular
20# env based on tag
21
22build_the_dotfile_dev:
23    stage: create_env
24    script:
25      - echo "$DOTFILE_DEV" > .env
26    artifacts:
27      paths:
28        - .env
29    rules:
30      - if: '$CI_COMMIT_TAG =~ /^development/'
31
32build_the_dotfile_stg:
33    stage: create_env
34    script:
35      - echo "$DOTFILE_STG" > .env
36    artifacts:
37      paths:
38        - .env
39    rules:
40      - if: '$CI_COMMIT_TAG =~ /^staging/'
41
42build_the_dotfile_prod:
43    stage: create_env
44    script:
45      - echo "$DOTFILE_PROD" > .env
46    artifacts:
47      paths:
48        - .env
49    rules:
50      - if: '$CI_COMMIT_TAG =~ /^production/'
51
52# now we build our app
53# this step is generic
54build_the_app:
55    image: node:12-buster
56    stage: build
57    script:
58        - git checkout -b '$CI_COMMIT_TAG'
59        - yarn install
60        - NODE_ENV=production yarn build
61        - tar --warning=no-file-changed --exclude=current.tar.gz --exclude-vcs -zcf current.tar.gz ./ || [[ $? -eq 1 ]]
62    artifacts:
63      paths:
64        - current.tar.gz
65      expire_in: 2 hrs
66    rules:
67      - if: '$CI_COMMIT_TAG =~ /^development|^staging|^production/'
68
69# logic is simple
70# 1. we transfer our artifact to VM
71# 2. run the deployment script with some mv/pm2 logic
72# 3. environment based on TAG again
73deploy_front_to_dev:
74    stage: deploy
75    script:
76        - scp current.tar.gz deployer@$DEV_SERVER_IP:/home/appuser/front
77        - ssh deployer@$DEV_SERVER_IP "bash /home/appuser/deploy.sh front $CI_COMMIT_TAG"
78    rules:
79      - if: '$CI_COMMIT_TAG =~ /^development/'
80
81deploy_afront_to_stg:
82    stage: deploy
83    script:
84        - scp current.tar.gz deployer@$STG_SERVER_IP:/home/appuser/front
85        - ssh deployer@$STG_SERVER_IP "bash /home/appuser/deploy.sh front $CI_COMMIT_TAG"
86    rules:
87      - if: '$CI_COMMIT_TAG =~ /^staging/'
88
89deploy_front_to_prod:
90    stage: deploy
91    script:
92        - scp current.tar.gz deployer@$PROD_SERVER_IP:/home/appuser/front
93        - ssh deployer@$PROD_SERVER_IP "bash /home/appuser/deploy.sh front $CI_COMMIT_TAG"
94    rules:
95      - if: '$CI_COMMIT_TAG =~ /^production/'

That's nice, but also too long - 83 lines of code. Likewise, it produces too many typo possibilities...

Solution

Here comes templates. It's some kind of object, which we can just call in our pipeline. After adding templates code looks like that:

 1stages:
 2  - create_env
 3  - build
 4  - deploy
 5
 6# note dot on the begining
 7# templates must start with dot, otherwise
 8# will be executed
 9.env_template: &envs
10  stage: create_env
11  image: alpine
12  script:
13    - echo "$DOTFILE" > .env
14  artifacts:
15      paths:
16        - .env
17
18.deploy_template: &deploy_tmpl
19  stage: deploy
20  image: ubuntu:latest
21  before_script:
22    - apt update
23    - apt install openssh-client -y
24    - eval $(ssh-agent -s)
25    - ssh-add <(echo "$DEPLOYER_PRIVATE_KEY" | base64 -d)
26    - mkdir -p ~/.ssh
27    - 'echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'
28  script:
29    - scp current.tar.gz deployer@$SERVER_IP:/home/appuser/front
30    - ssh deployer@$SERVER_IP "bash /home/appuser/bin/deploy.sh front $CI_COMMIT_TAG"
31
32build_the_dotfile_dev:
33  variables:
34    DOTFILE: "$DOTFILE_DEV"
35  <<: *envs
36  rules:
37    - if: '$CI_COMMIT_TAG =~ /^development/'
38
39build_the_dotfile_stg:
40  variables:
41    DOTFILE: "$DOTFILE_STG"
42  <<: *envs
43  rules:
44    - if: '$CI_COMMIT_TAG =~ /^staging/'
45
46build_the_dotfile_prod:
47  variables:
48    DOTFILE: "$DOTFILE_PROD"
49  <<: *envs
50  rules:
51    - if: '$CI_COMMIT_TAG =~ /^production/'
52
53build_the_app:
54  image: node:12.22.1-buster
55  stage: build
56  before_script:
57    - yarn policies set-version 1.22.10
58  script:
59    - git checkout -b '$CI_COMMIT_TAG'
60    - yarn install
61    - NODE_ENV=production yarn build
62    - tar --warning=no-file-changed --exclude=current.tar.gz --exclude-vcs -zcf current.tar.gz ./ || [[ $? -eq 1 ]]
63  artifacts:
64    paths:
65      - current.tar.gz
66  rules:
67    - if: '$CI_COMMIT_TAG =~ /^development|^staging|^production/'
68
69deploy_front_to_dev:
70  variables:
71    SERVER_IP: "$DEV_SERVER_IP"
72  <<: *deploy_tmpl
73  rules:
74    - if: '$CI_COMMIT_TAG =~ /^development/'
75
76deploy_afront_to_stg:
77  variables:
78    SERVER_IP: "$STG_SERVER_IP"
79  <<: *deploy_tmpl
80  rules:
81    - if: '$CI_COMMIT_TAG =~ /^staging/'
82
83deploy_front_to_prod:
84  variables:
85    SERVER_IP: "$PROD_SERVER_IP"
86  <<: *deploy_tmpl
87  rules:
88    - if: '$CI_COMMIT_TAG =~ /^production/'

I know it's still long. But the logic is in one place not in 3, also managing the new environment will be easy and clean. You can add templates as separated files and then just import them.

Testing

Another intresting topic, it's important to check the syntax before committing to dev. So there is an easy option to test your stages. The only thing you need is to run this command in your local repository.

1docker run -d \
2  --name gitlab-runner \
3  --restart always \
4  -v $PWD:$PWD \
5  -v /var/run/docker.sock:/var/run/docker.sock \
6  gitlab/gitlab-runner:latest

Then to execute the task type.

1docker exec -it -w $PWD gitlab-runner gitlab-runner exec docker <task name>

There is one issue that makes me cry... I can't find an option to run the whole pipeline. Running stages is fine, but there is no option to test caching, or sharing files between tasks.

Summary

No summary require. It's clean and easy-to-use templates, GitLab is awesome, but life is hard. I'm 27yo, yesterday I bought my first bicycle helmet. Safety first. Take care.