Gitlab - templates
Welcome
Working as a consultant is hard. You have so many tasks to do. The day is a bit too short, but the opportunities to learn are awesome. Today I have time to compile my notes about GitLab, mostly CI part. However, as a code repository for organizations, GitLab is my favorite solution. Fast, easy to use, permission matrix, and deployment keys management. All this stuff is awesome. It's not a marketing post, but could be :)
GitLabCI
It's a very popular solution because it's easy to use and fast to set up, fortunately, it's free -
if you don't cross the limit (400 CI/CD minutes per month). The main thing is setup here. It's just one
file called .gitlab-ci.yml, placed in the repository root.
Problem
There are no problems in this article. That will be text about templating, and why they are helpful. Let's assume that we have a simple pipeline for deploying on VM. Also, take a look at code comments.
1image: ubuntu:latest
2
3before_script:
4 - apt update
5 - apt install openssh-client -y
6 - eval $(ssh-agent -s)
7 # base64 is very usefull in case of id_rsa formating
8 - ssh-add <(echo "$DEPLOYER_PRIVATE_KEY" | base64 -d)
9 - mkdir -p ~/.ssh
10 - 'echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'
11
12stages:
13 # order of stages is important and is configure here
14 - create_env
15 - build
16 - deploy
17
18# now we need to build .env file for compilation time
19# that's tag driven deployment, so we deploy to particular
20# env based on tag
21
22build_the_dotfile_dev:
23 stage: create_env
24 script:
25 - echo "$DOTFILE_DEV" > .env
26 artifacts:
27 paths:
28 - .env
29 rules:
30 - if: '$CI_COMMIT_TAG =~ /^development/'
31
32build_the_dotfile_stg:
33 stage: create_env
34 script:
35 - echo "$DOTFILE_STG" > .env
36 artifacts:
37 paths:
38 - .env
39 rules:
40 - if: '$CI_COMMIT_TAG =~ /^staging/'
41
42build_the_dotfile_prod:
43 stage: create_env
44 script:
45 - echo "$DOTFILE_PROD" > .env
46 artifacts:
47 paths:
48 - .env
49 rules:
50 - if: '$CI_COMMIT_TAG =~ /^production/'
51
52# now we build our app
53# this step is generic
54build_the_app:
55 image: node:12-buster
56 stage: build
57 script:
58 - git checkout -b '$CI_COMMIT_TAG'
59 - yarn install
60 - NODE_ENV=production yarn build
61 - tar --warning=no-file-changed --exclude=current.tar.gz --exclude-vcs -zcf current.tar.gz ./ || [[ $? -eq 1 ]]
62 artifacts:
63 paths:
64 - current.tar.gz
65 expire_in: 2 hrs
66 rules:
67 - if: '$CI_COMMIT_TAG =~ /^development|^staging|^production/'
68
69# logic is simple
70# 1. we transfer our artifact to VM
71# 2. run the deployment script with some mv/pm2 logic
72# 3. environment based on TAG again
73deploy_front_to_dev:
74 stage: deploy
75 script:
76 - scp current.tar.gz deployer@$DEV_SERVER_IP:/home/appuser/front
77 - ssh deployer@$DEV_SERVER_IP "bash /home/appuser/deploy.sh front $CI_COMMIT_TAG"
78 rules:
79 - if: '$CI_COMMIT_TAG =~ /^development/'
80
81deploy_afront_to_stg:
82 stage: deploy
83 script:
84 - scp current.tar.gz deployer@$STG_SERVER_IP:/home/appuser/front
85 - ssh deployer@$STG_SERVER_IP "bash /home/appuser/deploy.sh front $CI_COMMIT_TAG"
86 rules:
87 - if: '$CI_COMMIT_TAG =~ /^staging/'
88
89deploy_front_to_prod:
90 stage: deploy
91 script:
92 - scp current.tar.gz deployer@$PROD_SERVER_IP:/home/appuser/front
93 - ssh deployer@$PROD_SERVER_IP "bash /home/appuser/deploy.sh front $CI_COMMIT_TAG"
94 rules:
95 - if: '$CI_COMMIT_TAG =~ /^production/'
That's nice, but also too long - 83 lines of code. Likewise, it produces too many typo possibilities...
Solution
Here comes templates. It's some kind of object, which we can just call in our pipeline. After adding templates code looks like that:
1stages:
2 - create_env
3 - build
4 - deploy
5
6# note dot on the begining
7# templates must start with dot, otherwise
8# will be executed
9.env_template: &envs
10 stage: create_env
11 image: alpine
12 script:
13 - echo "$DOTFILE" > .env
14 artifacts:
15 paths:
16 - .env
17
18.deploy_template: &deploy_tmpl
19 stage: deploy
20 image: ubuntu:latest
21 before_script:
22 - apt update
23 - apt install openssh-client -y
24 - eval $(ssh-agent -s)
25 - ssh-add <(echo "$DEPLOYER_PRIVATE_KEY" | base64 -d)
26 - mkdir -p ~/.ssh
27 - 'echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'
28 script:
29 - scp current.tar.gz deployer@$SERVER_IP:/home/appuser/front
30 - ssh deployer@$SERVER_IP "bash /home/appuser/bin/deploy.sh front $CI_COMMIT_TAG"
31
32build_the_dotfile_dev:
33 variables:
34 DOTFILE: "$DOTFILE_DEV"
35 <<: *envs
36 rules:
37 - if: '$CI_COMMIT_TAG =~ /^development/'
38
39build_the_dotfile_stg:
40 variables:
41 DOTFILE: "$DOTFILE_STG"
42 <<: *envs
43 rules:
44 - if: '$CI_COMMIT_TAG =~ /^staging/'
45
46build_the_dotfile_prod:
47 variables:
48 DOTFILE: "$DOTFILE_PROD"
49 <<: *envs
50 rules:
51 - if: '$CI_COMMIT_TAG =~ /^production/'
52
53build_the_app:
54 image: node:12.22.1-buster
55 stage: build
56 before_script:
57 - yarn policies set-version 1.22.10
58 script:
59 - git checkout -b '$CI_COMMIT_TAG'
60 - yarn install
61 - NODE_ENV=production yarn build
62 - tar --warning=no-file-changed --exclude=current.tar.gz --exclude-vcs -zcf current.tar.gz ./ || [[ $? -eq 1 ]]
63 artifacts:
64 paths:
65 - current.tar.gz
66 rules:
67 - if: '$CI_COMMIT_TAG =~ /^development|^staging|^production/'
68
69deploy_front_to_dev:
70 variables:
71 SERVER_IP: "$DEV_SERVER_IP"
72 <<: *deploy_tmpl
73 rules:
74 - if: '$CI_COMMIT_TAG =~ /^development/'
75
76deploy_afront_to_stg:
77 variables:
78 SERVER_IP: "$STG_SERVER_IP"
79 <<: *deploy_tmpl
80 rules:
81 - if: '$CI_COMMIT_TAG =~ /^staging/'
82
83deploy_front_to_prod:
84 variables:
85 SERVER_IP: "$PROD_SERVER_IP"
86 <<: *deploy_tmpl
87 rules:
88 - if: '$CI_COMMIT_TAG =~ /^production/'
I know it's still long. But the logic is in one place not in 3, also managing the new environment will be easy and clean. You can add templates as separated files and then just import them.
Testing
Another intresting topic, it's important to check the syntax before committing to dev. So there is an easy option to test your stages. The only thing you need is to run this command in your local repository.
1docker run -d \
2 --name gitlab-runner \
3 --restart always \
4 -v $PWD:$PWD \
5 -v /var/run/docker.sock:/var/run/docker.sock \
6 gitlab/gitlab-runner:latest
Then to execute the task type.
1docker exec -it -w $PWD gitlab-runner gitlab-runner exec docker <task name>
There is one issue that makes me cry... I can't find an option to run the whole pipeline. Running stages is fine, but there is no option to test caching, or sharing files between tasks.
Summary
No summary require. It's clean and easy-to-use templates, GitLab is awesome, but life is hard. I'm 27yo, yesterday I bought my first bicycle helmet. Safety first. Take care.