Mastodon

Clojure app on Google Cloud Run

calendar Mar 18, 2020 · 11 min read · Google Cloud Clojure GH Action

Welcome

I want to create a small pipeline based on GCP Run and GitHub Action. Also increasing skills while #statathome is the best possible solution for spending time. Let's make some real stuff.

Tools used in this episode

  • Google Cloud Platform
  • Terraform
  • Docker
  • Clojure
  • Github Action

Google Cloud Platform

Why Google Cloud Platform

I like Web GUI, command-line tools, GKE. I also have some credits to use. Cloud Run is very nice and cheap for a one-docker small app.

Let's code - Google Cloud Platform

  1. Create a new project

    1gcloud projects create [PROJECT_ID] --enable-cloud-apis
2
3# --enable-cloud-apis
4# enable cloudapis.googleapis.com during creation
5# example
6# gcloud projects create my-small-gcp-project --enable-cloud-apis

  2. Enable services

    1gcloud services list --available | grep -e run -e compute -e container
2gcloud services enable compute.googleapis.com
3gcloud services enable container.googleapis.com
4gcloud services enable run.googleapis.com

  3. Set project name

    1gcloud config set project my-small-gcp-project

  4. Create a service account and add necessary permission

    1gcloud iam service-accounts create ci-cd-user \
2--description "Account for interact with GCP Run, CR and GitHub" \
3--display-name "my-github-user"

    1gcloud projects add-iam-policy-binding my-small-gcp-project \
2--member \
3serviceAccount:[email protected] \
4--role roles/compute.admin

    1gcloud projects add-iam-policy-binding my-small-gcp-project \
2--member \
3serviceAccount:[email protected] \
4--role roles/run.serviceAgent

    1gcloud projects add-iam-policy-binding my-small-gcp-project \
2--member \
3serviceAccount:[email protected] \
4--role roles/run.admin

    1gcloud projects add-iam-policy-binding my-small-gcp-project \
2--member \
3serviceAccount:[email protected] \
4--role roles/storage.admin

Terraform

Why Terraform

I like working with IaaC approach and I need some testing machine. Especially for docker part, I don't like to work with Docker for Windows. Also if we want to reproduce solutions everywhere that's the fastest solution. Also sometimes ago I decided that running playgrounds in clouds is cheaper and faster than taking care of the workstation. With IaaC I need to have only terminal and code editor.

Let's code - Terraform

  1. Getting project credentials in JSON.

    1gcloud iam service-accounts keys create auth.json \
2--iam-account [email protected]

  2. Add auth.json to .gitigonre

    1echo "auth.json" > .gitignore
2echo ".terraform/" >> .gitignore

  3. Create main.tf

     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58

    locals {
    region_eu = "europe-west3-a"
    p_name = "my-small-gcp-project"
}

provider "google" {
    credentials = file("auth.json")
    project     = local.p_name
    region      = local.region_eu
}

// Terraform plugin for creating random ids
resource "random_id" "instance_id" {
    byte_length = 8
}

// A single Google Cloud Engine instance
resource "google_compute_instance" "default" {
count = 1
name         = "build-machine${random_id.instance_id.hex}"
machine_type = "e2-medium"
zone         = local.region_eu

boot_disk {
initialize_params {
    image = "ubuntu-1804-bionic-v20200129a"
}
}
metadata = {
    // everyone has rsa key, right ?
    ssh-keys = "kuba:${file("~/.ssh/id_rsa.pub")}"
}

// Make sure flask is installed on all new instances for later steps
metadata_startup_script = "sudo apt-get update; sudo apt-get upgrade -y; "

network_interface {
network = "default"
    access_config {
    // Include this section to give the VM an external ip address
    }
}
}

resource "google_compute_firewall" "default" {
    name    = "app-firewall"
    network = "default"

allow {
    protocol = "tcp"
    ports    = ["80"]
    }
}

// A variable for extracting the external ip of the instance
output "m1" {
    value = "${google_compute_instance.default.0.network_interface.0.access_config.0.nat_ip}"
}

  4. Initialize a working directory containing Terraform configuration files

    1terraform init

  5. Apply the changes required to reach the desired state of the configuration

    1terraform apply

  6. Connect to instance via ssh

    1ssh user@ip
2
3# user = form line `metadata` secion
4# ip = from `ip` variable output
5# Example
6# ssh [email protected]

Clojure

Why Clojure

Functional, dynamic type language. Dialect of Lisp, with Lisp the code-as-data philosophy and a powerful macro system. Not very popular, but I like it. Also working with a various solution is always fun.

Let's code - Clojure

  1. Setup Clojure

    1. Install java on Linux

      1sudo apt install openjdk-8-jre-headless -y
2java -version

    2. Install Clojure on Linux

      1curl -O https://download.clojure.org/install/linux-install-1.10.1.536.sh
2chmod +x linux-install-1.10.1.536.sh
3sudo ./linux-install-1.10.1.536.sh

    3. Install Leiningen

      1wget https://raw.githubusercontent.com/technomancy/leiningen/stable/bin/lein \
2-O /usr/bin/lein
3chmod a+x /usr/bin/lein
4lein

  2. Run new project

    1lein new app <app-name>
2cd <app-name>
3
4# example
5# lein new app clojure-raw-rest-api

  3. Check it works

     1cd <app-name>
 2lein run
 3
 4# example
 5# cd clojure-raw-rest-api
 6
 7# output
 8➜  clojure-raw-rest-api lein run
 9Hello, World!
10➜  clojure-raw-rest-api

  4. Let's do TDD so tests first

     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

    ;test/clojure_raw_rest_api/core_test.clj
...
(ns clojure-raw-rest-api.core-test
(:require [clojure.test :refer :all]
            [clojure-raw-rest-api.core :refer :all]
            [ring.mock.request :as mock]))

(deftest simple-page-test
(is (= (simple-page (mock/request :get "/"))
        {:status  200
        :headers {"Content-Type" "text/html"}
        :body    "<h1>Hello World</h1>"})))

(deftest app-status-test
(is (= (app-status (mock/request :get "/status"))
        {:status  200
        :headers {"Content-Type" "text/json"}
        :body    {:status "ok"}})))

(deftest enviroment-name-test
(is (= (enviroment-name (mock/request :get "/env"))
        {:status  200
        :headers {"Content-Type" "text/json"}
        :body    {:enviroment "dev"}})))

(deftest enviroment-missing-handler
(is (= (missing-handler (mock/request :get "/test"))
        {:status  404
        :headers {"Content-Type" "text/html"}
        :body {:status "Error, path not found!"}})))

  5. Make a basic REST API implementation

     1;src/clojure_raw_rest_api/core.clj
 2...
 3(ns clojure-raw-rest-api.core
 4(:require [ring.adapter.jetty :as jetty]
 5    [ring.middleware.params :refer [wrap-params]]
 6    [ring.middleware.reload :refer [wrap-reload]]
 7    [ring.middleware.keyword-params :refer [wrap-keyword-params]]
 8    [ring.middleware.json :refer [wrap-json-params wrap-json-response]]
 9    [clojure.java.io :as io]
10    [clj-http.client :as client])
11(:gen-class))
12
13; Read enviroment variable
14(def env (or (System/getenv "env") "dev"))
15
16; Simple page
17(defn simple-page [req]
18{:status  200
19:headers {"Content-Type" "text/html"}
20:body    "<h1>Hello World</h1>"})
21
22; Return Health Check
23(defn app-status [req]
24{:status  200
25:headers {"Content-Type" "text/json"}
26:body    {:status "ok"}})
27
28; Return env(env var)
29(defn enviroment-name [req]
30{:status  200
31:headers {"Content-Type" "text/json"}
32:body    {:enviroment env}})
33
34; Return when no path
35(defn missing-handler [request]
36{:status 404
37:headers {"Content-Type" "text/html"}
38:body {:status "Error, path not found!"}})
39
40(def routes [
41    {:methods #{:get} :path "/" :handler simple-page}
42    {:methods #{:get} :path "/status" :handler app-status}
43    {:methods #{:get} :path "/env" :handler enviroment-name}
44])
45
46(defn route-match? [request route]
47(and ((:methods route) (:request-method request))
48    (= (:path route) (:uri request))))
49
50(defn app [request]
51(let [route (first (filter (partial route-match? request) routes))
52        handler (get route :handler missing-handler)]
53    (println "app request "
54    (:request-method request) (:uri request) (pr-str route))
55    (handler request)))
56
57(defn with-middleware [handler]
58(-> handler
59    (wrap-reload)
60    (wrap-keyword-params)
61    (wrap-json-params {})
62    (wrap-params {})
63    (wrap-json-response {:pretty true})))
64
65(defn -main []
66(let [port (Integer. (or (System/getenv "PORT") 8081))]
67    (jetty/run-jetty (with-middleware app) {:port port :join? false})))

  6. Ahh and project.clj update

     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

    (defproject clojure-raw-rest-api "1.0.0"
:license {:name "EPL-2.0 OR GPL-2.0-or-later WITH Classpath-exception-2.0"
            :url "https://www.eclipse.org/legal/epl-2.0/"}
:dependencies [[org.clojure/clojure "1.10.0"]
    [ring/ring-jetty-adapter "1.4.0"]
    [ring/ring-json "0.4.0"]
    [ring/ring-devel "1.4.0"]
    [clj-http "2.2.0"]
    [ring/ring-mock "0.4.0"]]
:main ^:skip-aot clojure-raw-rest-api.core
:target-path "target/%s"
:profiles {:uberjar {:aot :all}})

  7. Then run test

     1lein test
 2
 3# output
 4➜  clojure-raw-rest-api lein test
 5
 6lein test clojure-raw-rest-api.core-test
 7
 8Ran 4 tests containing 4 assertions.
 90 failures, 0 errors.
10➜  clojure-raw-rest-api ✗

  8. Create jar

    1lein uberjar
2
3# output
4➜  clojure-raw-rest-api git:(master) ✗ lein uberjar
5Compiling clojure-raw-rest-api.core
62020-03-19 11:36:40.298:INFO::main: Logging initialized @3998ms
7Created /home/kuba/Desktop/Projekty/clojure-raw-rest-api/target/uberjar/clojure-raw-rest-api-1.0.0.jar
8Created /home/kuba/Desktop/Projekty/clojure-raw-rest-api/target/uberjar/clojure-raw-rest-api-1.0.0-standalone.jar

  9. Great now I'm ready to pack it into Docker container

Docker

Why Docker

I want to have a container. At this point, I'm familiar with Docker, however, I need to take a look an another solution like distroless.

Let's code - Docker

  1. Install docker on VM

    1sudo apt install docker.io

  2. Add user to correct group

    1sudo usermod -aG docker $USER

  3. Re-login to apply changes

  4. Check installation

    1docker run hello-world

Dockerfile

  1. Create Dockerfile

    1touch Dockerfile

     1FROM clojure as builder
 2RUN mkdir -p /usr/src/app
 3WORKDIR /usr/src/app
 4COPY COPY clojure-raw-rest-api/ ./
 5RUN lein test
 6RUN mv "$(lein uberjar | sed -n 's/^Created \(.*standalone\.jar\)/\1/p')" app-standalone.jar
 7
 8FROM openjdk:8-jre-alpine
 9
10COPY --from=builder /usr/src/app/app-standalone.jar ./
11ENTRYPOINT ["java", "-jar", "app-standalone.jar"]
12
13EXPOSE 8081

  2. Test build

    1docker build . -t clojure-app

  3. Run app with an environment variable

    1docker run -p 8081:80 -d -e env="staging" clojure-app

  4. Test it

    1curl  $(curl -s -H "Metadata-Flavor: Google" http://metadata/computeMetadata/v1/instance/network-interfaces/0/access-configs/0/external-ip):8081/environment
2# {"environment": "staging"}

  5. Tag image

    1export PROJECT_ID="my-small-gcp-project"
2docker tag clojure-app gcr.io/$PROJECT_ID/clojure-app:1.0.0

  6. Copy auth.json from workstation

    1scp auth.json kuba@<ip>:<user_home>
2
3# IP of remote machine
4# place to copy file
5# example
6# scp auth.json [email protected]:/home/kuba

  7. Push to Container Registry

     1gcloud auth configure-docker
 2gcloud auth activate-service-account \
 3[email protected] \
 4--key-file=auth.json
 5
 6gcloud auth print-access-token | docker login \
 7-u oauth2accesstoken \
 8--password-stdin https://gcr.io
 9
10docker push gcr.io/$PROJECT_ID/clojure-app:1.0.0

  8. Create Cloud Run Service for prod

    1gcloud run deploy prod-awesome-clojure-api \
2--platform managed \
3--allow-unauthenticated \
4--region europe-west1 \
5--port 8081 \
6--set-env-vars=env='production' \
7--image gcr.io/$PROJECT_ID/clojure-app:1.0.0

  9. Create Cloud Run Service for non-prod

    1gcloud run deploy staging-awesome-clojure-api \
2--platform managed \
3--allow-unauthenticated \
4--region europe-west1 \
5--port 8081 \
6--set-env-vars=env='staging' \
7--image gcr.io/$PROJECT_ID/clojure-app:1.0.0

  10. Destroy infra that's important

    1terraform destroy

GitHub Action - CI/CD base

Why GitHub Action

I like GitHub Action, I enjoy working with it. It just works and is fast, well-documented, free, etc. It's not a complicated application so I can't see any reason to use Jenkins.

Let's code - GitHub Action

  1. Add 3 variables as GitHub Secrets

    1. GCP_SA_EMAIL - service account email: ci-cd-user@$my-small-gcp-project.iam.gserviceaccount.com

    2. GCP_SA_KEY - encoded auth.json

      1cat auth.json | base64

    3. PROJECT_ID - project name: my-small-gcp-project

  2. Put two file into .github\workflows

    1touch master.yml
2touch no-master.yml

  3. Push/PR to master - master.yml

     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67

    name: Build from master

on:
push:
    branches:
    - master

env:
    STG_SERVICE: "staging-awesome-clojure-api"
    STG_URL: "https://staging-awesome-clojure-api-jsrwhbaamq-ew.a.run.app"
    PROD_SERVICE: "prod-awesome-clojure-api"
    PROD_URL: "https://prod-awesome-clojure-api-jsrwhbaamq-ew.a.run.app"
    APP_NAME: "clojure-app"

jobs:
build-the-app:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v2
    - name: Setup GCP
      uses: GoogleCloudPlatform/github-actions/setup-gcloud@master
      with:
        version: '281.0.0'
        service_account_email: ${{ secrets.GCP_SA_EMAIL }}
        service_account_key: ${{ secrets.GCP_SA_KEY }}
        export_default_credentials: true
    - name: auth to CR
      run: gcloud auth configure-docker
    - name: build app
      run: docker build . -t $APP_NAME
    - name: tag app
      run: docker tag $APP_NAME gcr.io/${{ secrets.PROJECT_ID }}/$APP_NAME:${{ github.sha }}
    - name: push image
      run: docker push gcr.io/${{ secrets.PROJECT_ID }}/$APP_NAME:${{ github.sha }}

deploy-to-stg:
    needs: build-the-app
    runs-on: ubuntu-latest
    steps:
    - name: Setup GCP
      uses: GoogleCloudPlatform/github-actions/setup-gcloud@master
      with:
        version: '281.0.0'
        service_account_email: ${{ secrets.GCP_SA_EMAIL }}
        service_account_key: ${{ secrets.GCP_SA_KEY }}
        export_default_credentials: true
    - name: update staging
      run: gcloud run deploy $STG_SERVICE --project ${{ secrets.PROJECT_ID }} --platform managed --region europe-west1 --image gcr.io/${{ secrets.PROJECT_ID }}/$APP_NAME:${{ github.sha }}
    - name: Check stg
      run: if [[ ! $(curl -s $STG_URL/status | grep ok) ]]; then exit 1; fi

deploy-to-prod:
    needs: [build-the-app, deploy-to-stg]
    runs-on: ubuntu-latest
    steps:
    - name: Setup GCP
      uses: GoogleCloudPlatform/github-actions/setup-gcloud@master
      with:
        version: '281.0.0'
        service_account_email: ${{ secrets.GCP_SA_EMAIL }}
        service_account_key: ${{ secrets.GCP_SA_KEY }}
        export_default_credentials: true
    - name: update prod
      run: gcloud run deploy $PROD_SERVICE --project ${{ secrets.PROJECT_ID }} --platform managed --region europe-west1 --image gcr.io/${{ secrets.PROJECT_ID }}/$APP_NAME:${{ github.sha }}
    - name: Check stg
      run: if [[ ! $(curl -s $PROD_URL/status | grep ok) ]]; then exit 1; fi
    - run: "echo PROD status: ${{ steps.get_prod_status.response }}"

  4. Push/PR to not master - no-master.yml

     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18

    name: Build from no-master

on:
push:
    branches:
    - '*'
    - '!master'
pull_request:
    branches:
    - '*'

jobs:
build-test-docker:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v2
    - name: build app
    run: docker build . -t clojure-app

Bonus - Telegram

Why Telegram

I use Telegram every day, build are long so I decided to add a notification.

Let's code - Telegram

  1. Add to master.yaml

     1
 2
 3
 4
 5
 6
 7
 8
 9
10

    send-notification:
  needs: [build-the-app, deploy-to-stg, deploy-to-prod]
  runs-on: ubuntu-latest
  steps:
  - name: test telegram notification
    uses: appleboy/telegram-action@master
    with:
      to: ${{ secrets.TELEGRAM_TO }}
      token: ${{ secrets.TELEGRAM_TOKEN }}
      message: Build number ${{ github.run_number }} of ${{ github.repository }} is complete ;)

  2. Telegram configuration

    1. type /help

    2. type /newbot

    3. generate bot name like superbot, not unique

    4. generate bot username like super-uniqe-bot, must be unique

    5. get a token /token

    6. save token

    7. subscribe bot

    8. use REST API to received TELEGRAM_TO

      1curl -s https://api.telegram.org/bot<token>/getUpdates | jq.
2
3# example URL
4# https://api.telegram.org/bot123456:ABC-DEF1234ghIkl-zyx57W2v1u123ew11/getUpdates

    9. TELEGRAM_TO is field chat.id - 353824382

  3. Configure notification

    I would like to get some notification after the build. Telegram is a nice tool, and there is already created [GH Action][7].

    1
2
3
4
5
6

    - name: test telegram notification
  uses: appleboy/telegram-action@master
  with:
    to: ${{ secrets.TELEGRAM_TO }}
    token: ${{ secrets.TELEGRAM_TOKEN }}
    message: Build number ${{ github.run_number }} of ${{ github.repository }} is complete ;)

    That snipped contains two secrets secrets.TELEGRAM_TO and secrets.TELEGRAM_TOKEN. Again I can recommend this docs. But I received this value in the previous section. \

    There are also two context variable github.run_number and github.repository. And again docs are more than enough.

Logging

I use Google Cloud Run, so Google manages all logs from services. In the case of this architecture usage, normal logs collector is an overstatement.

just logs

Monitoring and alerts

For this, I decided to use the basic features of GCP.

Uptime Checks

  1. Go to Monitoring -> Create uptime checks

just uptime

Alerting policies

  1. Go to Monitoring -> Create alerting policies

just alert

Summary

That was a nice project. I have the opportunity to learn about IAM and GCP Run. Also, I have a better understanding of GitHub Action pipelines. Clojure looks a bit strange, but testing functional code is sweet. I never write such readable test cases. The source code of the project is here, feel free to use.