Welcome

I like GitHub Actions, I like Kubernetes and I want to learn more about Helm. So maybe I should join these tools and make a smooth pipeline? Why not? Also, I switched to Fedora, and that's a great moment to checkout Podman in action. No time to wait, let's go.

Tools used in this episode

• GitHub Action
• Podman
• Kubernetes
• Terraform
• Helm
• GCP
• A bit of Golang :)

Build the app

The first step is building a small app. I decided to use Golang because it's an awesome language for microservices and testing is clear.

1. Create a directory for app and infra part

   1mkdir -pv app infra
   

2. Go to the app directory and create main.go

   1 2 3 4 5 6 7

   package main import "fmt" func main() { fmt.Println("Hello World!") }

3. Init go mod

   1go mod init 3sky/k8s-app
   

Write code

I decided to use Echo framework, I like it, it's fast and logger is easy to use. \

App has two endpoint:

• /hello - which return Hello World!

• /status - which retrun app status = OK

   1package main
   2
   3import (
   4    "net/http"
   5    "time"
   6    "github.com/labstack/echo/v4"
   7    "github.com/labstack/echo/v4/middleware"
   8)
   9// Greetings ...
  10type Greetings struct {
  11    Greet string    `json:"greet"`
  12    Date  time.Time `json:"date"`
  13}
  14// Status ...
  15type Status struct {
  16    Status string `json:"status"`
  17}
  18func main() {
  19     // Echo instance
  20     e := echo.New()
  21    // Middleware
  22    e.Use(middleware.LoggerWithConfig(middleware.LoggerConfig{
  23       Format: "method=${method}, uri=${uri}, status=${status}\n",
  24    }))
  25    e.Use(middleware.Recover())
  26    // Routes
  27    e.GET("/hello", HelloHandler)
  28    e.GET("/status", StatusHandler)
  29    // Start server
  30    e.Logger.Fatal(e.Start(":1323"))
  31}
  32// HelloHandler ...
  33func HelloHandler(c echo.Context) error {
  34    return c.JSON(http.StatusOK, &Greetings{Greet: "Hello, World!", Date: time.Now()})
  35}
  36// StatusHandler ...
  37func StatusHandler(c echo.Context) error {
  38    return c.JSON(http.StatusOK, &Status{Status: "OK"})
  39}
  

1. Download dependences

   1go mod tidy
   

2. Run the code

   1go run main.go
   

3. Add some basic tests

   1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41

   package main import ( "encoding/json" "net/http" "net/http/httptest" "testing" "github.com/labstack/echo/v4" ) var ( g = Greetings{} s = Status{} ) func TestGreetings(t *testing.T) { e := echo.New() req := httptest.NewRequest(http.MethodGet, "/", nil) rec := httptest.NewRecorder() c := e.NewContext(req, rec) HelloHandler(c) if rec.Code != 200 { t.Errorf("Expected status code is %d, but it was %d instead.", http.StatusOK, rec.Code) } json.NewDecoder(rec.Body).Decode(&g) if g.Greet != "Hello, World!" { t.Errorf("Expected value is \"Hello, World!\", but it was %s instead.", g.Greet) } } func TestStatus(t *testing.T) { e := echo.New() req := httptest.NewRequest(http.MethodGet, "/status", nil) rec := httptest.NewRecorder() c := e.NewContext(req, rec) StatusHandler(c) if rec.Code != 200 { t.Errorf("Expected status code is %d, but it was %d instead.", http.StatusOK, rec.Code) } json.NewDecoder(rec.Body).Decode(&s) if s.Status != "OK" { t.Errorf("Expected value is \"OK\", but it was %s instead.", s.Status) } }

4. Run tests

   1go test ./...
   

Containerization with Podman

We need to pack out an awesome app. To do that I decided to use Podman. What Podman is? It is a daemonless container engine for developing, managing, and running OCI Containers on your Linux System. Unfortunately, I prefer creating Dockerfile in Docker's way, Buildah is not for me at least now.

Create contianer

1. Create Dockerfile

    1# Dockerfile
    2FROM golang:alpine as builder
    3RUN apk add --no-cache git gcc libc-dev
    4WORKDIR /build/app
    5# Get depedences
    6COPY go.mod ./
    7RUN go mod download
    8# Run Testss
    9COPY . ./
   10RUN go test -v ./...
   11# Build app
   12RUN go build -o myapp
   13FROM alpine
   14COPY --from=builder /build/app/myapp ./myapp
   15EXPOSE 1323
   16CMD ["./myapp"]
   

2. Build an image

   1podman build -t k8s-app .
   

3. Run image

   1podman run -d -p 8080:1323 k8s-app:latest
   

4. Run basic curl's test

   1curl -s localhost:8080/status | jq .
   2curl -s localhost:8080 | jq .
   

Configure GCP

OK, we have working app now we need to create our Kubernetes cluster for our deployment.

Working with GCP

1. Auth into GCP

   1gcloud auth login
   

2. Create a new project

   1gcloud projects create [PROJECT_ID] --enable-cloud-apis
   2
   3# --enable-cloud-apis
   4# enable cloudapis.googleapis.com during creation
   5# example
   6# gcloud projects create calcium-hobgoblins --enable-cloud-apis
   

3. Check existing projects

   1gcloud projects list
   2PROJECT_ID               NAME                     PROJECT_NUMBER
   3calcium-hobgoblins       calcium-hobgoblins       xxxx
   

4. Set gcloud project

   1gcloud config set project calcium-hobgoblins
   

5. Create a service account and add necessary permission

    1gcloud iam service-accounts create calcium-hobgoblins-user \
    2--description "Service user for GKE and GitHub Action" \
    3--display-name "calcium-hobgoblins-user"
    4
    5gcloud projects add-iam-policy-binding calcium-hobgoblins --member \
    6serviceAccount:calcium-hobgoblins-user@calcium-hobgoblins.iam.gserviceaccount.com \
    7--role roles/compute.admin
    8
    9gcloud projects add-iam-policy-binding calcium-hobgoblins --member \
   10serviceAccount:calcium-hobgoblins-user@calcium-hobgoblins.iam.gserviceaccount.com \
   11--role roles/storage.admin
   12
   13gcloud projects add-iam-policy-binding creeping-hobgoblins --member \
   14serviceAccount:calcium-hobgoblins-user@calcium-hobgoblins.iam.gserviceaccount.com \
   15--role roles/container.admin
   16
   17gcloud projects add-iam-policy-binding calcium-hobgoblins --member \
   18serviceAccount:calcium-hobgoblins-user@calcium-hobgoblins.iam.gserviceaccount.com \
   19--role roles/iam.serviceAccountUser
   

6. List permission calcium-hobgoblins

   1gcloud projects get-iam-policy calcium-hobgoblins  \
   2--flatten="bindings[].members" \
   3--format='table(bindings.role)' \
   4--filter="bindings.members:calcium-hobgoblins-user@calcium-hobgoblins.iam.gserviceaccount.com"
   

Push initial image to container registry

After setting up cloud project we have finally access to the container registry.

Auth and Push

1. Authenticate container registry

   1gcloud auth activate-service-account \
   2calcium-hobgoblins-user@calcium-hobgoblins.iam.gserviceaccount.com \
   3--key-file=/home/kuba/.gcp/calcium-hobgoblins.json
   4
   5gcloud auth print-access-token | podman login \
   6-u oauth2accesstoken \
   7--password-stdin https://gcr.io
   

2. Push image into gcr.io

   1podman push localhostk8s-app:latest docker://gcr.io/calcium-hobgoblins/k8s-app:0.0.1
   

Provide Kubernetes Cluster

After setting up our GCP's project we need to provision out K8S cluster.

1. Create auth file

   1mkdir -pv ~/.gcp
   2cloud iam service-accounts keys create ~/.gcp/calcium-hobgoblins.json \
   3--iam-account calcium-hobgoblins-user@calcium-hobgoblins.iam.gserviceaccount.com
   

2. Create a basic directory structure

   1cd ../infra
   2mkdir -pv DEV Module/GKE
   

3. Terraform directory structure looks like that:

   1.
   2├── DEV
   3│   ├── main.tf
   4│   └── variables.tf
   5└── Module
   6    └── GKE
   7        ├── main.tf
   8        └── variables.tf
   

4. Init Terrafrom

   1cd DEV
   2terraform init
   

5. Permission are important

   If we forget about devstorage out cluster will have a problem with pulling images...

   1oauth_scopes = [
   2  "https://www.googleapis.com/auth/logging.write",
   3  "https://www.googleapis.com/auth/monitoring",
   4  "https://www.googleapis.com/auth/devstorage.read_only"
   5]
   

6. Terraform apply

   1terraform apply -var="path=~/.gcp/calcium-hobgoblins.json"
   

7. Config kubectl

   1export cls_name=my-gke-cluster
   2export cls_zone=europe-west3-a
   3gcloud container clusters list
   4gcloud container clusters get-credentials cls_name --zone cls_zone
   5kubectl get node
   

Prepare Helm release

When we have a working cluster, we can prepare helm chart. Also, it's a good time to install an ingress controller.

1. Init example helm chart

   1cd ../..
   2mkdir helm-chart
   3cd helm-chart
   4helm create k8s-app
   

2. Install Ingress with Helm(nginx)

   1helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
   2helm install release ingress-nginx/ingress-nginx
   

Add GitHub Action Pipeline

As an easy and great CI tool I decided to use GitHub Action again.

1. Add two files

   1mkdir -pv .github/workflows
   2touch .github/workflows/no-release.yml
   3touch .github/workflows/release.yml
   

2. Add content to no-release file

   This file will execute every time when code will be pushed to the repository.

3. Add content to release file

   This file will execute only when pushed code will be tagged with v* expression.

4. Set GH Secrets

   PROJECT_ID - it's project - calcium-hobgoblins GCP_SA_KEY - auth file in base64

   1cat ~/.gcp/calcium-hobgoblins.json | base64
   

5. Push some code into the repo

   1git push origin master
   2git push origin v.0.0.1
   

6. Check the status of pods

   1kubectl get pods
   2kubectl describe pod <pod-name>
   3helm list release-k8s-app
   

Summary

As you can see there is no source file for terraform and helm. I decided for that move because the post is long enough even without it :)
What else? I like Podman it just works without root permission on the host. I still have some problems with Buildah, it's a bit uncomfortable for me. Maybe in the future, or after another attempt.
Setting K8S cluster is easy with Terraform, but If we are planning production deployment all factors become more complicated.
Helm also looks like a nice tool in case a lot of similar deployment, also tracking release history is a cool feature. Unfortunately, it's not a magic tool and doesn't resolve all our CI/CD problems.

All code you can find here