Cloud-init with DigitalOcean

Welcome

Another build something yourself article. The last one was focused on Kubernetes. Now I will try to build infrastructure for a bit different side-project. I always want to learn SaltStack. To be honest, I'm a huge fan of Ansible which is boring, but works. However, sometimes I have a problem with keeping my developers account synced. Also adding multiple users and track their settings is hard without states. So I decided to get into the new toolbox. Additionally cloud-init with DigitalOcean API sounds like fun.

Components

No SaltStack for now, just an infra setup.

Setting the environment

To build our infrastructure we don't need much stuff. The first one is Access Token. DigitalOcean has great docs so, I just refer to it. When we get our token, the next step is getting the region's name for our server. That can be done via API, doctl, or just google search. However I very like their(DO) CLI tool, also there is ZSH autocompletion(yay!).

 1$ doctl compute region list
 2Slug    Name               Available
 3nyc1    New York 1         true
 4sfo1    San Francisco 1    false
 5nyc2    New York 2         false
 6ams2    Amsterdam 2        false
 7sgp1    Singapore 1        true
 8lon1    London 1           true
 9nyc3    New York 3         true
10ams3    Amsterdam 3        true
11fra1    Frankfurt 1        true
12tor1    Toronto 1          true
13sfo2    San Francisco 2    false
14blr1    Bangalore 1        true
15sfo3    San Francisco 3    true

We're interested in slug variable. In my case it will be fra1, I'm based in Poland. Fortunately, you can choose whatever you want. Setup is ready, so let's "code" something.

Cloud-init

Why cloud-init? Because it's a popular init script mechanism. It can be used in AWS as well in DO, or OpenStack. It's quite simple, started almost at boot time and what is important doesn't require any ready VM image. To be clear, immutable infrastructure is awesome. Sometimes if we just want a cheap solution(no storage cost), or boot time is not an important factor, then init-scripts are very handy.

File content

 1#cloud-config
 2# first create a user with
 3# ssh key, permission, etc
 4users:
 5  - name: kuba
 6    ssh-authorized-keys:
 7      - [email protected] AAAAGnNrLXNzaC1lZDI1NTE5QG9wZW5zc2guY29tAAAAILjtneTPZiRcjxcM2xiAvvk1BvZGyBibtTx0i+szJutMAAAABHNzaDo= jakub.wolynko@yubi
 8    sudo: ['ALL=(ALL) NOPASSWD:ALL']
 9    groups: sudo
10    shell: /bin/bash
11
12# then upgrade base image
13package_update: true
14package_upgrade: true
15package_reboot_if_required: true
16
17# add security to ssh config
18write_files:
19  - path: /etc/ssh/sshd_config
20    content: |
21         Port 31
22         Protocol 2
23         HostKey /etc/ssh/ssh_host_dsa_key
24         HostKey /etc/ssh/ssh_host_ecdsa_key
25         HostKey /etc/ssh/ssh_host_ed25519_key
26         UsePrivilegeSeparation yes
27         KeyRegenerationInterval 3600
28         ServerKeyBits 1024
29         SyslogFacility AUTH
30         LogLevel INFO
31         LoginGraceTime 120
32         PermitRootLogin no
33         StrictModes yes
34         RSAAuthentication yes
35         PubkeyAuthentication yes
36         IgnoreRhosts yes
37         RhostsRSAAuthentication no
38         HostbasedAuthentication no
39         PermitEmptyPasswords no
40         ChallengeResponseAuthentication no
41         X11Forwarding no
42         X11DisplayOffset 10
43         PrintMotd no
44         PrintLastLog yes
45         TCPKeepAlive yes
46         AcceptEnv LANG LC_*
47         UsePAM yes
48         AllowUsers kuba         
49
50# install salt-master and salt-ssh 
51runcmd:
52  - systemctl restart ssh
53  - curl -fsSL -o /usr/share/keyrings/salt-archive-keyring.gpg https://repo.saltproject.io/py3/ubuntu/20.04/amd64/latest/salt-archive-keyring.gpg
54  - echo "deb [signed-by=/usr/share/keyrings/salt-archive-keyring.gpg arch=amd64] https://repo.saltproject.io/py3/ubuntu/20.04/amd64/latest focal main" | tee /etc/apt/sources.list.d/salt.list
55  - apt-get update -y
56  - apt-get install salt-ssh salt-master -y

Terraformation

Terraform is industry standard. We can use DigitalOcean API, GCP API, or even CloudFlare API. Some people prefer CloudFormation, some Pulumi or Azure ARM. Personally I like terraform. It is stable and works. The bucket mechanism is helpful in case of working with teammates. So why not use it?

Content

To be honest, the project is really small. Everything goes to one file called main.tf.

 1terraform {
 2
 3  required_version = ">= 1.0.6"
 4
 5  required_providers {
 6    digitalocean = {
 7      source = "digitalocean/digitalocean"
 8      version = "~> 2.0"
 9    }
10  }
11}
12
13variable "do_token" {
14  type = string
15}
16
17provider "digitalocean" {
18  token = var.do_token
19}
20
21resource "digitalocean_droplet" "droplet" {
22  image      = "ubuntu-20-04-x64"
23  name       = "saltmaster"
24  region     = "FRA1"
25  size       = "s-1vcpu-1gb"
26  monitoring = true
27  tags       = ["salt", "edu", "terrafom"]
28  user_data  = "${file("cloud-init.yaml")}"
29}
30
31output "public_ip" {
32  description = "Droplet public IP"
33  value       = digitalocean_droplet.droplet.ipv4_address
34}

Also, we need to put do_token somewhere. For example in a file called sample.tfvars or in CLI variable. Both solutions have some props and cons. The file is easy to use, but it's important to keep it in your .gitignore.

One command leter

At the end of the day we just need to run our terraform project with:

1terraform init && terraform plan -var-file=sample.tfvar

If everything looks fine we can build our infrastructure.

1terrafom apply -var-file=sample.tfvars -auto-approve

Login into the new machine

1ssh kuba@$(tf output -raw public_ip) -p 31

Now you can check the cloud-init logs. I can assume that some updates will be still running.

1sudo tail -f /var/logs/cloud-init.log

Summary

There is no repository for the article. The project structure is simple.

1.
2├── cloud-init.yaml
3├── main.tf
4└── sample.tfvars

The most interesting tool here is cloud-init which could be very helpful in many, many cases. We can use it in cloud environment provisioning, small internal projects, or even rebuilding the DC fleet on OpenStack. In my opinion, is important for every DevOps person to be at least familiar with this concept. Even if you will never use it in production. Terraform is a terrafrom, nothing else. DigitalOcean ocean is simple, affordable, and stable. What next? I will try to focus on SaltStack.