How to use Lambda as a glue?
Let's keep this article short. The topic is rather popular, but not very well documented. It's based on quite a popular issue, which is about untagged EC2. Sometimes developers launch virtual machines, without tags. Why it's so important? Because without tags, you have no idea about the machine's purpose. You have no owner, now, project, no cost center, etc. I know that some of these things can be checked with CloudTrail, however, it's still about keeping the environment clean. After kindly asking, reminders, etc, I decided to go with a slightly different kind of solution. The brutal one.
Intro
What is the brutal solution then? It's simple if a new machine will be spawned without mandatory tags, an instance will be immediately terminated. Probably you have seen a lot of GUI-based tutorials for it. Most of them are great, if you want to implement a similar solution then, please go there. I would like to focus on governance, an as code solution. Also, I want to meet SAM.
What the SAM is? In simple words, it's an AWS alternative to the well-known Serverless Framework. It provides app skeletons, dummy EventBridge events, unit and integration tests, and basic code in many popular languages. Also, it allows users to define all needed resources, with CloudFormation templates, and an easy deployment tool.
However please remember that this is not a magic solution and in many cases, a few adjustments will be needed. And that's what I will include in this article. SAM intro with a little tweaking.
Tools used in this episode
- SAM CLI
- Python
- CloudFormation(scripts)
Implementation
First, we need to install SAM. Fortunately it's well documented via Amazon, and can be shortened to:
1# linux
2wget https://github.com/aws/aws-sam-cli/releases/latest/download/aws-sam-cli-linux-x86_64.zip -O aws-sam-cli-linux-x86_64.zip
3unzip aws-sam-cli-linux-x86_64.zip -d sam-installation
4sudo ./sam-installation/install
5
6# mac
7brew tap aws/tap
8brew install aws-sam-cli
Init the project
Great, now we're able to spin the project with one simple command. Nevertheless, I strongly recommend running through regular creators and exploring multiple options. For my instance stopper project, the bootstrap command looks like that:
1sam init --name instance_stopper \
2 --runtime python3.9 \
3 --dependency-manager pip \
4 --app-template eventBridge-hello-world \
5 --architecture arm64
I think that the most important flag here is runtime and app-template. Python runtime is good, for my case, as It's easy to understand and fast to implement. Maybe someday I will rewrite it into Go, however today I want simplicity. App-template from another hand download pre-define application template. After all, the script will produce a falling directory tree.
1tree instance_stopper/
2instance_stopper/
3├── README.md
4├── __init__.py
5├── conftest.py
6├── events
7│ └── event.json
8├── hello_world_function
9│ ├── __init__.py
10│ ├── hello_world
11│ │ ├── __init__.py
12│ │ └── app.py
13│ ├── model
14│ │ ├── __init__.py
15│ │ └── aws
16│ │ ├── __init__.py
17│ │ └── ec2
18│ │ ├── __init__.py
19│ │ ├── aws_event.py
20│ │ ├── ec2_instance_state_change_notification.py
21│ │ └── marshaller.py
22│ └── requirements.txt
23├── template.yaml
24└── tests
25 ├── __init__.py
26 ├── integration
27 │ ├── __init__.py
28 │ └── test_ec2_event.py
29 ├── requirements.txt
30 └── unit
31 ├── __init__.py
32 └── test_handler.py
33
3410 directories, 21 files
The trip
Ok, let's run some fast directory structure overview.
events- contains an example EventBridge event, useful for testinghello_world_function\hello_world- main app directoryhello_world_function\model- event model, for data deserializationtemplate.yaml- CloudFormation templatetests\integration- integration test which requires a real environmenttests\unit- unit tests, PyTest for local validation
Python code
The first thing you need to take a look at is general logic. What do we want to archive? In my case, the flow is simple. If someone spins the instance, without the tag Owner, an instance will be immediately terminated. How to do it? With the usage of the boto3 library again!
At the beginning please remember about python venv and fetching requirements for testing.
1python3 -m venv deploy
2source deploy/bin/activate
3pip install -r tests/requirements.txt
Then let's write some code:
1# cat instance_stopper_function/instance_stopper/app.py
2import boto3
3
4from model.aws.ec2 import Marshaller
5from model.aws.ec2 import AWSEvent
6from model.aws.ec2 import EC2InstanceStateChangeNotification
7
8
9def lambda_handler(event, context):
10
11 # Deserialize event into strongly typed object - yea its possible
12 awsEvent, ec2StateChangeNotification = deserialize_event(event)
13
14 # use boto3 as client
15 ec2 = boto3.resource('ec2')
16 instance = ec2.Instance(ec2StateChangeNotification.instance_id)
17 tags: dict = instance.tags
18
19 # Execute business logic
20 if search_owner(tags) is None:
21 print("Kill it: " + ec2StateChangeNotification.instance_id)
22 # Termiante instance with Owner tag
23 instance.terminate()
24
25 # Make updates to event payload
26 awsEvent.detail_type = "Lambda function terminated the machine " + ec2StateChangeNotification.instance_id + " " + awsEvent.detail_type
27
28 # Return event for further processing
29 return Marshaller.marshall(awsEvent)
30
31
32def search_owner(tags):
33
34 if (next((x for x in tags if x["Key"] == "Owner"), None)) is None:
35 return None
36 else:
37 return "Owner exist"
38
39
40def deserialize_event(event):
41
42 awsEvent: AWSEvent = Marshaller.unmarshall(event, AWSEvent)
43 ec2StateChangeNotification: EC2InstanceStateChangeNotification = awsEvent.detail
44
45 print("Region is " + awsEvent.region)
46 print("Instance " + ec2StateChangeNotification.instance_id + " transitioned to " + ec2StateChangeNotification.state)
47
48 return awsEvent, ec2StateChangeNotification
Probably you can see, that code it's slightly different than the original. Also, I modified unit tests, because of that.
1# cat tests/unit/test_handler.py
2import pytest
3
4from instance_stopper import app
5from model.aws.ec2.ec2_instance_state_change_notification import EC2InstanceStateChangeNotification
6
7
8@pytest.fixture()
9def eventBridgeec2InstanceEvent():
10 """ Generates EventBridge EC2 Instance Notification Event"""
11
12 return {
13 "version": "0",
14 "id": "7bf73129-1428-4cd3-a780-95db273d1602",
15 "detail-type": "EC2 Instance State-change Notification",
16 "source": "aws.ec2",
17 "account": "123456789012",
18 "time": "2015-11-11T21:29:54Z",
19 "region": "us-east-1",
20 "resources": [
21 "arn:aws:ec2:us-east-1:123456789012:instance/i-abcd1111"
22 ],
23 "detail": {
24 "instance-id": "i-abcd1111",
25 "state": "pending"
26 }
27 }
28
29
30def test_lambda_handler(eventBridgeec2InstanceEvent):
31
32 awsEvent, details = app.deserialize_event(eventBridgeec2InstanceEvent)
33
34 assert awsEvent.region == "us-east-1"
35 assert details.instance_id == "i-abcd1111"
36 assert details.state == "pending"
37
38
39def test_tag_filter():
40
41 tags: list = [
42 {'Key': 'Owner', 'Value': 'Kuba'},
43 {'Key': 'Project', 'Value': 'Infra'},
44 {'Key': 'Managedby', 'Value': 'SAM'},
45 ]
46 owner = app.search_owner(tags)
47 assert owner.startswith("Owner")
48
49
50def test_wrong_tags_filter():
51
52 tags: list = [
53 {'Key': 'Project', 'Value': 'Infra'},
54 {'Key': 'Managedby', 'Value': 'SAM'},
55 ]
56
57 owner = app.search_owner(tags)
58 assert owner is None
After writing all this code, let's just test it.
1pytest .
2============================================================================= test session starts =============================================================================
3platform darwin -- Python 3.10.9, pytest-7.2.1, pluggy-1.0.0
4rootdir: /Users/kuba/Code/sam/instance_stopper
5plugins: mock-3.10.0
6collected 3 items
7
8tests/unit/test_handler.py ... [100%]
9
10============================================================================== 3 passed in 0.08s ==============================================================================
In the end, I decided to skip integration tests. The task is too simple, to spin the whole stack, for testing.
AWS environment
Now, let's take a look at template.yaml. Things which was changed:
- Tagging - as it will be funny to skip it!
- State of EventBridge, as I wanted to track the transition from
pendingtorunningstate.
1 Events:
2 ChangeStateEvent:
3 Type: CloudWatchEvent
4 Properties:
5 Pattern:
6 source:
7 - aws.ec2
8 detail-type:
9 - EC2 Instance State-change Notification
10 detail:
11 state:
12 - running
- Then IAM Role. Initially, SAM provides a
basic lambda executionrole. Our case needs more complex scope.
1 InstanceStopperRole:
2 Type: 'AWS::IAM::Role'
3 Properties:
4 Tags:
5 - Key: Owner
6 Value: kuba
7 - Key: Usecase
8 Value: Infra
9 - Key: ManagedBy
10 Value: SAM
11 AssumeRolePolicyDocument:
12 Version: "2012-10-17"
13 Statement:
14 - Effect: Allow
15 Principal:
16 Service:
17 - lambda.amazonaws.com
18 Action:
19 - 'sts:AssumeRole'
20 Path: /
21 Policies:
22 - PolicyName: StopTheInstance
23 PolicyDocument:
24 Version: "2012-10-17"
25 Statement:
26 - Effect: Allow
27 Action:
28 - "logs:CreateLogGroup"
29 - "logs:CreateLogStream"
30 - "logs:PutLogEvents"
31 Resource: "arn:aws:logs:*:*:*"
32 - Effect: Allow
33 Action:
34 - "ec2:DescribeInstances"
35 - "ec2:TerminateInstances"
36 Resource: "*"
And that's it! All changes. Now we can go perform the deployment.
Deployment
That part is awesome. We can deploy all these objects with managed SAM stack, with just one command. At first time, --guided mode could be helpful.
1sam deploy --guided
After that, we will receive samconfig.toml file with the whole deployment configuration.
In general, we're done. Lambda Function with all needed resources was deployed, and the whole config is stored as a code, so it's waiting for basic CI/CD, which will be provided (maybe) in the future.
After all, we can just list our lambdas:
1aws lambda list-functions --query='Functions[]'
Or logs:
1sam logs --stack-name instance_stopper
Summary
In a few above steps and less than 20 minutes, we built and deployed Lambda Function with prerequisites, waiting for EventBridge events. Maybe it wasn’t fast and simple, but the whole stack was delivered. Easy to move, reuse, or modify. Also, it's a CloudFormation stack, not GUI based function.
What next? As I said, CI/CD is missing, so we can investigate the best tool for small projects. Also, we can add some notifications, which will be sent to the developer, as well as to the admin. All communication can also be put in some database as a record of illegal usage.
Ah, and the project, which I built can be found here.