AWS Lambda
An example integration where a Data Connector forwards events to an AWS Lambda Function.

Overview

This example uses a Data Connector to forward the events of all devices in a project to an AWS Lambda Function. When receiving the HTTPS POST request, our function will verify both the origin and content of the request using a Signature Secret, then decode the data.

Prerequisites

The following points are assumed.

Amazon Web Services

The following preliminaries should be met.

Create a Deployment Package

AWS Lambda Functions can be created and deployed using either a Docker Image or a .zip archive. Both will achieve the same result, but we will here use a .zip archive in the interest of simplicity.

Root Directory

Create and enter a new directory root in which your function will be developed.
1
mkdir new-lambda-function
2
cd new-lambda-function
Copied!

Lambda Function Source

Python 3.8
Node.js 14
Go 1.x
In your directory root, create a file main.py in which you paste the following snippet. The implementation is explained in detail on the Data Connector Advanced Configurations page.
1
import os
2
import hashlib
3
import jwt # pip install pyjwt==2.0.1
4
5
# Fetch environment variables.
6
SIGNATURE_SECRET = os.environ.get('DT_SIGNATURE_SECRET')
7
8
9
def verify_request(body, token):
10
# Decode the token using signature secret.
11
try:
12
payload = jwt.decode(token, SIGNATURE_SECRET, algorithms=["HS256"])
13
except Exception as err:
14
print(err)
15
return False
16
17
# Verify the request body checksum.
18
m = hashlib.sha1()
19
m.update(body.encode('ascii'))
20
checksum = m.digest().hex()
21
if payload["checksum"] != checksum:
22
print('Checksum Mismatch')
23
return False
24
25
return True
26
27
28
def lambda_handler(event, context):
29
# Extract necessary request information.
30
body = event['body']
31
token = event['headers']['X-Dt-Signature']
32
33
# Validate request origin and content integrity.
34
if not verify_request(body, token):
35
return {"statusCode": 400}
36
37
#
38
# Further processing here.
39
#
40
41
return {"statusCode": 200}
Copied!
In your directory root, create a file index.js in which you paste the following snippet. The implementation is explained in detail on the Data Connector Advanced Configurations page.
1
const crypto = require('crypto')
2
const jwt = require('jsonwebtoken') // npm install [email protected]
3
4
// Fetch environment variables.
5
const signatureSecret = process.env.DT_SIGNATURE_SECRET
6
7
function verifyRequest(body, token) {
8
// Decode the token using signature secret.
9
let decoded;
10
try {
11
decoded = jwt.verify(token, signatureSecret)
12
} catch(err) {
13
console.log(err)
14
return false
15
}
16
17
// Verify the request body checksum.
18
let shasum = crypto.createHash('sha1')
19
let checksum = shasum.update(JSON.stringify(body)).digest('hex')
20
if (checksum !== decoded.checksum) {
21
console.log('Checksum Mismatch')
22
return false
23
}
24
25
return true
26
}
27
28
exports.handler = async function(event, context) {
29
// Extract necessary request information.
30
let body = JSON.parse(event.body)
31
let token = event.multiValueHeaders['X-Dt-Signature'][0]
32
33
// Validate request origin and content integrity.
34
if (verifyRequest(body, token) === false) {
35
return {'statusCode': 400}
36
}
37
38
//
39
// Further processing here.
40
//
41
42
return {'statusCode': 200}
43
}
Copied!
In your directory root, initialize a new go module.
1
go mod init example.com/m/v2
Copied!
Create a file main.go in which you paste the following snippet.
1
package main
2
3
import (
4
"context"
5
"crypto/sha1"
6
"encoding/hex"
7
jwt "github.com/dgrijalva/jwt-go" // go get github.com/dgrijalva/[email protected]
8
"log"
9
"os"
10
11
"github.com/aws/aws-lambda-go/events"
12
"github.com/aws/aws-lambda-go/lambda"
13
)
14
15
// Environment variables.
16
var signatureSecret = os.Getenv("DT_SIGNATURE_SECRET")
17
18
// verifyRequest validates the request origin and content integrity.
19
func verifyRequest(bodyBytes []byte, tokenString string) bool {
20
// Decode the token using signature secret.
21
claims := jwt.MapClaims{}
22
_, err := jwt.ParseWithClaims(tokenString, claims, func(token *jwt.Token) (interface{}, error) {
23
return []byte(signatureSecret), nil
24
})
25
if err != nil {
26
log.Println(err)
27
return false
28
}
29
30
// Verify the request body checksum.
31
sha1Bytes := sha1.Sum(bodyBytes)
32
sha1String := hex.EncodeToString(sha1Bytes[:])
33
if sha1String != claims["checksum"] {
34
log.Println("Checksum mismatch.")
35
return false
36
}
37
38
return true
39
}
40
41
func handleRequest(ctx context.Context, req events.APIGatewayProxyRequest) (events.APIGatewayProxyResponse, error) {
42
// Extract necessary request information.
43
tokenString := req.Headers["X-Dt-Signature"]
44
bodyBytes := []byte(req.Body)
45
46
// Validate request origin and content integrity.
47
if !verifyRequest(bodyBytes, tokenString) {
48
return events.APIGatewayProxyResponse{StatusCode: 400}, nil
49
}
50
51
return events.APIGatewayProxyResponse{StatusCode: 200}, nil
52
}
53
54
func main() {
55
lambda.Start(handleRequest)
56
}
Copied!

Dependencies & Pre-Deployment

In your root directory, install the following dependencies.
Python 3.8
Node.js 14
Go 1.x
1
pip install --target ./package pyjwt==2.0.1
Copied!
Your directory tree should now look something like this.
1
.
2
├── lambda_function.py
3
└── package
4
├── jwt
5
├── PyJWT-1.7.1.dist-info
6
├── ...
7
└── ...
Copied!
1
npm init
2
npm install [email protected]
3
npm install [email protected]
Copied!
Your directory tree should now look something like this.
1
.
2
├── index.js
3
├── node_modules
4
│   ├── aws-xray-sdk
5
│   ├── jsonwebtoken
6
│   ├── ...
7
│   ├── ...
8
├── package.json
9
└── package-lock.json
Copied!
1
go get github.com/dgrijalva/[email protected]
2
go get github.com/aws/aws-lambda-go/lambda
Copied!
Your directory tree should now look something like this.
1
.
2
├── go.mod
3
├── go.sum
4
└── main.go
Copied!

Deploy the Function

To deploy the Lambda Function, the deployment package must be packaged.

Create .zip archive

The following steps create the deployment package .zip archive.
Python 3.8
Node.js 14
Go 1.x
    1.
    From your directory root, enter the package directory. cd package
    2.
    In the package directory, create the deployment package archive. zip -r ../deployment-package.zip .
    3.
    Navigate back to the directory root. cd ..
    4.
    Add the Lambda Function source to the deployment package archive. zip -g deployment-package.zip lambda_function.py
From your directory root, create the deployment package with the following command.
1
zip -r deployment-package.zip .
Copied!
The following steps are for Linux and MacOS. If you're on Windows or want more details about the process, refer to the official AWS Guide.
Compile your executable.
1
GOOS=linux CGO_ENABLED=0 go build main.go
Copied!
From your directory root, create the deployment package with the following command.
1
zip deployment-package.zip main
Copied!

Create a New Lambda Function

If this is the first time deploying your function, the following command will create a new one and publish it. Arguments in the form <ARGUMENT> should be replaced with your own details.
Python 3.8
Node.js 14
Go 1.x
1
aws lambda create-function \
2
--function-name <YOUR_FUNCTION_NAME> \
3
--zip-file fileb://deployment-package.zip \
4
--handler lambda_function.lambda_handler \
5
--runtime python3.8 \
6
--role arn:aws:iam::<YOUR_AWS_ACCOUNT_ID>:role/<YOUR_ROLE_NAME>
Copied!
1
aws lambda create-function \
2
--function-name <YOUR_FUNCTION_NAME> \
3
--zip-file fileb://deployment-package.zip \
4
--handler index.handler \
5
--runtime nodejs14.x \
6
--role arn:aws:iam::<YOUR_AWS_ACCOUNT_ID>:role/<YOUR_ROLE_NAME>
Copied!
1
aws lambda create-function \
2
--function-name <YOUR_FUNCTION_NAME> \
3
--zip-file fileb://deployment-package.zip \
4
--handler main \
5
--runtime go1.x \
6
--role arn:aws:iam::<YOUR_AWS_ACCOUNT_ID>:role/<YOUR_ROLE_NAME>
Copied!

Update an Existing Lambda Function

If you make changes to the code post-deployment, the following command can be used to deploy your changes to the existing function.
1
aws lambda update-function-code \
2
--function-name <YOUR_FUNCTION_NAME> \
3
--zip-file fileb://deployment-package.zip
Copied!

Post Deployment

To receive events from our Data Connector, an endpoint to which the requests can be sent must be created in your new function. Following this guide, create a new trigger with the following configurations.
    Trigger: API Gateway
    API: Create an API
    API type: HTTP API
    Security: Open
    Additional Settings: as desired
Once created, under your new trigger's details, copy the API endpoint URL and save it for later as we will use it when configuring our Data Connector.

Environment Variables

Finally, follow this guide on how to add an environment variable for the signature secret. This will be used to verify the origin and content of received requests.
    Name: DT_SIGNATURE_SECRET
    Value: A unique password. We will use it later, so write it down.

Data Connector

To continuously forward the data to our newly created Lambda Function, a Data Connector with almost all default settings is sufficient. If you are unfamiliar with how Data Connectors can be created, refer to our Creating a Data Connector guide. The following configurations should be set.
    Endpoint URL: The API endpoint URL found in the previous step.
    Signature Secret: The value of DT_SIGNATURE_SECRET parameter set earlier.
Depending on your integration, it can also be smart to disable the event types you are not interested in. For instance, the NetworkStatusEvent is sent every Periodic Heartbeat and will by default be forwarded by the Data Connector if not explicitly unticked.

Test the Integration

If the integration was correctly implemented, the Success counter for your Data Connector should increment for each new event forwarded. This happens each Periodic Heartbeat or by touching a sensor to force a new event.
If instead the Error counter increments, a response containing a non-200 status code is returned.
    Verify that the Data Connector endpoint URL is correct.
    AWS provides a host of tools that can be used to monitor Lambda Functions. Check the logs for any tracebacks that could explain why an error is returned.

Next steps

Your sensor data is now in the AWS environment, and you can start using it in their various services. Fortunately, AWS has some well-documented guides to get you started.

PostgreSQL Database

A database should be tailored to each specific use case. However, if you're uncertain, PostgreSQL (Postgres) is a good place to get started. The following guides will show you how to create a new Postgres database, then connect your Lambda Function to execute queries.
Last modified 3mo ago