Heroku
An example integration on forwarding Data Connector events to a server hosted on Heroku.

Overview

This example uses a Data Connector to forward the events of all devices in a project to a server hosted on Heroku. When receiving the HTTPS POST request, our application will verify both the origin and content of the request using a Signature Secret.

Prerequisites

The following points are assumed.

Heroku

Ensure you have the following software installed on your local machine.
    Git, the popular version control tool.
    Heroku CLI, for interacting with Heroku.
    A functioning local environment for the language of your choice.

Create a New App Locally

On your machine, create and enter a new directory for your app.
1
mkdir my-app
2
cd my-app
Copied!
Python 3.9
Python API
Node.js 16
Go 1.16
In the directory, create a new file app.py where you paste the following snippet which contains the Flask server code for receiving and validating the Data Connector request.
1
import os
2
import hashlib
3
import jwt
4
from flask import Flask, request
5
6
app = Flask(__name__)
7
8
# Fetch environment variables.
9
SIGNATURE_SECRET = os.environ.get('DT_SIGNATURE_SECRET')
10
11
12
def verify_request(body, token):
13
# Decode the token using signature secret.
14
try:
15
payload = jwt.decode(token, SIGNATURE_SECRET, algorithms=["HS256"])
16
except Exception as err:
17
print(err)
18
return False
19
20
# Verify the request body checksum.
21
m = hashlib.sha1()
22
m.update(body)
23
checksum = m.digest().hex()
24
if payload["checksum"] != checksum:
25
print('Checksum Mismatch')
26
return False
27
28
return True
29
30
31
@app.route('/', methods=['POST'])
32
def endpoint():
33
# Extract necessary request information.
34
body = request.data
35
token = request.headers['x-dt-signature']
36
37
# Validate request origin and content integrity.
38
if not verify_request(body, token):
39
return ('Could not verify request.', 400)
40
41
# Print the request body.
42
print(body)
43
44
return ('OK', 200)
45
Copied!
In the same directory, create a new file requirements.txt with the following contents. Heroku will install these dependencies in its environment before starting the server.
requirements.txt
1
gunicorn==20.1.0
2
flask==2.0.1
3
pyjwt==2.1.0
Copied!
Finally, specify the command Heroku should run by adding a Procfile with the following snippet. This instructs Heroku that the server should run a single process of type web, where Gunicorn should be used to serve the app using a variable called app in a module called app (app.py)
Procfile
1
web: gunicorn app:app
Copied!
In the directory, create a new file app.py where you paste the following snippet which contains the Flask server code for receiving, validating, and decoding the Data Connector request.
app.py
1
import os
2
from flask import Flask, request
3
4
from dtintegrations import data_connector, provider
5
6
app = Flask(__name__)
7
8
9
@app.route('/', methods=['POST'])
10
def endpoint():
11
# Use the provider-specific validation function.
12
event = data_connector.http_push.decode_request(
13
request,
14
provider=provider.FLASK,
15
secret=os.getenv('DT_SIGNATURE_SECRET'),
16
)
17
18
# Print the event data.
19
print(event)
20
21
# If all is well, return 200 response.
22
return 'Success'
Copied!
In the same directory, create a new file requirements.txt with the following contents. Heroku will install these dependencies in its environment before starting the server.
requirements.txt
1
gunicorn==20.1.0
2
flask==2.0.1
3
dtintegrations
Copied!
Finally, specify the command Heroku should run by adding a Procfile with the following snippet. This instructs Heroku that the server should run a single process of type web, where Gunicorn should be used to serve the app using a variable called app in a module called app (app.py)
Procfile
1
web: gunicorn app:app
Copied!
In the directory, initialize a node application with the following command. This generates the file package.json which contains information used by Heroku when deploying the app.
1
npm init
Copied!
Create a new file index.js where you paste the following snippet which contains the Express server code for receiving and validating the Data Connector request.
1
const express = require("express")
2
const bodyParser = require("body-parser")
3
const crypto = require('crypto')
4
const jwt = require('jsonwebtoken')
5
6
// Create the express server.
7
const app = express()
8
app.use(bodyParser.urlencoded({ extended: false }))
9
app.use(bodyParser.json())
10
11
// Fetch environment variables.
12
const signatureSecret = process.env.DT_SIGNATURE_SECRET
13
14
function verifyRequest(body, token) {
15
// Decode the token using signature secret.
16
let decoded;
17
try {
18
decoded = jwt.verify(token, signatureSecret)
19
} catch(err) {
20
console.log(err)
21
return false
22
}
23
24
// Verify the request body checksum.
25
let shasum = crypto.createHash('sha1')
26
let checksum = shasum.update(JSON.stringify(body)).digest('hex')
27
if (checksum !== decoded.checksum) {
28
console.log('Checksum Mismatch')
29
return false
30
}
31
32
return true
33
}
34
35
app.post("/", (req, res) => {
36
// Extract necessary request information.
37
let body = req.body
38
let token = req.get('X-Dt-signature')
39
40
// Validate request origin and content integrity.
41
if (verifyRequest(body, token) === false) {
42
res.sendStatus(400)
43
return
44
}
45
46
// Log the request body.
47
console.log(body)
48
49
res.sendStatus(200)
50
})
51
52
app.listen(process.env.PORT)
Copied!
In the same directory, install the following packages. This updates package.json so that Heroku can replicate the environment when deploying.
1
npm install body-parser==1.19.0
2
npm install express==4.17.1
3
npm install jsonwebtoken==8.5.1
Copied!
Create a file Procfile with the following snippet. This instructs Heroku that the server should run a single process of type web, where node index.js should be used to serve the app.
1
web: node index.js
Copied!
In the directory, initialize a go application with the following command. This generates the file go.mod which specifies version and requirements. Give any name you like.
1
go mod init example.com/m/v2
Copied!
Create a new file main.go where you paste the following snippet which contains the server code for receiving and validating the Data Connector request.
1
package main
2
3
import (
4
"crypto/sha1"
5
"encoding/hex"
6
"fmt"
7
jwt "github.com/dgrijalva/jwt-go" // go get github.com/dgrijalva/[email protected]
8
"io/ioutil"
9
"log"
10
"net/http"
11
"os"
12
)
13
14
// Environment variables.
15
var signatureSecret = os.Getenv("DT_SIGNATURE_SECRET")
16
17
// verifyRequest validates the request origin and content integrity.
18
func verifyRequest(bodyBytes []byte, tokenString string) error {
19
// Decode the token using signature secret.
20
claims := jwt.MapClaims{}
21
_, err := jwt.ParseWithClaims(tokenString, claims, func(token *jwt.Token) (interface{}, error) {
22
return []byte(signatureSecret), nil
23
})
24
if err != nil {
25
return err
26
}
27
28
// Verify the request body checksum.
29
sha1Bytes := sha1.Sum(bodyBytes)
30
sha1String := hex.EncodeToString(sha1Bytes[:])
31
if sha1String != claims["checksum"] {
32
return fmt.Errorf("Checksum mismatch.")
33
}
34
35
return nil
36
}
37
38
func endpoint(w http.ResponseWriter, r *http.Request) {
39
if r.URL.Path != "/" {
40
http.Error(w, "404 not found.", http.StatusNotFound)
41
return
42
}
43
44
// Take an action depending on the request method.
45
switch r.Method {
46
case "POST":
47
if err := r.ParseForm(); err != nil {
48
fmt.Fprintf(w, "ParseForm() err: %v", err)
49
return
50
}
51
52
// Attempt to extract the body bytes of the request.
53
body, err := ioutil.ReadAll(r.Body)
54
if err != nil {
55
log.Printf("Error reading body: %v", err)
56
http.Error(w, "Can't read body", http.StatusBadRequest)
57
return
58
}
59
60
// Validate the origin and contents of the request.
61
if err = verifyRequest(body, r.Header.Get("X-Dt-Signature")); err != nil {
62
log.Printf("Could not validate request: %v", err)
63
http.Error(w, "Could not validate request", http.StatusBadRequest)
64
return
65
}
66
67
//
68
// Further processing here.
69
//
70
71
log.Println("Success")
72
73
default:
74
http.Error(w, "Only POST methods are allowed.", http.StatusMethodNotAllowed)
75
}
76
}
77
78
func main() {
79
http.HandleFunc("/", endpoint)
80
81
port := ":" + string(os.Getenv("PORT"))
82
log.Printf("Listening on port %s.", port)
83
if err := http.ListenAndServe(port, nil); err != nil {
84
log.Fatal(err)
85
}
86
}
Copied!
In the same directory, run the following command to install the required package.
1
go get github.com/dgrijalva/[email protected]
Copied!
Build your application into a new directory bin.
1
mkdir bin
2
GOOS=linux CGO_ENABLED=0 go build -o bin/main -v .
Copied!
Finally, specify the command Heroku should run by adding a Procfile with the following snippet. This instructs Heroku that the server should run a single process of type web, where the app should be server from the binary bin/main that we built in the previous step.
Procfile
1
web: bin/main
Copied!

Deploy the App

The following steps will deploy your application to a new app in Heroku. Take note of the app name given to the application in step 4.
    1.
    Initialize your app's directory as a Git repository: git init
    2.
    Add all changes: git add .
    3.
    Commit the changes: git commit -m "initial commit"
    4.
    Create the Heroku app, and add it as a Git remote: heroku create <APP_NAME>. You can pick a unique app name yourself or let Heroku create a random one by omitting <APP_NAME>. This will be used as a part of the URL for your server.
    5.
    Push your application to Heroku: git push heroku main

Post Deployment Configuration

Add a new signature secret configuration variable to your app using the following command. This can be read by the python instance and imported as an environment variable to use. Use a strong secret.
1
heroku config:set -a <APP_NAME> DT_SIGNATURE_SECRET=<YOUR_SECRET>
Copied!
Your app is now ready to receive requests, but we need to know the endpoint URL. This can be acquired by looking at the Web URL field in the results of the following command. Save it for later.
1
heroku apps:info -a <APP_NAME>
Copied!

Create a Data Connector

To continuously forward the data to our newly created Heroku server, a Data Connector with almost all default settings is sufficient. Refer to Creating a Data Connector if you are unfamiliar with the process. The following configurations should be set.
    Endpoint URL: The Web URL found in the previous step.
    Signature Secret: The value of DT_SIGNATURE_SECRET environment variable.
Depending on your integration, it can also be smart to disable the event types you are not interested in. For instance, the NetworkStatusEvent is sent every Periodic Heartbeat and will by default be forwarded by the Data Connector if not explicitly unticked.

Test the Integration

If the integration was correctly implemented, the Success counter for your Data Connector should increment for each new event forwarded. This happens each Periodic Heartbeat or by touching a sensor to force a new event.
If instead the Error counter increments, a response containing a non-2xx status code is returned.
    Verify that the Data Connector endpoint URL is correct.
    You can view the logs of your Heroku app with the following command.
    1
    heroku logs --app=<APP_NAME> --tail
    Copied!
Last modified 1mo ago