Local Development with ngrok

Develop cloud services locally by continuously forwarding real data from a Data Connector using ngrok.

Overview

We will here go through a few tools that can be used to create a local development environment for Data Connector applications. The continuous stream of events emitted by a Data Connector is directed to a randomly generated HTTPS URL generated by ngrok, a free proxy service that forwards the events to a specified port on your localhost. There, a local server app captures the request, prints the content, and responds with a success message.

Prerequisites

  • Service Account Your User or Service Account must have a role of Project Developer or higher.

  • Local Development It is assumed that you are familiar with the bare minimums of local development, like using a shell.

ngrok

ngrok is a free proxy service that enables forwarding HTTPS traffic from a publicly accessible URL to a port on localhost.

Installation

Most official- or user-maintained package managers contain ngrok. If not, the binary can be downloaded directly from the official download page.

  • Debian-based distributions: apt install ngrok

  • Arch-based distributions: ngrok AUR

  • Mac: brew install ngrok

  • Windows: The ngrok client binary can be found here.

Starting the Proxy

In your shell of choice, run the ngrok command with the http argument, followed by the port to which you want to forward incoming traffic.

ngrok http 3000

Once the proxy has started, all traffic directed to the randomly generated HTTPS URL will be directed to the specified port on your localhost.

ngrok by @inconshreveable (Ctrl+C to quit)
Session Status online
Session Expires 1 hour, 59 minutes
Version 2.3.35
Region United States (us)
Web Interface http://127.0.0.1:4040
Forwarding http://5a880278718b.ngrok.io -> http://localhost:3000
Forwarding https://5a880278718b.ngrok.io -> http://localhost:3000
Connections ttl opn rt1 rt5 p50 p90
0 0 0.00 0.00 0.00 0.00

If you're using the free version of ngrok, your session will expire after two hours. To restart, simply run the command again, but note that the URL will have changed.

Localhost Path

Any modifications to the path of the generated URL will be reflected in the localhost. This can be useful to know as some local development frameworks, such as the one used by Azure Functions, utilizes pathing.

  • Original https://5a880278718b.ngrok.io -> http://localhost:3000

  • Added path https://5a880278718b.ngrok.io/some/path -> http://localhost:3000/some/path

Data Connector

To continuously forward events to localhost, either create a new Data Connector or use an existing one, setting the following configurations.

  • Endpoint URL The generated HTTPS URL of the form https://5a880278718b.ngrok.io as provided by ngrok. Note that the HTTP URL will not be accepted by the Data Connector.

Local Server App

When forwarding Data Connector events to localhost using ngrok, if no response is given to the request, an 502 Bad Gateway error will the thrown.

The following snippets implement a simple local server app that receives the Data Connector request, prints the content, and responds with a success message.

Python 3.9 - Flask
Node.js 14 - Express
Python 3.9 - Flask

Copy the following snippet to a local file app.py.

app.py
from flask import Flask, request # pip install flask
app = Flask(__name__)
@app.route('/', methods=['POST'])
def print_request_contents():
print(f'\nHeaders\n-------\n{request.headers}')
print(f'Body\n----\n{request.get_json()}')
return 'Success'

Start the local server by running flask run --port 3000 in your shell.

Node.js 14 - Express

Copy the following snippet to a local file index.js.

index.js
const express = require("express") // npm install [email protected]
const bodyParser = require("body-parser") // npm install [email protected]
const app = express()
app.use(bodyParser.urlencoded({ extended: false }))
app.use(bodyParser.json())
app.post("/", (req, res) => {
console.log(req.body)
res.send("Success")
})
app.listen(3000)

Start the local server by running node index.js in your shell.

Sensor Emulator

Our sensors emit events every Periodic Heartbeat, or when touched. When developing an application locally, being able to emit events at will can be quite useful. Our Sensor Emulator lets you create an emulated device for which events can be sent using a click of a button in DT Studio or the REST API.

You can read about how to create and emit emulated events on our Sensor Emulator page.