A quick guide on how to create a Data Connector using DT Studio or our APIs.
To illustrate the process, we will here create a new Data Connector that forwards all events to dweet.io, a free hosted service that fulfills the same role as a server. If you already have a hosted HTTPS POST endpoint set up, feel free to use that instead.
Creating, deleting, and interacting with Data Connectors require that your User or Service Account has been granted the role of Project Developer or higher.
Data Connectors sends events as POST requests over HTTPS. In order to receive these events you need to have a service up and running that can accept incoming HTTPS connections, HTTP is not supported. In this guide, we will use dweet.io as an example.
Please note that Disruptive Technologies and dweet.io are in no way affiliated and that the data sent there is publicly available.
Creating a New Data Connector
Data Connectors can be created using either the DT Studio interface or our REST API. For simplicity, most of the configurations will be left default in this example. For those interested in a more custom Data Connector, the Advanced Configurations page contains the explanations for the rest.
In DT Studio, navigate to your Project. In the left menu, locate Data Connectors and press Add Data Connector. This will open the Data Connector configuration menu.
Edit the following parameters.
Give the Data Connector some identifiable display name.
The URL to which the Data Connector will forward events. As we use dweet.io in this example, simply use an URL on the form https://dweet.io/dweet/for/<SOME_UNIQUE_NAME>.
Remember to save your Data Connector configuration at the bottom of the page.
Using cURL with a Service Account for authentication, the following example creates a new Data Connector with a given name "new-dcon" and a dweet.io endpoint which can be monitored at https://dweet.io/follow/<SOME_UNIQUE_NAME>.
curl -X POST "https://api.d21s.com/v2/projects/<PROJECT_ID>/dataconnectors"\
Using our Python API with Service Account credentials for authentication, the following example creates a new Data Connector with a given name "new-dcon" and a dweet.io endpoint which can be monitored at https://dweet.io/follow/<SOME_UNIQUE_NAME>.
import disruptive as dt
# Authenticate the package using Service Account credentials.
dt.default_auth = dt.Auth.service_account(
# Create a new HTTP Push Data Connector called "new-dcon".
For dweet.io, navigate to your generated URL, now on the form https://dweet.io/follow/<SOME_UNIQUE_NAME>. Under the Raw tab, the event body should become visible as they are forwarded. Any errors will also be shown here.
When you are certain an event has been emitted, the Data Connector metrics will be iterated.
The metrics can be found under the same page on which your Data Connector was created.