Querying and Manipulating Data

Reading and writing data from external systems from Connection Builder

Integrations enable the application read and write data from external systems using the HTTP request Data Operation in a Data Flow. The HTTP Request data operation can be used to call any web API such as a REST service or SOAP service, and can also be used with an authenticated integration. This section will walk through how to retrieve tweets using the Twitter API with an authenticated integration.

Go to Connection Builder and add a Data Flow.

Choose the HTTP Request data operation.

In the Service dropdown, select the integration that you added in Configuration Builder. This example will cover connecting to the Twitter integration created using OAuth.

Now this HTTP Request data operation will retrieve the access tokens using your integration and allow you to call the Twitter API. Here we are doing a search across tweets using the search endpoint looking for 'airkitcx'. After running a test against the endpoint by hitting the play icon, the results will populate in the Run Results area.

Once the Data Flow is created with the HTTP Request, the data flow is usable throughout the journey.

Reading and writing data with scheduled jobs

Periodic Task can be used to read and write data from an external system in a recurring fashion. For example, if you wanted to check for new tweets every hour, you would set up a Data Flow first and connect the Periodic Task to that Data Flow. To create a Periodic Task, go to Connection Builder and click '+' next to Periodic Tasks.

Then, choose your Connection which is referring to the Data Operation you created.

Define the schedule that you would like your Periodic Task to run. This field uses cron schedule expressions and the times are based in UTC. Once the Periodic Task is set up, you can save and publish your app to start your scheduled jobs!