Data Flows are used to send and receive data from external systems, comport and reformat data, and send data to AirData. This document will walk through the basics of Data Flows and outline the different types of Data Operation Steps in Data Flows.
This article will cover:
The layout of a Data Flow
Data Flows are created in Connection Builder. Data Flows are one type of connection. Data Flows are initiated by some other part of your application. With a Data Flow selected, the layout of Connection Builder looks like:
From Left to right, the screengrab shows the Tree of Data Flows, the Data Operation Steps of the selected Data Flow, and the Inspector with available parameters at the different steps throughout the Data Flow.
When a data flow is called, the operation steps are executed sequentially starting at the top and working down the stage. The outputs from each Data Operation Step will be available in the following steps below.
Start and End of Data Flows
There are two special boxes in the stage of the Data Flow. The Start block represents variables that are to be sent into the Data Flow. In the Start box, provide demo data that will be used throughout the testing of the Data Flow in Connection Builder. This is NOT the same as default values. This data will have no bearing on the actual running of the data operation. Because the start parameters are user entered, be careful to make sure the properties of the start parameters match the format of the values the Data Flow will receive. You will receive a warning if the data is not in the format expected. Start parameters are available in any of the data operations throughout the flow.
The End block represents the value to be returned in the output block of the Data Flow. In Activity Builder, this is the output you specify in the Run Data Flow action.
Data Operation Frame
Other than the start and end Frame of the Data Flow, each block in the stage represents an individual Data Operation. Data Operations have a specific type. See the Types of Data Operations below. A sample Data Operation looks like:
On the left of the frame is the frame number. Frames are numbered sequentially and used to match up the variables on the right in the inspector. This Data Operation is of type Transform, and that is denoted at the top of the frame. The Top part of the frame is about the action to be performed. In this case, there is a Transform Expression that contains Airscript that is run when the operation is called. The Output Format specifies the variable options for return. Many data Ops will have different configuration options.
The bottom of the Data Frame deals with the Run Results of the Data Operation. Most data Operations return a single item, with a default name. In the case above, the transform returns a default transform variable. It is possible to rename the output by double-clicking the value in the Inspector under the frame number. Some operations, like the HTTP Request, will have multiple different outputs. Each output will have a tab in the Run Results section.
Types of Data Operation Steps
|HTTP Request||Send HTTP requests and get responses.|
|Use Airscript to reformat, modify, or search through Data.|
|Secure Value Retrieval||Getting data from a secure string input.|
|Swagger Schema Validation||Use a Swagger Schema to check the validity of a data packet.|
|AirData||Send or receive data from AirData|
|Call another Data Flow.|
|Zapier Service||Make a call to the existing zap on Zapier.|
|Calendar Search||Search a calendar for the next available window.|
|Journey Mapping||Used to create new journeys.|
|Execute a journey event for a specified journey.|
|Send an email.|
|For creating a byte length specified line of text.|
|Sending files via the SFTP protocol|
|Convert HTML into a formatted PDF.|
|Compress a file or files for sending.|
|Create a file asset.|
|Merge multiple PDFs together.|
|Pause the operation for a specified time.|
|Given a PDF and data, merge the data into a PDF.|
|Encrypt a document with PGP encryption.|
|Convert an XML document into a JSON document.|
|Get information for a specified asset, including a download URL.|
|Remove an asset.|
|Make a request to external Zendesk Integration.|
|Make a request to external Salesforce Integration.|