App Events to AWS S3 Buckets

Data collected around App Events can be streamed from Airkit to AWS S3 buckets, allowing you to investigate the data in external analytics platforms.

Streaming App Events to S3

Here, we walk through how to set up your S3 bucket to receive App Event Logs logs from Airkit.

  1. Create your S3 Bucket in AWS. When creating the bucket, select ACLs disabled.

  1. After creating your S3 Bucket, provide Airkit permission to your S3 bucket via the following AWS IAM policy.
    {{BUCKET-NAME}}: name of the S3 bucket you created
    {{aws_region}}: name of the region your organization is provisioned
    • US - us-west-2
    • AP - ap-southeast-2
    • EU - eu-central-1
      {{organization_id}}: The ID of your organization. To get your org id, see Getting Application Metadata
    {
       "Version": "2012-10-17",
       "Statement": [
          {
             "Sid": "AirkitWritePermission",
             "Effect": "Allow",
             "Principal": {
                "AWS": "arn:aws:sts::113997530994:assumed-role/airkit-{{aws_region}}-reporting-export-role/{{organization_id}}"
             },
             "Action": [
                "s3:PutObject"
             ],
             "Resource": [
                "arn:aws:s3:::{{BUCKET-NAME}}/*"
             ]
          }
       ]
    }
    
{
   "Version": "2012-10-17",
   "Statement": [
      {
         "Sid": "AirkitWritePermission",
         "Effect": "Allow",
         "Principal": {
            "AWS": "arn:aws:iam::113997530994:root"
         },
         "Action": [
            "s3:PutObject"
         ],
         "Resource": [
            "arn:aws:s3:::{{BUCKET-NAME}}/*"
         ]
      }
   ]
}
  1. In Airkit Console,visit Settings > Logs and App Notifiers. The UI will look as follows:
1616

Under App Event Logs > S3 bucket, click edit to set an new S3 bucket, and then insert the S3 bucket name you created previously into the pop-up window that appears:

649

Click Verify. Airkit will write a test file named airkit-verify-test-{{timestamp}}

  1. Once configured, every few minutes, Airkit will send relevant Events in a new file to the S3 bucket.

Schema and Example

The structure of the exported data depends on the type Event being sent. Broadly speaking, there are two kinds of Events that can be sent as App Events: Session Events and Platform Events. However, there is a lot of variation to the data structure even within these categories, as the relevant information depends on the Event details.

There are a couple standard fields that are part of every Event:

  • organizationId
  • eventTime
  • appId
  • branchId
  • sessionId
  • deployId
  • type
  • kind - for App Events, this will always be "Session"

It is the type field that usually determines what other fields that will be part of the export. For instance, consider the following export where type = session_start:

{
    "organizationId": "482f2a38-8e9d-4329-b2a0-e97d5bc05d36",
    "eventTime": "2022-11-19T23:23:16.792323Z",
    "appId": "e0cef201-2697-4e5c-9aa4-9befb2a2e2d7",
    "branchId": "743d626d-e1f1-46d5-858e-c65787b57cb3",
    "sessionId": "69175e43-25d0-47c7-bdec-224e49753e99",
    "deployId": "d0e64edb-7a23-497b-b38f-21efeb3d04a3",
    "type": "session_start",
    "actorId": "dd445fb0-fbc5-4087-bf9c-a3d5641252d9",
    "sourceRuntime": "MANAGER",
    "sourceDetail": "INIT",
    "appEventSchema": "/element/event2/session/on-start.json",
    "appEventId": "04368795-905c-4875-a4c8-5b266ec5f438",
    "ipAddresses": [
        "52.9.13.199"
    ],
    "billingCode": "Journey",
    "kind": "Session",
    "defaultMetrics": {
        "/element/event2/session/on-start.json": {
            "04368795-905c-4875-a4c8-5b266ec5f438": {
                "count": 1
            }
        }
    }
}

In addition to the standard fields, this export also contains the fields actorId, sourceRuntime, sourceDetail, appEventSchema, appEventId, ipAddresses, billingCode, and defaultMetrics, which all contain pertinent information regarding a newly-started Journey.