Overview

With the Kinesis Firehose Log Destination, you can send the full stream of Reporting events from Sym to any destination supported by Kinesis Firehose.

This is a powerful integration that can sit upstream of any number of logging destinations, including:

• AWS S3
• DataDog
• New Relic
• Redshift
• Splunk

📘

Prerequisites

If you have not yet, follow the Connecting Sym to AWS tutorial to set up your runtime.tf before continuing.

Give the Runtime Connector Role Permissions to Publish to Kinesis Firehose

Create a firehose.tf file with the following contents:

# An AWS IAM Policy that grants the permission to publish to Kinesis Firehose Delivery Streams tagged with SymEnv
# and the perimssion to list Delivery Streams.
resource "aws_iam_policy" "aws_kinesis_firehose" {
  name = "SymKinesisFirehose${title(local.environment_name)}"
  path = "/sym/"

  description = "Addon policy granting access to Kinesis Firehose"
  policy      = <<EOT
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "firehose:PutRecord",
        "firehose:PutRecordBatch"
      ],
      "Resource": "*",
      "Condition": { "StringEquals": { "firehose:ResourceTag/SymEnv": "${local.environment_name}" } }
    },
    {
      "Effect": "Allow",
      "Action": [
        "firehose:ListDeliveryStreams"
      ],
      "Resource": "*"
    }
  ]
}
EOT
}

# Attach the IAM policy declared above to the Runtime Connector Role defined in runtime.tf
resource "aws_iam_role_policy_attachment" "aws_kinesis_firehose_attach" {
  policy_arn = aws_iam_policy.aws_kinesis_firehose.arn
  role       = aws_iam_role.sym_runtime_connector_role.name
}

The aws_iam_policy.aws_kinesis_firehose resource defines an AWS IAM Policy that grants Put Record permissions to Kinesis Firehose destinations that are tagged with SymEnv = environment_name and grants the permission to list delivery streams.

Downstream logging destinations

Once you've set up a basic Kinesis Firehose, you will be able to pipe your Sym logs downstream to a number of destinations. The simplest approach is to send logs to an S3 bucket, but the combination of Kinesis + Terraform's mutual support for sending logs to HTTP endpoints makes most downstream integrations a breeze.

Downstream configurations for AWS Kinesis Firehose will usually take one of three types: Kinesis firehose built-in support, support via HTTP endpoint, or abstracted support via Sym module.

Further reading

For more information on sending logs downstream from AWS Kinesis Firehose, see the HashiCorp AWS Provider docs here.