Datadog
Overview
Connecting Sym reporting logs to Datadog is a similar process to New Relic, abstracted with a datadog-connector
module that provisions a Kinesis Firehose configured to pipe logs to Datadog.
For more information about streaming logs from Kinesis Firehose to Datadog, checkout the Datadog Docs.
Prerequisites
- An
environment.tf
file generated bysymflow init
a. If you have not runsymflow init
, please follow the instructions in Installing Sym- A Runtime Connector Role defined in
runtime.tf
a. If you do not have aruntime.tf
, please follow the instructions in Connecting Sym to AWS- A
firehose.tf
file created during the tutorial on the main AWS Kinesis Firehose
Create a DataDog API key
On the Datadog Configuration page, create an API Key. Save the key, so that you can use it to configure the Datadog Connector in the next section.
Configure the Datadog Connector
Declare a datadog_access_key
variable
datadog_access_key
variableWe will be utilizing a sensitive variable to pass your Datadog access key to the datadog-connector
module.
Security Note
The
datadog_access_key
variable is sensitive. Be sure to manage this value either:
- In AWS Secrets Manager, where the value can be retrieved using
aws_secretsmanager_secret_version
- In an environment variable or a
tfvars
file that is excluded from version control. For example,export TF_VAR_datadog_access_key="my-access-key"
.The Access Key will also still be stored in Terraform state, so ensure that your Terraform state file is securely stored. See Terraform's recommendations for storing sensitive state.
Declare a variable datadog_access_key
in a variables.tf
file in the same directory as your main.tf
file.
variable "datadog_access_key" {
description = "Secret used by the Firehose to send logs to Datadog. DO NOT check this into version control."
type = string
sensitive = true
}
Add the Datadog Connector Module
The Datadog Connector is a extended version of the Kinesis Firehose Connector, and will create a Kinesis Firehose Delivery Stream that will pipe reporting logs to Datadog, given an API key. The output of this module will be used to configure a Sym Log Destination next.
Check your environments!
Make sure that the
environment
in yourdatadog_connector
module matches the value in the SymEnv tag defined in theaws_iam_policy.aws_kinesis_firehose
policy!
module "datadog_connector" {
source = "symopsio/datadog-connector/aws"
version = ">= 2.0.0"
# This environment value MUST match the SymEnv value specified in the aws_iam_policy.aws_kinesis_firehose policy
environment = local.environment_name
# This variable should NOT be checked into tfvars!
datadog_access_key = var.datadog_access_key
# If you are in the EU, you will need to specify the EU intake URL!
# datadog_intake_url = "https://aws-kinesis-http-intake.logs.datadoghq.eu/v1/input"
}
Add a Log Destination
Define a sym_log_destination
resource with type = kinesis_firehose
.
integration_id
: The integration containing the permissions to push to Kinesis Firehose. This should be set to your Runtime Permission Context Integration, which has the permissions defined by theaws_iam_policy.aws_kinesis_firehose
resource.stream_name
: The name of the Kinesis Firehose Delivery Stream
resource "sym_log_destination" "datadog" {
type = "kinesis_firehose"
# The Runtime Permission Context has Kinesis Firehose permissions from aws_iam_policy.aws_kinesis_firehose policy
integration_id = sym_integration.runtime_context.id
settings = {
# The firehose stream name is outputted by the datadog_connector module
stream_name = module.datadog_connector.firehose_name
}
}
Add the Log Destination to your Environment
Each sym_environment
accepts a list of Log Destinations to send reporting logs to. Add the ID of the Log Destination you just defined to the log_destination_ids
list.
resource "sym_environment" "this" {
name = "main"
runtime_id = sym_runtime.this.id
error_logger_id = sym_error_logger.slack.id
# Add your log destinations here
log_destination_ids = [sym_log_destination.datadog.id]
integrations = {
slack_id = sym_integration.slack.id
}
}
Full Example
You can find the complete code for this example in our Datadog Log Destination Example.
Updated 11 days ago