AWS Datadog integration module.

Terraform module which creates Datadog integration on AWS.

Usage

module "datadog" {
  source                      = "toluna-terraform/datadog-integration/aws"
  version                     = "~>2.0.0"
  dd_api_key                  = "<string>"
  dd_app_key                  = "<string>"
}

What module does?

By default this module will provision:

1. Role that allows datadog aws account to collect data.
2. Policy that allows datadog account to access different resources.
3. Integration between the AWS account and Datadog portal (by default collects logs from lambda functions).
4. Official datadog cloudformation stack that creates a lambda (by default pointed to datadoghq.com) which can forward logs to datadog portal.

Create Datadog forwarder.

By default create_datadog_forwarder is true.
In case you have multiple projects on the same account with different state files,
You should keep create_datadog_forwarder disabled false and all projects besides one that you choose as your main.
This done to avoid errors of resource X already exist when other project will try to create additional forwarder.
In that case you will still be able to subscribe to forwarder by passing cloudwatch_log_groups.
in order to change the default add an attribute create_datadog_forwarder with desired value.

module "datadog" {
  source                      = "toluna-terraform/datadog-integration/aws"
  version                     = "~>2.0.0"
  dd_api_key                  = "<string>"
  dd_app_key                  = "<string>"
  create_datadog_forwarder    = false
}

AWS Regions.

The regions of your AWS account for datadog_integration_aws.
By default aws_regions is ["us-east-1"].
All other regions are excluded by default.
in order to change the default add an attribute aws_regions with desired value.
You can find a list of all excluded regions down below in README.md.

module "datadog" {
  source                      = "toluna-terraform/datadog-integration/aws"
  version                     = "~>2.0.0"
  dd_api_key                  = "<string>"
  dd_app_key                  = "<string>"
  aws_regions                 = ["<list of strings>"]
}

Datadog forwarder AWS Region.

The region of your AWS account on which Datadog forwarder will be installed.
By default datadog_farwarder_aws_region is us-east-1.
in order to change the default add an attribute datadog_farwarder_aws_region with desired value.

module "datadog" {
  source                       = "toluna-terraform/datadog-integration/aws"
  version                      = "~>2.0.0"
  dd_api_key                   = "<string>"
  dd_app_key                   = "<string>"
  datadog_farwarder_aws_region = "<string>"
}

Log groups as list.

Log groups you want to be subscribed to datadog forwarder.
By default cloudwatch_log_groups is {}.
in order to change the default add an attribute cloudwatch_log_groups with desired value.
The value should be a list where each element is a string of log group name.

toset(["/aws/lambda/log_group1","/aws/ecs/log_group2"])

Please see example in examples/datadog-integration-with-log-groups folder.

module "datadog" {
  source                      = "toluna-terraform/datadog-integration/aws"
  version                     = "~>2.0.0"
  dd_api_key                  = "<string>"
  dd_app_key                  = "<string>"
  cloudwatch_log_groups       = toset(["<list of strings>"])
}

Log groups as map.

Log groups you want to be subscribed to datadog forwarder.
By default cloudwatch_log_groups_as_list is {}.
in order to change the default add an attribute cloudwatch_log_groups_as_list with desired value.
The value should be a map of maps where each is identified by a string label and have a key name with the value of a log group name.

{log_group1={name="/aws/lambda/log_group1"},log_group2={name="/aws/ecs/log_group2"}}

Please see example in examples/datadog-integration-with-log-groups folder.

module "datadog" {
  source                        = "toluna-terraform/datadog-integration/aws"
  version                       = "~>2.0.0"
  dd_api_key                    = "<string>"
  dd_app_key                    = "<string>"
  cloudwatch_log_groups_as_list = {{map of maps}}
}

Datadog site.

Datadog Site to send data to.
By default dd_site is "datadoghq.com".
in order to change the default datadog site add an attribute dd_site with desired value.

module "datadog" {
  source                      = "toluna-terraform/datadog-integration/aws"
  version                     = "~>2.0.0"
  dd_api_key                  = "<string>"
  dd_app_key                  = "<string>"
  dd_site                     = "<string>"
}

You can find here a list of datadog sites.

Datadog tags.

Add custom tags to forwarded logs, comma-delimited string, no trailing comma, e.g., "env:prod,stack:classic"
By default dd_tags is empty
in order to change the default datadog tags add an attribute dd_tags with desired value.

module "datadog" {
  source                      = "toluna-terraform/datadog-integration/aws"
  version                     = "~>2.0.0"
  dd_api_key                  = "<string>"
  dd_app_key                  = "<string>"
  dd_tags                     = "<string>"
}

Add custom tags to forwarded logs,
comma-delimited string,
no trailing comma,
e.g., env:prod,stack:classic

Datadog log collection services.

A list of services which Datadog will automatically collect logs from.
By default log_collection_services is empty
In order to change the default log collection services add an attribute log_collection_services with desired value.

module "datadog" {
  source                      = "toluna-terraform/datadog-integration/aws"
  version                     = "~>2.0.0"
  dd_api_key                  = "<string>"
  dd_app_key                  = "<string>"
  log_collection_services     = ["<list of strings>"]
}

For more info on log_collection_services please visit this link.

Datadog metrics collection.

Provides to Datadog the ability to collect metrics for your AWS account.
By default metrics_collection_enabled is "true".
But by default we decided to disable all metrics and enable only the provided via metrics_to_collect parameter.
In order to change the default metrics_collection_enabled add an attribute metrics_collection_enabled with desired value.

module "datadog" {
  source                      = "toluna-terraform/datadog-integration/aws"
  version                     = "~>2.0.0"
  dd_api_key                  = "<string>"
  dd_app_key                  = "<string>"
  metrics_collection_enabled  = "<string>"
}

For more info on metrics_collection_enabled please visit this link.

Datadog metrics to collect.

A list of specific metrics which we want Datadog to collect.
By default metrics_to_collect is ["lambda"].
In order to change the default metrics_to_collect add an attribute metrics_to_collect with desired value.

module "datadog" {
  source                      = "toluna-terraform/datadog-integration/aws"
  version                     = "~>2.0.0"
  dd_api_key                  = "<string>"
  dd_app_key                  = "<string>"
  metrics_to_collect           = ["<list of strings>"]
}

For more info on available metrics to collect please visit this link.

Datadog resource collection.

Datadog collects a standard set of resources from your AWS account.
By default resource_collection_enabled is "false".
In order to change the default resource_collection_enabled add an attribute resource_collection_enabled with desired value.

module "datadog" {
  source                      = "toluna-terraform/datadog-integration/aws"
  version                     = "~>2.0.0"
  dd_api_key                  = "<string>"
  dd_app_key                  = "<string>"
  resource_collection_enabled = "<string>"
}

For more info on resource_collection_enabled please visit this link.

Datadog excluded logs pattern.

You can pass a specific logs pattern which you want to be excluded from forwarding.
By default exclude_logs_pattern is "\"(START|END|REPORT) RequestId:\\s || \"(EXTENSION|TELEMETRY) Name:\\s"
to exclude Lambda invocation logs and datadog-agent status logs
In order to change the default exclude logs pattern add an attribute exclude_logs_pattern with desired value.
For more info please visit this link.

module "datadog" {
  source                      = "toluna-terraform/datadog-integration/aws"
  version                     = "~>2.0.0"
  dd_api_key                  = "<string>"
  dd_app_key                  = "<string>"
  exclude_logs_pattern        = "<string>"
}

For more information about Datadog forwarder please visit link.

Requirements

Name	Version
datadog	3.22.0

Providers

Name	Version
aws	n/a
datadog	3.22.0

Modules

No modules.

Resources

Name	Type
aws_cloudformation_stack.datadog_forwarder	resource
aws_cloudwatch_log_subscription_filter.datadog_log_subscription_filter	resource
aws_iam_policy.datadog_integration_policy	resource
aws_iam_role.datadog_integration_role	resource
aws_iam_role_policy_attachment.datadog_policy_attach	resource
aws_secretsmanager_secret.dd_api_key	resource
aws_secretsmanager_secret_version.dd_api_key	resource
datadog_integration_aws.integration	resource
datadog_integration_aws_lambda_arn.main_collector	resource
datadog_integration_aws_log_collection.main	resource
aws_caller_identity.current	data source

Inputs

Name	Description	Type	Default	Required
aws_regions	An array of AWS regions to include for metrics collection.	list	[ "us-east-1" ]	no
cloudwatch_log_groups	List of cloudwatch log groups.	map	{}	no
create_datadog_forwarder	n/a	bool	true	no
datadog_aws_account_id	The AWS account ID Datadog's integration servers use for all integrations	string	"464622532012"	no
datadog_cloudformation_template	Official CloudFormation template provided by Datadog	string	"https://datadog-cloudformation-template.s3.amazonaws.com/aws/forwarder/latest.yaml"	no
datadog_forwarder_aws_region	A region on which datadog forwarder is deployed.	string	"us-east-1"	no
datadog_forwarder_function_name	Datadog forwarder lambda function name	string	"datadog-forwarder"	no
datadog_policy_name	The set of permissions necessary to use all the integrations for individual AWS services.	string	"DatadogAWSIntegrationPolicy"	no
datadog_role_name	Enable Datadog to collect metrics, tags, CloudWatch events, and other data necessary to monitor your AWS environment.	string	"DatadogAWSIntegrationRole"	no
dd_api_key	The Datadog API key	string	n/a	yes
dd_app_key	The Datadog APP key	string	n/a	yes
dd_site	Datadog Site to send data to.	string	"datadoghq.com"	no
dd_tags	Add custom tags to forwarded logs, comma-delimited string, no trailing comma, e.g., env:prod,stack:classic	string	""	no
exclude_logs_pattern	This pattern will exclude lambda execution report only ERROR report will be forwarded. By default forwarder will exclude reports of Agent and Tracer	string	`""(START	END
excluded_aws_regions	An array of AWS regions to exclude from metrics collection.	list	[ "us-east-2", "us-east-1", "us-west-1", "us-west-2", "af-south-1", "ap-east-1", "ap-south-2", "ap-southeast-3", "ap-south-1", "ap-northeast-3", "ap-northeast-2", "ap-southeast-1", "ap-southeast-2", "ap-northeast-1", "ca-central-1", "eu-central-1", "eu-west-1", "eu-west-2", "eu-south-1", "eu-west-3", "eu-south-2", "eu-north-1", "eu-central-2", "me-south-1", "me-central-1", "sa-east-1" ]	no
log_collection_services	A list of services which Datadog will automatically collect logs from. See the api docs (README.md) for more details on which services are supported.	list	[]	no
metrics_collection_enabled	Datadog collects metrics for this AWS account.	string	"true"	no
resource_collection_enabled	Datadog collects a standard set of resources from your AWS account.	string	"false"	no

Outputs

No outputs.

Authors

Module is maintained by Evgeny Gigi.