Initial commit with README and module files
This commit is contained in:
commit
35d568c682
9
.gitignore
vendored
Executable file
9
.gitignore
vendored
Executable file
@ -0,0 +1,9 @@
|
|||||||
|
*.tfstate
|
||||||
|
*.tfstate.backup
|
||||||
|
.terraform
|
||||||
|
provider.tf
|
||||||
|
*.tfvars
|
||||||
|
**/*.tfvars
|
||||||
|
#provider.tf
|
||||||
|
.github
|
||||||
|
.circleci
|
||||||
102
README.md
Executable file
102
README.md
Executable file
@ -0,0 +1,102 @@
|
|||||||
|
# Terraform Datadog Slack Module
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
This Terraform module creates a Kubernetes/Docker application monitoring dashboard in Datadog configured specifically for Slack/team deployments with account-scoped filtering.
|
||||||
|
|
||||||
|
## Features
|
||||||
|
|
||||||
|
- **Kubernetes Resource Monitoring**: Visualizes pod and node resource utilization
|
||||||
|
- **Account-Scoped Filtering**: Filters metrics by team and environment
|
||||||
|
- **CPU & Memory Tracking**: Timeseries visualization for top containers
|
||||||
|
- **Read-Only Dashboard**: Prevents accidental modifications
|
||||||
|
- **EU Datadog Instance**: Configured for European GDPR compliance
|
||||||
|
|
||||||
|
## Resources Created
|
||||||
|
|
||||||
|
- `datadog_dashboard`: Application monitoring dashboard with account-scoped widgets
|
||||||
|
|
||||||
|
## Dashboard Widgets
|
||||||
|
|
||||||
|
1. **Kubernetes Pods Hostmap**: CPU utilization by Docker image (account-scoped)
|
||||||
|
2. **CPU Utilization Timeseries**: Container CPU usage
|
||||||
|
3. **Kubernetes Nodes Hostmap**: CPU utilization by host
|
||||||
|
4. **Memory Utilization Timeseries**: Container memory usage
|
||||||
|
|
||||||
|
## Requirements
|
||||||
|
|
||||||
|
| Name | Version |
|
||||||
|
|------|---------|
|
||||||
|
| terraform | >= 0.12 |
|
||||||
|
| datadog | >= 3.5.0 |
|
||||||
|
|
||||||
|
## Usage
|
||||||
|
|
||||||
|
```hcl
|
||||||
|
module "slack_dashboard" {
|
||||||
|
source = "./terraform-datadog-slack"
|
||||||
|
|
||||||
|
opco_name = "sanoma"
|
||||||
|
app_name = "slack-integration"
|
||||||
|
team_name = "platform-team"
|
||||||
|
image_name = "slack-app"
|
||||||
|
aws_region = "eu-west-1"
|
||||||
|
env = "prd"
|
||||||
|
api_key = var.datadog_api_key
|
||||||
|
app_key = var.datadog_app_key
|
||||||
|
url = "https://slack.example.com/health"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Inputs
|
||||||
|
|
||||||
|
| Name | Description | Type | Required |
|
||||||
|
|------|-------------|------|----------|
|
||||||
|
| `opco_name` | Name of the OPCO | `string` | yes |
|
||||||
|
| `app_name` | Name of the application | `string` | yes |
|
||||||
|
| `team_name` | Name of the responsible team | `string` | yes |
|
||||||
|
| `image_name` | Docker image name | `string` | yes |
|
||||||
|
| `aws_region` | AWS region for resources | `string` | yes |
|
||||||
|
| `env` | Environment (dev, tst, stg, prd) | `string` | yes |
|
||||||
|
| `api_key` | Datadog API key | `string` | yes |
|
||||||
|
| `app_key` | Datadog APP key | `string` | yes |
|
||||||
|
| `url` | Synthetics URL | `string` | yes |
|
||||||
|
|
||||||
|
## Outputs
|
||||||
|
|
||||||
|
Currently, this module does not export any outputs.
|
||||||
|
|
||||||
|
## Account Scoping
|
||||||
|
|
||||||
|
The dashboard filters metrics using account-based scoping:
|
||||||
|
- **Scope**: `account:{team_name}_{env}`
|
||||||
|
- This ensures each team only sees their environment's metrics
|
||||||
|
|
||||||
|
## Differences from terraform-datadog-app-dashboard
|
||||||
|
|
||||||
|
While similar to the app-dashboard module, this version:
|
||||||
|
- Uses account-scoped filtering instead of namespace
|
||||||
|
- Configured specifically for EU Datadog API
|
||||||
|
- Simplified widget set (no alert graph)
|
||||||
|
- Synthetics integration is commented out
|
||||||
|
- Uses OPCO name instead of CFA name
|
||||||
|
|
||||||
|
## Provider Configuration
|
||||||
|
|
||||||
|
- **API URL**: `https://api.datadoghq.eu` (EU instance)
|
||||||
|
- **Datadog Provider**: Version 3.5.0 or higher
|
||||||
|
|
||||||
|
## Notes
|
||||||
|
|
||||||
|
- Dashboard is read-only to prevent accidental modifications
|
||||||
|
- All widgets filter by account scope `{team_name}_{env}`
|
||||||
|
- Designed for multi-tenant Kubernetes environments
|
||||||
|
- Uses EU Datadog API endpoint for GDPR compliance
|
||||||
|
|
||||||
|
## License
|
||||||
|
|
||||||
|
Internal use only - Sanoma/WeBuildYourCloud
|
||||||
|
|
||||||
|
## Authors
|
||||||
|
|
||||||
|
Created and maintained by the Platform Engineering team.
|
||||||
7
backend.tf
Executable file
7
backend.tf
Executable file
@ -0,0 +1,7 @@
|
|||||||
|
# Backend configuration for Terraform so we can centralize the state into an S3 bucket
|
||||||
|
# Do note that most of the settings to configure the backend need to be set in the repository variables in Bitbucket
|
||||||
|
|
||||||
|
terraform {
|
||||||
|
backend "s3" {
|
||||||
|
}
|
||||||
|
}
|
||||||
62
bitbucket-pipelines.yml
Executable file
62
bitbucket-pipelines.yml
Executable file
@ -0,0 +1,62 @@
|
|||||||
|
# Template Terraform to deploy to Cloud Infrastructure
|
||||||
|
|
||||||
|
# This template allows you to deploy your infrastructure using Terraform to supported cloud providers.
|
||||||
|
# The workflow allows running tests, security scans on feature branches (as well as master).
|
||||||
|
# After merging code to master the infrastructure will be deployed to cloud according to the given terraform template.
|
||||||
|
|
||||||
|
# Prerequisites: credentials according to used cloud provider.
|
||||||
|
# For advanced cases, please, follow terraform docs https://www.terraform.io/docs/index.html.
|
||||||
|
|
||||||
|
|
||||||
|
image: hashicorp/terraform
|
||||||
|
|
||||||
|
pipelines:
|
||||||
|
default:
|
||||||
|
- parallel:
|
||||||
|
- step:
|
||||||
|
name: Test
|
||||||
|
script:
|
||||||
|
- terraform init
|
||||||
|
- terraform validate
|
||||||
|
- step:
|
||||||
|
name: Security Scan
|
||||||
|
script:
|
||||||
|
# Run a security scan for sensitive data.
|
||||||
|
# See more security tools at https://bitbucket.org/product/features/pipelines/integrations?&category=security
|
||||||
|
- pipe: atlassian/git-secrets-scan:0.4.3
|
||||||
|
branches:
|
||||||
|
master:
|
||||||
|
#- step:
|
||||||
|
# name: Security Scan
|
||||||
|
# script:
|
||||||
|
# # Run a security scan for sensitive data.
|
||||||
|
# # See more security tools at https://bitbucket.org/product/features/pipelines/integrations?&category=security
|
||||||
|
# - pipe: atlassian/git-secrets-scan:0.4.3
|
||||||
|
- step:
|
||||||
|
name: Run Terraform Plan
|
||||||
|
#deployment: Plan
|
||||||
|
script:
|
||||||
|
- export TF_BACKEND_BUCKET=${TF_BACKEND_BUCKET}
|
||||||
|
- export TF_BACKEND_DYNDB_TABLE=${TF_BACKEND_DYNDB_TABLE}
|
||||||
|
- export TF_BACKEND_REGION=${TF_BACKEND_REGION}
|
||||||
|
- export TF_BACKEND_KEY=${TF_BACKEND_KEY}
|
||||||
|
- export TF_VAR_api_key=${TF_VAR_api_key}
|
||||||
|
- export TF_VAR_app_key=${TF_VAR_app_key}
|
||||||
|
- chmod +x ci-cd/scripts/terraform/terraform_plan.sh
|
||||||
|
- ./ci-cd/scripts/terraform/terraform_plan.sh
|
||||||
|
artifacts:
|
||||||
|
- .terraform/**
|
||||||
|
- tfplan
|
||||||
|
- step:
|
||||||
|
name: Deploy to Production
|
||||||
|
#deployment: Apply
|
||||||
|
trigger: manual
|
||||||
|
script:
|
||||||
|
- pwd && ls -la ../artifact/
|
||||||
|
- pwd && ls -la ../data/
|
||||||
|
- pwd && ls -la ../tmp/
|
||||||
|
- pwd && ls -la
|
||||||
|
- export TF_IN_AUTOMATION=1
|
||||||
|
- terraform init
|
||||||
|
- terraform apply tfplan
|
||||||
|
|
||||||
30
ci-cd/scripts/terraform/terraform_apply.sh
Executable file
30
ci-cd/scripts/terraform/terraform_apply.sh
Executable file
@ -0,0 +1,30 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
#
|
||||||
|
# Check for and exit on errors
|
||||||
|
set -e
|
||||||
|
# Script to implement some functions to standardize the workflow for TerraformA In CI/CD Pipelines
|
||||||
|
#
|
||||||
|
#
|
||||||
|
|
||||||
|
## Global Variables ##
|
||||||
|
PS4='LINENO:' # Set line numbers on error output to ease debugging
|
||||||
|
|
||||||
|
## Terraform Apply Function
|
||||||
|
|
||||||
|
echo "Applying for environment ${TF_WORKSPACE}"
|
||||||
|
cd tf/templates
|
||||||
|
|
||||||
|
terraform_1.0.4 apply -input=false -auto-approve tfplan 2>&1 | \
|
||||||
|
sed -r "s/\x1B\[([0-9]{1,2}(;[0-9]{1,2})?)?[mGK]//g" ; \
|
||||||
|
test ${PIPESTATUS[0]} -eq 0
|
||||||
|
|
||||||
|
cd ../../
|
||||||
|
|
||||||
|
# Run Terraform apply on our workspaces
|
||||||
|
#for I in `cat tf/config/datadog-integrations.tfvars |scripts/hcl2json |jq -r '.aws_account_id | keys[]'`
|
||||||
|
# do
|
||||||
|
# echo "Planning Terraform run for workspace ${I}"
|
||||||
|
# export TF_WORKSPACE="${I}"
|
||||||
|
# terraform_apply
|
||||||
|
# done
|
||||||
|
|
||||||
60
ci-cd/scripts/terraform/terraform_plan.sh
Executable file
60
ci-cd/scripts/terraform/terraform_plan.sh
Executable file
@ -0,0 +1,60 @@
|
|||||||
|
# Check for and exit on errors
|
||||||
|
set -e
|
||||||
|
# Script to implement some functions to standardize the workflow for TerraformA In CI/CD Pipelines
|
||||||
|
#
|
||||||
|
#
|
||||||
|
|
||||||
|
## Global Variables ##
|
||||||
|
PS4='LINENO:' # Set line numbers on error output to ease debugging
|
||||||
|
|
||||||
|
|
||||||
|
if [ -z $TF_BACKEND_BUCKET ]; then
|
||||||
|
die 127 "No TF_BACKEND_BUCKET variable specified or empty variable"
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [ -z $TF_BACKEND_DYNDB_TABLE ]; then
|
||||||
|
die 127 "No TF_BACKEND_DYNDB_TABLE variable specified or empty variable"
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [ -z $TF_BACKEND_REGION ]; then
|
||||||
|
die 127 "No TF_BACKEND_REGION variable specified or empty variable"
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [ -z $TF_BACKEND_KEY ]; then
|
||||||
|
die 127 "No TF_BACKEND_KEY variable specified or empty variable"
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [ -z $TF_VAR_api_key ]; then
|
||||||
|
die 127 "No TF_VAR_api_key variable specified or empty variable"
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [ -z $TF_VAR_app_key ]; then
|
||||||
|
die 127 "No TF_VAR_app_key variable specified or empty variable"
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo "Planning for deployment of API and APP keys"
|
||||||
|
terraform init -backend-config="encrypt=true" \
|
||||||
|
-backend-config="bucket=$TF_BACKEND_BUCKET" \
|
||||||
|
-backend-config="dynamodb_table=$TF_BACKEND_DYNDB_TABLE" \
|
||||||
|
-backend-config="region=$TF_BACKEND_REGION" \
|
||||||
|
-backend-config="key=$TF_BACKEND_KEY" \
|
||||||
|
-input=false 2>&1 | sed -r "s/\x1B\[([0-9]{1,2}(;[0-9]{1,2})?)?[mGK]//g" # ; test ${PIPESTATUS[0]} -eq 0
|
||||||
|
|
||||||
|
terraform validate 2>&1 | \
|
||||||
|
sed -r "s/\x1B\[([0-9]{1,2}(;[0-9]{1,2})?)?[mGK]//g" ; #\
|
||||||
|
#test ${PIPESTATUS[0]} -eq 0
|
||||||
|
|
||||||
|
terraform plan -out=tfplan -input=false -detailed-exitcode \
|
||||||
|
-var-file=terraform.tfvars 2>&1 | \
|
||||||
|
sed -r "s/\x1B\[([0-9]{1,2}(;[0-9]{1,2})?)?[mGK]//g" ; #\
|
||||||
|
#test ${PIPESTATUS[0]} -ne 1
|
||||||
|
|
||||||
|
#terraform plan -out=tfplan -input=false -detailed-exitcode \
|
||||||
|
# -var-file=terraform.tfvars \
|
||||||
|
# -var="datadog_api_key=${TF_VAR_datadog_api_key}" \
|
||||||
|
# -var="datadog_app_key=${TF_VAR_datadog_api_key}" 2>&1 | \
|
||||||
|
# sed -r "s/\x1B\[([0-9]{1,2}(;[0-9]{1,2})?)?[mGK]//g" ; #\
|
||||||
|
# #test ${PIPESTATUS[0]} -ne 1
|
||||||
|
ls -la
|
||||||
|
|
||||||
|
|
||||||
105
main.tf
Executable file
105
main.tf
Executable file
@ -0,0 +1,105 @@
|
|||||||
|
resource "datadog_dashboard" "app_dashboard" {
|
||||||
|
title = var.app_name
|
||||||
|
description = "A Datadog Dashboard for the ${var.app_name} deployment on the ${var.env} environment"
|
||||||
|
layout_type = "ordered"
|
||||||
|
is_read_only = true
|
||||||
|
|
||||||
|
|
||||||
|
widget {
|
||||||
|
hostmap_definition {
|
||||||
|
no_group_hosts = true
|
||||||
|
no_metric_hosts = true
|
||||||
|
node_type = "container"
|
||||||
|
scope = ["account:${var.team_name}_${var.env}"]
|
||||||
|
title = "Kubernetes Pods"
|
||||||
|
|
||||||
|
request {
|
||||||
|
fill {
|
||||||
|
q = "avg:process.stat.container.cpu.total_pct{image_name:${var.image_name}} by {host}"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
style {
|
||||||
|
palette = "hostmap_blues"
|
||||||
|
palette_flip = false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
widget {
|
||||||
|
timeseries_definition {
|
||||||
|
show_legend = false
|
||||||
|
title = "CPU Utilization"
|
||||||
|
|
||||||
|
request {
|
||||||
|
display_type = "line"
|
||||||
|
q = "top(avg:docker.cpu.usage{image_name:${var.image_name}} by {docker_image,container_id}, 10, 'mean', 'desc')"
|
||||||
|
|
||||||
|
style {
|
||||||
|
line_type = "solid"
|
||||||
|
line_width = "normal"
|
||||||
|
palette = "dog_classic"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
yaxis {
|
||||||
|
include_zero = true
|
||||||
|
max = "auto"
|
||||||
|
min = "auto"
|
||||||
|
scale = "linear"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
# widget {
|
||||||
|
# alert_graph_definition {
|
||||||
|
# alert_id = datadog_monitor.app_monitor.id
|
||||||
|
# title = "Kubernetes Node CPU"
|
||||||
|
# viz_type = "timeseries"
|
||||||
|
# }
|
||||||
|
# }
|
||||||
|
|
||||||
|
widget {
|
||||||
|
hostmap_definition {
|
||||||
|
no_group_hosts = true
|
||||||
|
no_metric_hosts = true
|
||||||
|
node_type = "host"
|
||||||
|
title = "Kubernetes Nodes"
|
||||||
|
|
||||||
|
request {
|
||||||
|
fill {
|
||||||
|
q = "avg:system.cpu.user{*} by {host}"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
style {
|
||||||
|
palette = "hostmap_blues"
|
||||||
|
palette_flip = false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
widget {
|
||||||
|
timeseries_definition {
|
||||||
|
show_legend = false
|
||||||
|
title = "Memory Utilization"
|
||||||
|
request {
|
||||||
|
display_type = "line"
|
||||||
|
q = "top(avg:docker.mem.in_use{image_name:${var.image_name}} by {container_name}, 10, 'mean', 'desc')"
|
||||||
|
|
||||||
|
style {
|
||||||
|
line_type = "solid"
|
||||||
|
line_width = "normal"
|
||||||
|
palette = "dog_classic"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
yaxis {
|
||||||
|
include_zero = true
|
||||||
|
max = "auto"
|
||||||
|
min = "auto"
|
||||||
|
scale = "linear"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
0
outputs.tf
Executable file
0
outputs.tf
Executable file
68
variables.tf
Executable file
68
variables.tf
Executable file
@ -0,0 +1,68 @@
|
|||||||
|
variable "opco_name" {
|
||||||
|
description = "Name of the OPCO"
|
||||||
|
type = string
|
||||||
|
}
|
||||||
|
|
||||||
|
variable "app_name" {
|
||||||
|
description = "Name of the application"
|
||||||
|
type = string
|
||||||
|
}
|
||||||
|
|
||||||
|
variable "team_name" {
|
||||||
|
description = "Name of the responsible team"
|
||||||
|
type = string
|
||||||
|
}
|
||||||
|
|
||||||
|
variable "image_name" {
|
||||||
|
description = "Name of the responsible team"
|
||||||
|
type = string
|
||||||
|
}
|
||||||
|
|
||||||
|
variable "aws_region" {
|
||||||
|
description = "Defines the AWS region where the resources are located"
|
||||||
|
type = string
|
||||||
|
}
|
||||||
|
|
||||||
|
variable "env" {
|
||||||
|
description = "Specifies the environment to monitor (dev, tst, stg, prd)"
|
||||||
|
type = string
|
||||||
|
}
|
||||||
|
|
||||||
|
variable "api_key" {
|
||||||
|
description = "Set the Datadog API key"
|
||||||
|
type = string
|
||||||
|
}
|
||||||
|
|
||||||
|
variable "app_key" {
|
||||||
|
description = "Set the Datadog APP key"
|
||||||
|
type = string
|
||||||
|
}
|
||||||
|
|
||||||
|
#variable "api_url" {
|
||||||
|
# description = "Which API to Connect to, we are using the EU one for GDPR compliance"
|
||||||
|
# type = string
|
||||||
|
# default = "https://api.datadoghq.eu"
|
||||||
|
#}
|
||||||
|
|
||||||
|
#variable "http_client_retry_enabled" {
|
||||||
|
# description = "Enables Request retries on HTTP status codes 429 and 5xx"
|
||||||
|
# type = bool
|
||||||
|
# default = true
|
||||||
|
#}
|
||||||
|
|
||||||
|
#variable "http_client_retry_timeout" {
|
||||||
|
# description = "Sets the number of HTTP request retry timeout period"
|
||||||
|
# type = string
|
||||||
|
# default = ""
|
||||||
|
#}
|
||||||
|
|
||||||
|
#variable "validate" {
|
||||||
|
# description = "Validates the provided APP and API keys during provider initialization"
|
||||||
|
# type = bool
|
||||||
|
# default = true
|
||||||
|
#}
|
||||||
|
|
||||||
|
variable "url" {
|
||||||
|
description = "Synthetics URL"
|
||||||
|
type = string
|
||||||
|
}
|
||||||
Loading…
x
Reference in New Issue
Block a user