Initial commit with README and module files
This commit is contained in:
commit
35d568c682
9
.gitignore
vendored
Executable file
9
.gitignore
vendored
Executable file
@ -0,0 +1,9 @@
|
||||
*.tfstate
|
||||
*.tfstate.backup
|
||||
.terraform
|
||||
provider.tf
|
||||
*.tfvars
|
||||
**/*.tfvars
|
||||
#provider.tf
|
||||
.github
|
||||
.circleci
|
||||
102
README.md
Executable file
102
README.md
Executable file
@ -0,0 +1,102 @@
|
||||
# Terraform Datadog Slack Module
|
||||
|
||||
## Overview
|
||||
|
||||
This Terraform module creates a Kubernetes/Docker application monitoring dashboard in Datadog configured specifically for Slack/team deployments with account-scoped filtering.
|
||||
|
||||
## Features
|
||||
|
||||
- **Kubernetes Resource Monitoring**: Visualizes pod and node resource utilization
|
||||
- **Account-Scoped Filtering**: Filters metrics by team and environment
|
||||
- **CPU & Memory Tracking**: Timeseries visualization for top containers
|
||||
- **Read-Only Dashboard**: Prevents accidental modifications
|
||||
- **EU Datadog Instance**: Configured for European GDPR compliance
|
||||
|
||||
## Resources Created
|
||||
|
||||
- `datadog_dashboard`: Application monitoring dashboard with account-scoped widgets
|
||||
|
||||
## Dashboard Widgets
|
||||
|
||||
1. **Kubernetes Pods Hostmap**: CPU utilization by Docker image (account-scoped)
|
||||
2. **CPU Utilization Timeseries**: Container CPU usage
|
||||
3. **Kubernetes Nodes Hostmap**: CPU utilization by host
|
||||
4. **Memory Utilization Timeseries**: Container memory usage
|
||||
|
||||
## Requirements
|
||||
|
||||
| Name | Version |
|
||||
|------|---------|
|
||||
| terraform | >= 0.12 |
|
||||
| datadog | >= 3.5.0 |
|
||||
|
||||
## Usage
|
||||
|
||||
```hcl
|
||||
module "slack_dashboard" {
|
||||
source = "./terraform-datadog-slack"
|
||||
|
||||
opco_name = "sanoma"
|
||||
app_name = "slack-integration"
|
||||
team_name = "platform-team"
|
||||
image_name = "slack-app"
|
||||
aws_region = "eu-west-1"
|
||||
env = "prd"
|
||||
api_key = var.datadog_api_key
|
||||
app_key = var.datadog_app_key
|
||||
url = "https://slack.example.com/health"
|
||||
}
|
||||
```
|
||||
|
||||
## Inputs
|
||||
|
||||
| Name | Description | Type | Required |
|
||||
|------|-------------|------|----------|
|
||||
| `opco_name` | Name of the OPCO | `string` | yes |
|
||||
| `app_name` | Name of the application | `string` | yes |
|
||||
| `team_name` | Name of the responsible team | `string` | yes |
|
||||
| `image_name` | Docker image name | `string` | yes |
|
||||
| `aws_region` | AWS region for resources | `string` | yes |
|
||||
| `env` | Environment (dev, tst, stg, prd) | `string` | yes |
|
||||
| `api_key` | Datadog API key | `string` | yes |
|
||||
| `app_key` | Datadog APP key | `string` | yes |
|
||||
| `url` | Synthetics URL | `string` | yes |
|
||||
|
||||
## Outputs
|
||||
|
||||
Currently, this module does not export any outputs.
|
||||
|
||||
## Account Scoping
|
||||
|
||||
The dashboard filters metrics using account-based scoping:
|
||||
- **Scope**: `account:{team_name}_{env}`
|
||||
- This ensures each team only sees their environment's metrics
|
||||
|
||||
## Differences from terraform-datadog-app-dashboard
|
||||
|
||||
While similar to the app-dashboard module, this version:
|
||||
- Uses account-scoped filtering instead of namespace
|
||||
- Configured specifically for EU Datadog API
|
||||
- Simplified widget set (no alert graph)
|
||||
- Synthetics integration is commented out
|
||||
- Uses OPCO name instead of CFA name
|
||||
|
||||
## Provider Configuration
|
||||
|
||||
- **API URL**: `https://api.datadoghq.eu` (EU instance)
|
||||
- **Datadog Provider**: Version 3.5.0 or higher
|
||||
|
||||
## Notes
|
||||
|
||||
- Dashboard is read-only to prevent accidental modifications
|
||||
- All widgets filter by account scope `{team_name}_{env}`
|
||||
- Designed for multi-tenant Kubernetes environments
|
||||
- Uses EU Datadog API endpoint for GDPR compliance
|
||||
|
||||
## License
|
||||
|
||||
Internal use only - Sanoma/WeBuildYourCloud
|
||||
|
||||
## Authors
|
||||
|
||||
Created and maintained by the Platform Engineering team.
|
||||
7
backend.tf
Executable file
7
backend.tf
Executable file
@ -0,0 +1,7 @@
|
||||
# Backend configuration for Terraform so we can centralize the state into an S3 bucket
|
||||
# Do note that most of the settings to configure the backend need to be set in the repository variables in Bitbucket
|
||||
|
||||
terraform {
|
||||
backend "s3" {
|
||||
}
|
||||
}
|
||||
62
bitbucket-pipelines.yml
Executable file
62
bitbucket-pipelines.yml
Executable file
@ -0,0 +1,62 @@
|
||||
# Template Terraform to deploy to Cloud Infrastructure
|
||||
|
||||
# This template allows you to deploy your infrastructure using Terraform to supported cloud providers.
|
||||
# The workflow allows running tests, security scans on feature branches (as well as master).
|
||||
# After merging code to master the infrastructure will be deployed to cloud according to the given terraform template.
|
||||
|
||||
# Prerequisites: credentials according to used cloud provider.
|
||||
# For advanced cases, please, follow terraform docs https://www.terraform.io/docs/index.html.
|
||||
|
||||
|
||||
image: hashicorp/terraform
|
||||
|
||||
pipelines:
|
||||
default:
|
||||
- parallel:
|
||||
- step:
|
||||
name: Test
|
||||
script:
|
||||
- terraform init
|
||||
- terraform validate
|
||||
- step:
|
||||
name: Security Scan
|
||||
script:
|
||||
# Run a security scan for sensitive data.
|
||||
# See more security tools at https://bitbucket.org/product/features/pipelines/integrations?&category=security
|
||||
- pipe: atlassian/git-secrets-scan:0.4.3
|
||||
branches:
|
||||
master:
|
||||
#- step:
|
||||
# name: Security Scan
|
||||
# script:
|
||||
# # Run a security scan for sensitive data.
|
||||
# # See more security tools at https://bitbucket.org/product/features/pipelines/integrations?&category=security
|
||||
# - pipe: atlassian/git-secrets-scan:0.4.3
|
||||
- step:
|
||||
name: Run Terraform Plan
|
||||
#deployment: Plan
|
||||
script:
|
||||
- export TF_BACKEND_BUCKET=${TF_BACKEND_BUCKET}
|
||||
- export TF_BACKEND_DYNDB_TABLE=${TF_BACKEND_DYNDB_TABLE}
|
||||
- export TF_BACKEND_REGION=${TF_BACKEND_REGION}
|
||||
- export TF_BACKEND_KEY=${TF_BACKEND_KEY}
|
||||
- export TF_VAR_api_key=${TF_VAR_api_key}
|
||||
- export TF_VAR_app_key=${TF_VAR_app_key}
|
||||
- chmod +x ci-cd/scripts/terraform/terraform_plan.sh
|
||||
- ./ci-cd/scripts/terraform/terraform_plan.sh
|
||||
artifacts:
|
||||
- .terraform/**
|
||||
- tfplan
|
||||
- step:
|
||||
name: Deploy to Production
|
||||
#deployment: Apply
|
||||
trigger: manual
|
||||
script:
|
||||
- pwd && ls -la ../artifact/
|
||||
- pwd && ls -la ../data/
|
||||
- pwd && ls -la ../tmp/
|
||||
- pwd && ls -la
|
||||
- export TF_IN_AUTOMATION=1
|
||||
- terraform init
|
||||
- terraform apply tfplan
|
||||
|
||||
30
ci-cd/scripts/terraform/terraform_apply.sh
Executable file
30
ci-cd/scripts/terraform/terraform_apply.sh
Executable file
@ -0,0 +1,30 @@
|
||||
#!/usr/bin/env bash
|
||||
#
|
||||
# Check for and exit on errors
|
||||
set -e
|
||||
# Script to implement some functions to standardize the workflow for TerraformA In CI/CD Pipelines
|
||||
#
|
||||
#
|
||||
|
||||
## Global Variables ##
|
||||
PS4='LINENO:' # Set line numbers on error output to ease debugging
|
||||
|
||||
## Terraform Apply Function
|
||||
|
||||
echo "Applying for environment ${TF_WORKSPACE}"
|
||||
cd tf/templates
|
||||
|
||||
terraform_1.0.4 apply -input=false -auto-approve tfplan 2>&1 | \
|
||||
sed -r "s/\x1B\[([0-9]{1,2}(;[0-9]{1,2})?)?[mGK]//g" ; \
|
||||
test ${PIPESTATUS[0]} -eq 0
|
||||
|
||||
cd ../../
|
||||
|
||||
# Run Terraform apply on our workspaces
|
||||
#for I in `cat tf/config/datadog-integrations.tfvars |scripts/hcl2json |jq -r '.aws_account_id | keys[]'`
|
||||
# do
|
||||
# echo "Planning Terraform run for workspace ${I}"
|
||||
# export TF_WORKSPACE="${I}"
|
||||
# terraform_apply
|
||||
# done
|
||||
|
||||
60
ci-cd/scripts/terraform/terraform_plan.sh
Executable file
60
ci-cd/scripts/terraform/terraform_plan.sh
Executable file
@ -0,0 +1,60 @@
|
||||
# Check for and exit on errors
|
||||
set -e
|
||||
# Script to implement some functions to standardize the workflow for TerraformA In CI/CD Pipelines
|
||||
#
|
||||
#
|
||||
|
||||
## Global Variables ##
|
||||
PS4='LINENO:' # Set line numbers on error output to ease debugging
|
||||
|
||||
|
||||
if [ -z $TF_BACKEND_BUCKET ]; then
|
||||
die 127 "No TF_BACKEND_BUCKET variable specified or empty variable"
|
||||
fi
|
||||
|
||||
if [ -z $TF_BACKEND_DYNDB_TABLE ]; then
|
||||
die 127 "No TF_BACKEND_DYNDB_TABLE variable specified or empty variable"
|
||||
fi
|
||||
|
||||
if [ -z $TF_BACKEND_REGION ]; then
|
||||
die 127 "No TF_BACKEND_REGION variable specified or empty variable"
|
||||
fi
|
||||
|
||||
if [ -z $TF_BACKEND_KEY ]; then
|
||||
die 127 "No TF_BACKEND_KEY variable specified or empty variable"
|
||||
fi
|
||||
|
||||
if [ -z $TF_VAR_api_key ]; then
|
||||
die 127 "No TF_VAR_api_key variable specified or empty variable"
|
||||
fi
|
||||
|
||||
if [ -z $TF_VAR_app_key ]; then
|
||||
die 127 "No TF_VAR_app_key variable specified or empty variable"
|
||||
fi
|
||||
|
||||
echo "Planning for deployment of API and APP keys"
|
||||
terraform init -backend-config="encrypt=true" \
|
||||
-backend-config="bucket=$TF_BACKEND_BUCKET" \
|
||||
-backend-config="dynamodb_table=$TF_BACKEND_DYNDB_TABLE" \
|
||||
-backend-config="region=$TF_BACKEND_REGION" \
|
||||
-backend-config="key=$TF_BACKEND_KEY" \
|
||||
-input=false 2>&1 | sed -r "s/\x1B\[([0-9]{1,2}(;[0-9]{1,2})?)?[mGK]//g" # ; test ${PIPESTATUS[0]} -eq 0
|
||||
|
||||
terraform validate 2>&1 | \
|
||||
sed -r "s/\x1B\[([0-9]{1,2}(;[0-9]{1,2})?)?[mGK]//g" ; #\
|
||||
#test ${PIPESTATUS[0]} -eq 0
|
||||
|
||||
terraform plan -out=tfplan -input=false -detailed-exitcode \
|
||||
-var-file=terraform.tfvars 2>&1 | \
|
||||
sed -r "s/\x1B\[([0-9]{1,2}(;[0-9]{1,2})?)?[mGK]//g" ; #\
|
||||
#test ${PIPESTATUS[0]} -ne 1
|
||||
|
||||
#terraform plan -out=tfplan -input=false -detailed-exitcode \
|
||||
# -var-file=terraform.tfvars \
|
||||
# -var="datadog_api_key=${TF_VAR_datadog_api_key}" \
|
||||
# -var="datadog_app_key=${TF_VAR_datadog_api_key}" 2>&1 | \
|
||||
# sed -r "s/\x1B\[([0-9]{1,2}(;[0-9]{1,2})?)?[mGK]//g" ; #\
|
||||
# #test ${PIPESTATUS[0]} -ne 1
|
||||
ls -la
|
||||
|
||||
|
||||
105
main.tf
Executable file
105
main.tf
Executable file
@ -0,0 +1,105 @@
|
||||
resource "datadog_dashboard" "app_dashboard" {
|
||||
title = var.app_name
|
||||
description = "A Datadog Dashboard for the ${var.app_name} deployment on the ${var.env} environment"
|
||||
layout_type = "ordered"
|
||||
is_read_only = true
|
||||
|
||||
|
||||
widget {
|
||||
hostmap_definition {
|
||||
no_group_hosts = true
|
||||
no_metric_hosts = true
|
||||
node_type = "container"
|
||||
scope = ["account:${var.team_name}_${var.env}"]
|
||||
title = "Kubernetes Pods"
|
||||
|
||||
request {
|
||||
fill {
|
||||
q = "avg:process.stat.container.cpu.total_pct{image_name:${var.image_name}} by {host}"
|
||||
}
|
||||
}
|
||||
|
||||
style {
|
||||
palette = "hostmap_blues"
|
||||
palette_flip = false
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
widget {
|
||||
timeseries_definition {
|
||||
show_legend = false
|
||||
title = "CPU Utilization"
|
||||
|
||||
request {
|
||||
display_type = "line"
|
||||
q = "top(avg:docker.cpu.usage{image_name:${var.image_name}} by {docker_image,container_id}, 10, 'mean', 'desc')"
|
||||
|
||||
style {
|
||||
line_type = "solid"
|
||||
line_width = "normal"
|
||||
palette = "dog_classic"
|
||||
}
|
||||
}
|
||||
|
||||
yaxis {
|
||||
include_zero = true
|
||||
max = "auto"
|
||||
min = "auto"
|
||||
scale = "linear"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
# widget {
|
||||
# alert_graph_definition {
|
||||
# alert_id = datadog_monitor.app_monitor.id
|
||||
# title = "Kubernetes Node CPU"
|
||||
# viz_type = "timeseries"
|
||||
# }
|
||||
# }
|
||||
|
||||
widget {
|
||||
hostmap_definition {
|
||||
no_group_hosts = true
|
||||
no_metric_hosts = true
|
||||
node_type = "host"
|
||||
title = "Kubernetes Nodes"
|
||||
|
||||
request {
|
||||
fill {
|
||||
q = "avg:system.cpu.user{*} by {host}"
|
||||
}
|
||||
}
|
||||
|
||||
style {
|
||||
palette = "hostmap_blues"
|
||||
palette_flip = false
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
widget {
|
||||
timeseries_definition {
|
||||
show_legend = false
|
||||
title = "Memory Utilization"
|
||||
request {
|
||||
display_type = "line"
|
||||
q = "top(avg:docker.mem.in_use{image_name:${var.image_name}} by {container_name}, 10, 'mean', 'desc')"
|
||||
|
||||
style {
|
||||
line_type = "solid"
|
||||
line_width = "normal"
|
||||
palette = "dog_classic"
|
||||
}
|
||||
}
|
||||
yaxis {
|
||||
include_zero = true
|
||||
max = "auto"
|
||||
min = "auto"
|
||||
scale = "linear"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
0
outputs.tf
Executable file
0
outputs.tf
Executable file
68
variables.tf
Executable file
68
variables.tf
Executable file
@ -0,0 +1,68 @@
|
||||
variable "opco_name" {
|
||||
description = "Name of the OPCO"
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "app_name" {
|
||||
description = "Name of the application"
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "team_name" {
|
||||
description = "Name of the responsible team"
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "image_name" {
|
||||
description = "Name of the responsible team"
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "aws_region" {
|
||||
description = "Defines the AWS region where the resources are located"
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "env" {
|
||||
description = "Specifies the environment to monitor (dev, tst, stg, prd)"
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "api_key" {
|
||||
description = "Set the Datadog API key"
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "app_key" {
|
||||
description = "Set the Datadog APP key"
|
||||
type = string
|
||||
}
|
||||
|
||||
#variable "api_url" {
|
||||
# description = "Which API to Connect to, we are using the EU one for GDPR compliance"
|
||||
# type = string
|
||||
# default = "https://api.datadoghq.eu"
|
||||
#}
|
||||
|
||||
#variable "http_client_retry_enabled" {
|
||||
# description = "Enables Request retries on HTTP status codes 429 and 5xx"
|
||||
# type = bool
|
||||
# default = true
|
||||
#}
|
||||
|
||||
#variable "http_client_retry_timeout" {
|
||||
# description = "Sets the number of HTTP request retry timeout period"
|
||||
# type = string
|
||||
# default = ""
|
||||
#}
|
||||
|
||||
#variable "validate" {
|
||||
# description = "Validates the provided APP and API keys during provider initialization"
|
||||
# type = bool
|
||||
# default = true
|
||||
#}
|
||||
|
||||
variable "url" {
|
||||
description = "Synthetics URL"
|
||||
type = string
|
||||
}
|
||||
Loading…
x
Reference in New Issue
Block a user