MON-310 - Azure Postgre monitors added

README.md

@@ -101,6 +101,7 @@ The `//` is very important, it's a terraform specific syntax used to separate gi
 		- [eventhub](https://bitbucket.org/morea/terraform.feature.datadog/src/master/cloud/azure/eventhub/)
 		- [iothubs](https://bitbucket.org/morea/terraform.feature.datadog/src/master/cloud/azure/iothubs/)
 		- [keyvault](https://bitbucket.org/morea/terraform.feature.datadog/src/master/cloud/azure/keyvault/)
+		- [postgresql](https://bitbucket.org/morea/terraform.feature.datadog/src/master/cloud/azure/postgresql/)
 		- [redis](https://bitbucket.org/morea/terraform.feature.datadog/src/master/cloud/azure/redis/)
 		- [servicebus](https://bitbucket.org/morea/terraform.feature.datadog/src/master/cloud/azure/servicebus/)
 		- [sql-database](https://bitbucket.org/morea/terraform.feature.datadog/src/master/cloud/azure/sql-database/)

cloud/azure/postgresql/README.md

@@ -0,0 +1,85 @@
+# CLOUD AZURE POSTGRESQL DataDog monitors
+
+## How to use this module
+
+```
+module "datadog-monitors-cloud-azure-postgresql" {
+  source = "git::ssh://git@bitbucket.org/morea/terraform.feature.datadog.git//cloud/azure/postgresql?ref={revision}"
+
+  environment = "${var.environment}"
+  message     = "${module.datadog-message-alerting.alerting-message}"
+}
+
+```
+
+## Purpose
+
+Creates DataDog monitors with the following checks:
+
+- Postgresql Server compute consumption is high
+- Postgresql Server CPU usage is high
+- Postgresql Server has no connection
+- Postgresql Server IO consumption is high
+- Postgresql Server memory usage is high
+- Postgresql Server storage is running low
+
+## Inputs
+
+| Name | Description | Type | Default | Required |
+|------|-------------|:----:|:-----:|:-----:|
+| compute_consumption_message | Custom message for Postgresql compute consumption monitor | string | `` | no |
+| compute_consumption_silenced | Groups to mute for Postgresql compute consumption monitor | map | `{}` | no |
+| compute_consumption_threshold_critical | Postgresql compute consumption in percent (critical threshold) | string | `90` | no |
+| compute_consumption_threshold_warning | Postgresql compute consumption in percent (warning threshold) | string | `80` | no |
+| compute_consumption_time_aggregator | Monitor aggregator for Postgresql compute consumption [available values: min, max or avg] | string | `min` | no |
+| compute_consumption_timeframe | Monitor timeframe for Postgresql compute consumption [available values: `last_#m` (1, 5, 10, 15, or 30), `last_#h` (1, 2, or 4), or `last_1d`] | string | `last_15m` | no |
+| cpu_usage_message | Custom message for Postgresql CPU monitor | string | `` | no |
+| cpu_usage_silenced | Groups to mute for Postgresql CPU monitor | map | `{}` | no |
+| cpu_usage_threshold_critical | Postgresql CPU usage in percent (critical threshold) | string | `90` | no |
+| cpu_usage_threshold_warning | Postgresql CPU usage in percent (warning threshold) | string | `80` | no |
+| cpu_usage_time_aggregator | Monitor aggregator for Postgresql CPU [available values: min, max or avg] | string | `min` | no |
+| cpu_usage_timeframe | Monitor timeframe for Postgresql CPU [available values: `last_#m` (1, 5, 10, 15, or 30), `last_#h` (1, 2, or 4), or `last_1d`] | string | `last_15m` | no |
+| delay | Delay in seconds for the metric evaluation | string | `900` | no |
+| environment | Architecture environment | string | - | yes |
+| filter_tags_custom | Tags used for custom filtering when filter_tags_use_defaults is false | string | `*` | no |
+| filter_tags_use_defaults | Use default filter tags convention | string | `true` | no |
+| free_storage_message | Custom message for Postgresql Free Storage monitor | string | `` | no |
+| free_storage_silenced | Groups to mute for Postgresql Free Storage monitor | map | `{}` | no |
+| free_storage_threshold_critical | Postgresql Free Storage remaining in percent (critical threshold) | string | `10` | no |
+| free_storage_threshold_warning | Postgresql Free Storage remaining in percent (warning threshold) | string | `20` | no |
+| free_storage_time_aggregator | Monitor aggregator for Postgresql Free Storage [available values: min, max or avg] | string | `min` | no |
+| free_storage_timeframe | Monitor timeframe for Postgresql Free Storage [available values: `last_#m` (1, 5, 10, 15, or 30), `last_#h` (1, 2, or 4), or `last_1d`] | string | `last_15m` | no |
+| io_consumption_message | Custom message for Postgresql IO consumption monitor | string | `` | no |
+| io_consumption_silenced | Groups to mute for Postgresql IO consumption monitor | map | `{}` | no |
+| io_consumption_threshold_critical | Postgresql IO consumption in percent (critical threshold) | string | `90` | no |
+| io_consumption_threshold_warning | Postgresql IO consumption in percent (warning threshold) | string | `80` | no |
+| io_consumption_time_aggregator | Monitor aggregator for Postgresql IO consumption [available values: min, max or avg] | string | `min` | no |
+| io_consumption_timeframe | Monitor timeframe for Postgresql IO consumption [available values: `last_#m` (1, 5, 10, 15, or 30), `last_#h` (1, 2, or 4), or `last_1d`] | string | `last_15m` | no |
+| memory_usage_message | Custom message for Postgresql memory monitor | string | `` | no |
+| memory_usage_silenced | Groups to mute for Postgresql memory monitor | map | `{}` | no |
+| memory_usage_threshold_critical | Postgresql memory usage in percent (critical threshold) | string | `90` | no |
+| memory_usage_threshold_warning | Postgresql memory usage in percent (warning threshold) | string | `80` | no |
+| memory_usage_time_aggregator | Monitor aggregator for Postgresql memory [available values: min, max or avg] | string | `min` | no |
+| memory_usage_timeframe | Monitor timeframe for Postgresql memory [available values: `last_#m` (1, 5, 10, 15, or 30), `last_#h` (1, 2, or 4), or `last_1d`] | string | `last_15m` | no |
+| message | Message sent when a Redis monitor is triggered | string | - | yes |
+| no_connection_message | Custom message for Postgresql no connection monitor | string | `` | no |
+| no_connection_silenced | Groups to mute for Postgresql no connection monitor | map | `{}` | no |
+| no_connection_time_aggregator | Monitor aggregator for Postgresql no connection [available values: min, max or avg] | string | `min` | no |
+| no_connection_timeframe | Monitor timeframe for Postgresql no connection [available values: `last_#m` (1, 5, 10, 15, or 30), `last_#h` (1, 2, or 4), or `last_1d`] | string | `last_5m` | no |
+
+## Outputs
+
+| Name | Description |
+|------|-------------|
+| postgresql_compute_consumption_id | id for monitor postgresql_compute_consumption |
+| postgresql_cpu_usage_id | id for monitor postgresql_cpu_usage |
+| postgresql_free_storage_id | id for monitor postgresql_free_storage |
+| postgresql_io_consumption_id | id for monitor postgresql_io_consumption |
+| postgresql_memory_usage_id | id for monitor postgresql_memory_usage |
+| postgresql_no_connection_id | id for monitor postgresql_no_connection |
+
+Related documentation
+---------------------
+
+DataDog documentation: [https://docs.datadoghq.com/integrations/azure/](https://docs.datadoghq.com/integrations/azure/)
+You have to search `mysql`

cloud/azure/postgresql/inputs.tf

@@ -0,0 +1,220 @@
+# Global Terraform
+variable "environment" {
+  description = "Architecture environment"
+  type        = "string"
+}
+
+# Global DataDog
+variable "message" {
+  description = "Message sent when a Redis monitor is triggered"
+}
+
+variable "delay" {
+  description = "Delay in seconds for the metric evaluation"
+  default     = 900
+}
+
+variable "filter_tags_use_defaults" {
+  description = "Use default filter tags convention"
+  default     = "true"
+}
+
+variable "filter_tags_custom" {
+  description = "Tags used for custom filtering when filter_tags_use_defaults is false"
+  default     = "*"
+}
+
+# Azure Databases for Postgresql Servers specific variables
+variable "cpu_usage_silenced" {
+  description = "Groups to mute for Postgresql CPU monitor"
+  type        = "map"
+  default     = {}
+}
+
+variable "cpu_usage_message" {
+  description = "Custom message for Postgresql CPU monitor"
+  type        = "string"
+  default     = ""
+}
+
+variable "cpu_usage_time_aggregator" {
+  description = "Monitor aggregator for Postgresql CPU [available values: min, max or avg]"
+  type        = "string"
+  default     = "min"
+}
+
+variable "cpu_usage_timeframe" {
+  description = "Monitor timeframe for Postgresql CPU [available values: `last_#m` (1, 5, 10, 15, or 30), `last_#h` (1, 2, or 4), or `last_1d`]"
+  type        = "string"
+  default     = "last_15m"
+}
+
+variable "cpu_usage_threshold_warning" {
+  description = "Postgresql CPU usage in percent (warning threshold)"
+  default     = "80"
+}
+
+variable "cpu_usage_threshold_critical" {
+  description = "Postgresql CPU usage in percent (critical threshold)"
+  default     = "90"
+}
+
+variable "no_connection_silenced" {
+  description = "Groups to mute for Postgresql no connection monitor"
+  type        = "map"
+  default     = {}
+}
+
+variable "no_connection_message" {
+  description = "Custom message for Postgresql no connection monitor"
+  type        = "string"
+  default     = ""
+}
+
+variable "no_connection_time_aggregator" {
+  description = "Monitor aggregator for Postgresql no connection [available values: min, max or avg]"
+  type        = "string"
+  default     = "min"
+}
+
+variable "no_connection_timeframe" {
+  description = "Monitor timeframe for Postgresql no connection [available values: `last_#m` (1, 5, 10, 15, or 30), `last_#h` (1, 2, or 4), or `last_1d`]"
+  type        = "string"
+  default     = "last_5m"
+}
+
+variable "free_storage_silenced" {
+  description = "Groups to mute for Postgresql Free Storage monitor"
+  type        = "map"
+  default     = {}
+}
+
+variable "free_storage_message" {
+  description = "Custom message for Postgresql Free Storage monitor"
+  type        = "string"
+  default     = ""
+}
+
+variable "free_storage_time_aggregator" {
+  description = "Monitor aggregator for Postgresql Free Storage [available values: min, max or avg]"
+  type        = "string"
+  default     = "min"
+}
+
+variable "free_storage_timeframe" {
+  description = "Monitor timeframe for Postgresql Free Storage [available values: `last_#m` (1, 5, 10, 15, or 30), `last_#h` (1, 2, or 4), or `last_1d`]"
+  type        = "string"
+  default     = "last_15m"
+}
+
+variable "free_storage_threshold_warning" {
+  description = "Postgresql Free Storage remaining in percent (warning threshold)"
+  default     = "20"
+}
+
+variable "free_storage_threshold_critical" {
+  description = "Postgresql Free Storage remaining in percent (critical threshold)"
+  default     = "10"
+}
+
+variable "io_consumption_silenced" {
+  description = "Groups to mute for Postgresql IO consumption monitor"
+  type        = "map"
+  default     = {}
+}
+
+variable "io_consumption_message" {
+  description = "Custom message for Postgresql IO consumption monitor"
+  type        = "string"
+  default     = ""
+}
+
+variable "io_consumption_time_aggregator" {
+  description = "Monitor aggregator for Postgresql IO consumption [available values: min, max or avg]"
+  type        = "string"
+  default     = "min"
+}
+
+variable "io_consumption_timeframe" {
+  description = "Monitor timeframe for Postgresql IO consumption [available values: `last_#m` (1, 5, 10, 15, or 30), `last_#h` (1, 2, or 4), or `last_1d`]"
+  type        = "string"
+  default     = "last_15m"
+}
+
+variable "io_consumption_threshold_warning" {
+  description = "Postgresql IO consumption in percent (warning threshold)"
+  default     = "80"
+}
+
+variable "io_consumption_threshold_critical" {
+  description = "Postgresql IO consumption in percent (critical threshold)"
+  default     = "90"
+}
+
+variable "memory_usage_silenced" {
+  description = "Groups to mute for Postgresql memory monitor"
+  type        = "map"
+  default     = {}
+}
+
+variable "memory_usage_message" {
+  description = "Custom message for Postgresql memory monitor"
+  type        = "string"
+  default     = ""
+}
+
+variable "memory_usage_time_aggregator" {
+  description = "Monitor aggregator for Postgresql memory [available values: min, max or avg]"
+  type        = "string"
+  default     = "min"
+}
+
+variable "memory_usage_timeframe" {
+  description = "Monitor timeframe for Postgresql memory [available values: `last_#m` (1, 5, 10, 15, or 30), `last_#h` (1, 2, or 4), or `last_1d`]"
+  type        = "string"
+  default     = "last_15m"
+}
+
+variable "memory_usage_threshold_warning" {
+  description = "Postgresql memory usage in percent (warning threshold)"
+  default     = "80"
+}
+
+variable "memory_usage_threshold_critical" {
+  description = "Postgresql memory usage in percent (critical threshold)"
+  default     = "90"
+}
+
+variable "compute_consumption_silenced" {
+  description = "Groups to mute for Postgresql compute consumption monitor"
+  type        = "map"
+  default     = {}
+}
+
+variable "compute_consumption_message" {
+  description = "Custom message for Postgresql compute consumption monitor"
+  type        = "string"
+  default     = ""
+}
+
+variable "compute_consumption_time_aggregator" {
+  description = "Monitor aggregator for Postgresql compute consumption [available values: min, max or avg]"
+  type        = "string"
+  default     = "min"
+}
+
+variable "compute_consumption_timeframe" {
+  description = "Monitor timeframe for Postgresql compute consumption [available values: `last_#m` (1, 5, 10, 15, or 30), `last_#h` (1, 2, or 4), or `last_1d`]"
+  type        = "string"
+  default     = "last_15m"
+}
+
+variable "compute_consumption_threshold_warning" {
+  description = "Postgresql compute consumption in percent (warning threshold)"
+  default     = "80"
+}
+
+variable "compute_consumption_threshold_critical" {
+  description = "Postgresql compute consumption in percent (critical threshold)"
+  default     = "90"
+}

cloud/azure/postgresql/modules.tf

@@ -0,0 +1,8 @@
+module "filter-tags" {
+  source = "../../../common/filter-tags"
+
+  environment              = "${var.environment}"
+  resource                 = "postgresql"
+  filter_tags_use_defaults = "${var.filter_tags_use_defaults}"
+  filter_tags_custom       = "${var.filter_tags_custom}"
+}

cloud/azure/postgresql/monitors-postegresql.tf

@@ -0,0 +1,194 @@
+data "template_file" "filter" {
+  template = "$${filter}"
+
+  vars {
+    filter = "${var.filter_tags_use_defaults == "true" ? format("dd_monitoring:enabled,dd_azure_postgresql:enabled,env:%s", var.environment) : "${var.filter_tags_custom}"}"
+  }
+}
+
+resource "datadog_monitor" "postgresql_cpu_usage" {
+  name    = "[${var.environment}] Postgresql Server CPU usage is high {{#is_alert}}{{{comparator}}} {{threshold}}% ({{value}}%){{/is_alert}}{{#is_warning}}{{{comparator}}} {{warn_threshold}}% ({{value}}%){{/is_warning}}"
+  message = "${coalesce(var.cpu_usage_message, var.message)}"
+
+  query = <<EOF
+    ${var.cpu_usage_time_aggregator}(${var.cpu_usage_timeframe}): (
+      avg:azure.dbforpostgresql_servers.cpu_percent{${data.template_file.filter.rendered}} by {resource_group,region,name}
+    ) > ${var.cpu_usage_threshold_critical}
+  EOF
+
+  type = "metric alert"
+
+  thresholds {
+    critical = "${var.cpu_usage_threshold_critical}"
+    warning  = "${var.cpu_usage_threshold_warning}"
+  }
+
+  silenced = "${var.cpu_usage_silenced}"
+
+  notify_no_data      = true
+  evaluation_delay    = "${var.delay}"
+  renotify_interval   = 0
+  notify_audit        = false
+  timeout_h           = 0
+  include_tags        = true
+  locked              = false
+  require_full_window = false
+  new_host_delay      = "${var.delay}"
+
+  tags = ["env:${var.environment}", "resource:postgresql", "team:azure", "provider:azure"]
+}
+
+resource "datadog_monitor" "postgresql_no_connection" {
+  name    = "[${var.environment}] Postgresql Server has no connection"
+  message = "${coalesce(var.no_connection_message, var.message)}"
+
+  query = <<EOF
+    ${var.no_connection_time_aggregator}(${var.no_connection_timeframe}): (
+      avg:azure.dbforpostgresql_servers.active_connections{${data.template_file.filter.rendered}} by {resource_group,region,name}
+    ) < 1
+  EOF
+
+  type = "metric alert"
+
+  silenced = "${var.no_connection_silenced}"
+
+  notify_no_data      = true
+  evaluation_delay    = "${var.delay}"
+  renotify_interval   = 0
+  notify_audit        = false
+  timeout_h           = 0
+  include_tags        = true
+  locked              = false
+  require_full_window = false
+  new_host_delay      = "${var.delay}"
+
+  tags = ["env:${var.environment}", "resource:postgresql", "team:azure", "provider:azure"]
+}
+
+resource "datadog_monitor" "postgresql_free_storage" {
+  name    = "[${var.environment}] Postgresql Server storage is running low {{#is_alert}}{{{comparator}}} {{threshold}}% ({{value}}%){{/is_alert}}{{#is_warning}}{{{comparator}}} {{warn_threshold}}% ({{value}}%){{/is_warning}}"
+  message = "${coalesce(var.free_storage_message, var.message)}"
+
+  query = <<EOF
+    ${var.free_storage_time_aggregator}(${var.free_storage_timeframe}): (
+      100 - avg:azure.dbforpostgresql_servers.storage_percent{${data.template_file.filter.rendered}} by {resource_group,region,name}
+    ) < ${var.free_storage_threshold_critical}
+  EOF
+
+  type = "metric alert"
+
+  thresholds {
+    critical = "${var.free_storage_threshold_critical}"
+    warning  = "${var.free_storage_threshold_warning}"
+  }
+
+  silenced = "${var.free_storage_silenced}"
+
+  notify_no_data      = true
+  evaluation_delay    = "${var.delay}"
+  renotify_interval   = 0
+  notify_audit        = false
+  timeout_h           = 0
+  include_tags        = true
+  locked              = false
+  require_full_window = false
+  new_host_delay      = "${var.delay}"
+
+  tags = ["env:${var.environment}", "resource:postgresql", "team:azure", "provider:azure"]
+}
+
+resource "datadog_monitor" "postgresql_io_consumption" {
+  name    = "[${var.environment}] Postgresql Server IO consumption is high {{#is_alert}}{{{comparator}}} {{threshold}}% ({{value}}%){{/is_alert}}{{#is_warning}}{{{comparator}}} {{warn_threshold}}% ({{value}}%){{/is_warning}}"
+  message = "${coalesce(var.io_consumption_message, var.message)}"
+
+  query = <<EOF
+    ${var.io_consumption_time_aggregator}(${var.io_consumption_timeframe}): (
+      avg:azure.dbforpostgresql_servers.io_consumption_percent{${data.template_file.filter.rendered}} by {resource_group,region,name}
+    ) > ${var.io_consumption_threshold_critical}
+  EOF
+
+  type = "metric alert"
+
+  thresholds {
+    critical = "${var.io_consumption_threshold_critical}"
+    warning  = "${var.io_consumption_threshold_warning}"
+  }
+
+  silenced = "${var.io_consumption_silenced}"
+
+  notify_no_data      = true
+  evaluation_delay    = "${var.delay}"
+  renotify_interval   = 0
+  notify_audit        = false
+  timeout_h           = 0
+  include_tags        = true
+  locked              = false
+  require_full_window = false
+  new_host_delay      = "${var.delay}"
+
+  tags = ["env:${var.environment}", "resource:postgresql", "team:azure", "provider:azure"]
+}
+
+resource "datadog_monitor" "postgresql_memory_usage" {
+  name    = "[${var.environment}] Postgresql Server memory usage is high {{#is_alert}}{{{comparator}}} {{threshold}}% ({{value}}%){{/is_alert}}{{#is_warning}}{{{comparator}}} {{warn_threshold}}% ({{value}}%){{/is_warning}}"
+  message = "${coalesce(var.memory_usage_message, var.message)}"
+
+  query = <<EOF
+    ${var.memory_usage_time_aggregator}(${var.memory_usage_timeframe}): (
+      avg:azure.dbforpostgresql_servers.memory_percent{${data.template_file.filter.rendered}} by {resource_group,region,name}
+    ) > ${var.memory_usage_threshold_critical}
+  EOF
+
+  type = "metric alert"
+
+  thresholds {
+    critical = "${var.memory_usage_threshold_critical}"
+    warning  = "${var.memory_usage_threshold_warning}"
+  }
+
+  silenced = "${var.memory_usage_silenced}"
+
+  notify_no_data      = true
+  evaluation_delay    = "${var.delay}"
+  renotify_interval   = 0
+  notify_audit        = false
+  timeout_h           = 0
+  include_tags        = true
+  locked              = false
+  require_full_window = false
+  new_host_delay      = "${var.delay}"
+
+  tags = ["env:${var.environment}", "resource:postgresql", "team:azure", "provider:azure"]
+}
+
+resource "datadog_monitor" "postgresql_compute_consumption" {
+  name    = "[${var.environment}] Postgresql Server compute consumption is high {{#is_alert}}{{{comparator}}} {{threshold}}% ({{value}}%){{/is_alert}}{{#is_warning}}{{{comparator}}} {{warn_threshold}}% ({{value}}%){{/is_warning}}"
+  message = "${coalesce(var.compute_consumption_message, var.message)}"
+
+  query = <<EOF
+    ${var.compute_consumption_time_aggregator}(${var.compute_consumption_timeframe}): (
+      avg:azure.dbforpostgresql_servers.compute_consumption_percent{${data.template_file.filter.rendered}} by {resource_group,region,name}
+    ) > ${var.compute_consumption_threshold_critical}
+  EOF
+
+  type = "metric alert"
+
+  thresholds {
+    critical = "${var.compute_consumption_threshold_critical}"
+    warning  = "${var.compute_consumption_threshold_warning}"
+  }
+
+  silenced = "${var.compute_consumption_silenced}"
+
+  notify_no_data      = true
+  evaluation_delay    = "${var.delay}"
+  renotify_interval   = 0
+  notify_audit        = false
+  timeout_h           = 0
+  include_tags        = true
+  locked              = false
+  require_full_window = false
+  new_host_delay      = "${var.delay}"
+
+  tags = ["env:${var.environment}", "resource:postgresql", "team:azure", "provider:azure"]
+}

cloud/azure/postgresql/outputs.tf

@@ -0,0 +1,29 @@
+output "postgresql_cpu_usage_id" {
+  description = "id for monitor postgresql_cpu_usage"
+  value       = "${datadog_monitor.postgresql_cpu_usage.*.id}"
+}
+
+output "postgresql_no_connection_id" {
+  description = "id for monitor postgresql_no_connection"
+  value       = "${datadog_monitor.postgresql_no_connection.*.id}"
+}
+
+output "postgresql_free_storage_id" {
+  description = "id for monitor postgresql_free_storage"
+  value       = "${datadog_monitor.postgresql_free_storage.*.id}"
+}
+
+output "postgresql_io_consumption_id" {
+  description = "id for monitor postgresql_io_consumption"
+  value       = "${datadog_monitor.postgresql_io_consumption.*.id}"
+}
+
+output "postgresql_memory_usage_id" {
+  description = "id for monitor postgresql_memory_usage"
+  value       = "${datadog_monitor.postgresql_memory_usage.*.id}"
+}
+
+output "postgresql_compute_consumption_id" {
+  description = "id for monitor postgresql_compute_consumption"
+  value       = "${datadog_monitor.postgresql_compute_consumption.*.id}"
+}