wbyc/terraform-datadog-old-monitors
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

MON-237 - Cosmos RU utilization monitor added

Browse Source
This commit is contained in:
Alexandre Gaillet 2018-07-04 17:42:52 +02:00 committed by Laurent Piroelle
parent 8f62260c1b
commit 14403eddcb
6 changed files with 207 additions and 66 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

15
cloud/azure/README.md
View File

@ -83,14 +83,21 @@ Inputs
| appservices_response_time_threshold_warning | Warning threshold for response time in seconds | string | `5` | no |
| appservices_response_time_time_aggregator | Monitor aggregator for App Services response time [available values: min, max or avg] | string | `min` | no |
| appservices_response_time_timeframe | Monitor timeframe for App Services response time [available values: `last_#m` (1, 5, 10, 15, or 30), `last_#h` (1, 2, or 4), or `last_1d`] | string | `last_5m` | no |
| cosmos_db_4xx_request_rate_threshold_critical | | string | `80` | no |
| cosmos_db_4xx_request_rate_threshold_warning | | string | `50` | no |
| cosmos_db_4xx_request_rate_threshold_critical | Critical threshold for Cosmos DB 4xx requests monitor | string | `80` | no |
| cosmos_db_4xx_request_rate_threshold_warning | Warning threshold for Cosmos DB 4xx requests monitor | string | `50` | no |
| cosmos_db_4xx_requests_message | Custom message for Cosmos DB 4xx requests monitor | string | `` | no |
| cosmos_db_4xx_requests_silenced | Groups to mute for Cosmos DB 4xx requests monitor | map | `<map>` | no |
| cosmos_db_5xx_request_rate_threshold_critical | | string | `80` | no |
| cosmos_db_5xx_request_rate_threshold_warning | | string | `50` | no |
| cosmos_db_5xx_request_rate_threshold_critical | Critical threshold for Cosmos DB 5xx requests monitor | string | `80` | no |
| cosmos_db_5xx_request_rate_threshold_warning | Warning threshold for Cosmos DB 5xx requests monitor | string | `50` | no |
| cosmos_db_5xx_requests_message | Custom message for Cosmos DB 5xx requests monitor | string | `` | no |
| cosmos_db_5xx_requests_silenced | Groups to mute for Cosmos DB 5xx requests monitor | map | `<map>` | no |
| cosmos_db_no_request_message | Custom message for Cosmos DB no request monitor | string | `` | no |
| cosmos_db_no_request_silenced | Groups to mute for Cosmos DB no request monitor | map | `<map>` | no |
| cosmos_db_ru_utilization_collection | Group to associate Cosmos DB collection to RU max | map | - | yes |
| cosmos_db_ru_utilization_message | Custom message for Cosmos DB collection RU utilization monitor | string | `` | no |
| cosmos_db_ru_utilization_rate_threshold_critical | Critical threshold for Cosmos DB collection RU utilization monitor | string | `90` | no |
| cosmos_db_ru_utilization_rate_threshold_warning | Warning threshold for Cosmos DB collection RU utilization monitor | string | `80` | no |
| cosmos_db_ru_utilization_silenced | Groups to mute for Cosmos DB collection RU utilization monitor | map | `<map>` | no |
| delay | Delay in seconds for the metric evaluation | string | `900` | no |
| environment | Architecture environment | string | - | yes |
| eventhub_errors_rate_message | Custom message for Event Hub errors monitor | string | `` | no |

30
cloud/azure/cosmosdb/README.md
View File

@ -13,6 +13,22 @@ module "datadog-monitors-azure-cosmosdb" {
}
```
How to define cosmos_db_ru_utilization_collection variable
----------------------------------------------------------
At the time this module is defined, we can't define Cosmos DB collection with Terraform, so we have to define a variable making the connection between collections and RU max
```
variable cosmos_db_ru_utilization_collection {
type = "map"
default = {
"collection_1" = "ru_max_1"
"collection_2" = "ru_max_2"
...
}
}
```
Purpose
-------
Creates a DataDog monitors with the following checks :
@ -20,22 +36,28 @@ Creates a DataDog monitors with the following checks :
* No request
* Too many 4xx requests
* Too many 5xx requests
* Collection RU utilization
Inputs
------
| Name | Description | Type | Default | Required |
|------|-------------|:----:|:-----:|:-----:|
| cosmos_db_4xx_request_rate_threshold_critical | | string | `80` | no |
| cosmos_db_4xx_request_rate_threshold_warning | | string | `50` | no |
| cosmos_db_4xx_request_rate_threshold_critical | Critical threshold for Cosmos DB 4xx requests monitor | string | `80` | no |
| cosmos_db_4xx_request_rate_threshold_warning | Warning threshold for Cosmos DB 4xx requests monitor | string | `50` | no |
| cosmos_db_4xx_requests_message | Custom message for Cosmos DB 4xx requests monitor | string | `` | no |
| cosmos_db_4xx_requests_silenced | Groups to mute for Cosmos DB 4xx requests monitor | map | `<map>` | no |
| cosmos_db_5xx_request_rate_threshold_critical | | string | `80` | no |
| cosmos_db_5xx_request_rate_threshold_warning | | string | `50` | no |
| cosmos_db_5xx_request_rate_threshold_critical | Critical threshold for Cosmos DB 5xx requests monitor | string | `80` | no |
| cosmos_db_5xx_request_rate_threshold_warning | Warning threshold for Cosmos DB 5xx requests monitor | string | `50` | no |
| cosmos_db_5xx_requests_message | Custom message for Cosmos DB 5xx requests monitor | string | `` | no |
| cosmos_db_5xx_requests_silenced | Groups to mute for Cosmos DB 5xx requests monitor | map | `<map>` | no |
| cosmos_db_no_request_message | Custom message for Cosmos DB no request monitor | string | `` | no |
| cosmos_db_no_request_silenced | Groups to mute for Cosmos DB no request monitor | map | `<map>` | no |
| cosmos_db_ru_utilization_collection | Group to associate Cosmos DB collection to RU max | map | - | yes |
| cosmos_db_ru_utilization_message | Custom message for Cosmos DB collection RU utilization monitor | string | `` | no |
| cosmos_db_ru_utilization_rate_threshold_critical | Critical threshold for Cosmos DB collection RU utilization monitor | string | `90` | no |
| cosmos_db_ru_utilization_rate_threshold_warning | Warning threshold for Cosmos DB collection RU utilization monitor | string | `80` | no |
| cosmos_db_ru_utilization_silenced | Groups to mute for Cosmos DB collection RU utilization monitor | map | `<map>` | no |
| delay | Delay in seconds for the metric evaluation | string | `900` | no |
| environment | Architecture environment | string | - | yes |
| filter_tags_custom | Tags used for custom filtering when filter_tags_use_defaults is false | string | `*` | no |

31
cloud/azure/cosmosdb/inputs.tf
View File

@ -41,10 +41,12 @@ variable "cosmos_db_4xx_requests_silenced" {
}
variable "cosmos_db_4xx_request_rate_threshold_critical" {
description = "Critical threshold for Cosmos DB 4xx requests monitor"
default = 80
}
variable "cosmos_db_4xx_request_rate_threshold_warning" {
description = "Warning threshold for Cosmos DB 4xx requests monitor"
default = 50
}
@ -61,10 +63,12 @@ variable "cosmos_db_5xx_requests_silenced" {
}
variable "cosmos_db_5xx_request_rate_threshold_critical" {
description = "Critical threshold for Cosmos DB 5xx requests monitor"
default = 80
}
variable "cosmos_db_5xx_request_rate_threshold_warning" {
description = "Warning threshold for Cosmos DB 5xx requests monitor"
default = 50
}
@ -79,3 +83,30 @@ variable "cosmos_db_no_request_silenced" {
type = "map"
default = {}
}
variable "cosmos_db_ru_utilization_message" {
description = "Custom message for Cosmos DB collection RU utilization monitor"
type = "string"
default = ""
}
variable "cosmos_db_ru_utilization_silenced" {
description = "Groups to mute for Cosmos DB collection RU utilization monitor"
type = "map"
default = {}
}
variable "cosmos_db_ru_utilization_rate_threshold_critical" {
description = "Critical threshold for Cosmos DB collection RU utilization monitor"
default = 90
}
variable "cosmos_db_ru_utilization_rate_threshold_warning" {
description = "Warning threshold for Cosmos DB collection RU utilization monitor"
default = 80
}
variable "cosmos_db_ru_utilization_collection" {
description = "Group to associate Cosmos DB collection to RU max"
type = "map"
}

42
cloud/azure/cosmosdb/monitors-cosmosdb.tf
View File

@ -7,7 +7,7 @@ data "template_file" "filter" {
}
resource "datadog_monitor" "cosmos_db_4xx_requests" {
name = "[${var.environment}] Cosmos DB 4xx requests rate is too low {{#is_alert}}{{comparator}} {{threshold}}% ({{value}}%){{/is_alert}}{{#is_warning}}{{comparator}} {{warn_threshold}}% ({{value}}%){{/is_warning}}"
name = "[${var.environment}] Cosmos DB 4xx requests rate is high {{#is_alert}}{{comparator}} {{threshold}}% ({{value}}%){{/is_alert}}{{#is_warning}}{{comparator}} {{warn_threshold}}% ({{value}}%){{/is_warning}}"
message = "${coalesce(var.cosmos_db_4xx_requests_message, var.message)}"
query = <<EOF
@ -45,13 +45,12 @@ resource "datadog_monitor" "cosmos_db_4xx_requests" {
locked = false
require_full_window = true
new_host_delay = "${var.delay}"
no_data_timeframe = 20
tags = ["env:${var.environment}", "resource:cosmos_db", "team:azure", "provider:azure"]
}
resource "datadog_monitor" "cosmos_db_5xx_requests" {
name = "[${var.environment}] Cosmos DB 5xx requests rate is too low {{#is_alert}}{{comparator}} {{threshold}}% ({{value}}%){{/is_alert}}{{#is_warning}}{{comparator}} {{warn_threshold}}% ({{value}}%){{/is_warning}}"
name = "[${var.environment}] Cosmos DB 5xx requests rate is high {{#is_alert}}{{comparator}} {{threshold}}% ({{value}}%){{/is_alert}}{{#is_warning}}{{comparator}} {{warn_threshold}}% ({{value}}%){{/is_warning}}"
message = "${coalesce(var.cosmos_db_5xx_requests_message, var.message)}"
query = <<EOF
@ -81,7 +80,6 @@ resource "datadog_monitor" "cosmos_db_5xx_requests" {
locked = false
require_full_window = true
new_host_delay = "${var.delay}"
no_data_timeframe = 20
tags = ["env:${var.environment}", "resource:cosmos_db", "team:azure", "provider:azure"]
}
@ -109,7 +107,41 @@ resource "datadog_monitor" "cosmos_db_success_no_data" {
locked = false
require_full_window = true
new_host_delay = "${var.delay}"
no_data_timeframe = 20
tags = ["env:${var.environment}", "resource:cosmos_db", "team:azure", "provider:azure"]
}
resource "datadog_monitor" "cosmos_db_ru_utilization" {
count = "${length(var.cosmos_db_ru_utilization_collection)}"
name = "[${var.environment}] Cosmos DB collection ${element(keys(var.cosmos_db_ru_utilization_collection),count.index)} RU utilization is high {{#is_alert}}{{comparator}} {{threshold}}% ({{value}}%){{/is_alert}}{{#is_warning}}{{comparator}} {{warn_threshold}}% ({{value}}%){{/is_warning}}"
message = "${coalesce(var.cosmos_db_ru_utilization_message, var.message)}"
query = <<EOF
avg(last_5m): (
avg:azure.cosmosdb.total_request_units{${data.template_file.filter.rendered},collectionname:${element(keys(var.cosmos_db_ru_utilization_collection),count.index)}} by {resource_group,name} /
${element(values(var.cosmos_db_ru_utilization_collection),count.index)}
) * 100 > ${var.cosmos_db_ru_utilization_rate_threshold_critical}
EOF
type = "metric alert"
thresholds {
critical = "${var.cosmos_db_ru_utilization_rate_threshold_critical}"
warning = "${var.cosmos_db_ru_utilization_rate_threshold_warning}"
}
silenced = "${var.cosmos_db_ru_utilization_silenced}"
notify_no_data = false
evaluation_delay = "${var.delay}"
renotify_interval = 0
notify_audit = false
timeout_h = 0
include_tags = true
locked = false
require_full_window = true
new_host_delay = "${var.delay}"
tags = ["env:${var.environment}", "resource:cosmos_db", "collection:${element(keys(var.cosmos_db_ru_utilization_collection),count.index)}", "team:azure", "provider:azure"]
}

43
cloud/azure/inputs.tf
View File

@ -1532,10 +1532,12 @@ variable "cosmos_db_4xx_requests_silenced" {
}
variable "cosmos_db_4xx_request_rate_threshold_critical" {
description = "Critical threshold for Cosmos DB 4xx requests monitor"
default = 80
}
variable "cosmos_db_4xx_request_rate_threshold_warning" {
description = "Warning threshold for Cosmos DB 4xx requests monitor"
default = 50
}
@ -1552,9 +1554,50 @@ variable "cosmos_db_5xx_requests_silenced" {
}
variable "cosmos_db_5xx_request_rate_threshold_critical" {
description = "Critical threshold for Cosmos DB 5xx requests monitor"
default = 80
}
variable "cosmos_db_5xx_request_rate_threshold_warning" {
description = "Warning threshold for Cosmos DB 5xx requests monitor"
default = 50
}
variable "cosmos_db_no_request_message" {
description = "Custom message for Cosmos DB no request monitor"
type = "string"
default = ""
}
variable "cosmos_db_no_request_silenced" {
description = "Groups to mute for Cosmos DB no request monitor"
type = "map"
default = {}
}
variable "cosmos_db_ru_utilization_message" {
description = "Custom message for Cosmos DB collection RU utilization monitor"
type = "string"
default = ""
}
variable "cosmos_db_ru_utilization_silenced" {
description = "Groups to mute for Cosmos DB collection RU utilization monitor"
type = "map"
default = {}
}
variable "cosmos_db_ru_utilization_rate_threshold_critical" {
description = "Critical threshold for Cosmos DB collection RU utilization monitor"
default = 90
}
variable "cosmos_db_ru_utilization_rate_threshold_warning" {
description = "Warning threshold for Cosmos DB collection RU utilization monitor"
default = 80
}
variable "cosmos_db_ru_utilization_collection" {
description = "Group to associate Cosmos DB collection to RU max"
type = "map"
}

6
cloud/azure/monitors.tf
View File

@ -390,4 +390,10 @@ module "cosmosdb" {
cosmos_db_5xx_request_rate_threshold_warning = "${var.cosmos_db_5xx_request_rate_threshold_warning}"
cosmos_db_5xx_requests_message = "${var.cosmos_db_5xx_requests_message}"
cosmos_db_5xx_requests_silenced = "${var.cosmos_db_5xx_requests_silenced}"
cosmos_db_ru_utilization_rate_threshold_critical = "${var.cosmos_db_ru_utilization_rate_threshold_critical}"
cosmos_db_ru_utilization_rate_threshold_warning = "${var.cosmos_db_ru_utilization_rate_threshold_warning}"
cosmos_db_ru_utilization_message = "${var.cosmos_db_ru_utilization_message}"
cosmos_db_ru_utilization_silenced = "${var.cosmos_db_ru_utilization_silenced}"
cosmos_db_ru_utilization_collection = "${var.cosmos_db_ru_utilization_collection}"
}