wbyc/terraform-datadog-old-monitors
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merged in MON-136-use-rates-for-azure-iot-hub-moni (pull request #59)

Browse Source 
MON-136 Use rates for Azure IoT Hub monitors

Approved-by: Laurent Piroelle <laurent.piroelle@fr.clara.net>
Approved-by: Jérôme Respaut <shr3ps@gmail.com>
Approved-by: Alexandre Gaillet <alexandre.gaillet@fr.clara.net>
Approved-by: Quentin Manfroi <quentin.manfroi@yahoo.fr>
This commit is contained in:
Laurent Piroelle 2018-03-19 16:40:45 +00:00 committed by Quentin Manfroi
commit 2a4746b52d
8 changed files with 131 additions and 204 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

16
cloud/azure/README.md
View File

@ -84,9 +84,9 @@ Inputs
| filter_tags_custom | Tags used for custom filtering when filter_tags_use_defaults is false | string | `*` | no | | filter_tags_custom | Tags used for custom filtering when filter_tags_use_defaults is false | string | `*` | no |
| filter_tags_use_defaults | Use default filter tags convention | string | `true` | no | | filter_tags_use_defaults | Use default filter tags convention | string | `true` | no |
| iothub_dropped_d2c_telemetry_egress_message | Custom message for IoT Hub dropped d2c telemetry monitor | string | `` | no | | iothub_dropped_d2c_telemetry_egress_message | Custom message for IoT Hub dropped d2c telemetry monitor | string | `` | no |
| iothub_dropped_d2c_telemetry_egress_rate_threshold_critical | D2C Telemetry Dropped limit (critical threshold) | string | `90` | no |
| iothub_dropped_d2c_telemetry_egress_rate_threshold_warning | D2C Telemetry Dropped limit (warning threshold) | string | `50` | no |
| iothub_dropped_d2c_telemetry_egress_silenced | Groups to mute for IoT Hub dropped d2c telemetry monitor | map | `<map>` | no | | iothub_dropped_d2c_telemetry_egress_silenced | Groups to mute for IoT Hub dropped d2c telemetry monitor | map | `<map>` | no |
| iothub_dropped_d2c_telemetry_egress_threshold_critical | D2C Telemetry Dropped limit (critical threshold) | string | `1000` | no |
| iothub_dropped_d2c_telemetry_egress_threshold_warning | D2C Telemetry Dropped limit (warning threshold) | string | `500` | no |
| iothub_failed_c2d_methods_rate_message | Custom message for IoT Hub failed c2d method monitor | string | `` | no | | iothub_failed_c2d_methods_rate_message | Custom message for IoT Hub failed c2d method monitor | string | `` | no |
| iothub_failed_c2d_methods_rate_silenced | Groups to mute for IoT Hub failed c2d methods monitor | map | `<map>` | no | | iothub_failed_c2d_methods_rate_silenced | Groups to mute for IoT Hub failed c2d methods monitor | map | `<map>` | no |
| iothub_failed_c2d_methods_rate_threshold_critical | C2D Methods Failed rate limit (critical threshold) | string | `90` | no | | iothub_failed_c2d_methods_rate_threshold_critical | C2D Methods Failed rate limit (critical threshold) | string | `90` | no |
@ -119,18 +119,14 @@ Inputs
| iothub_failed_queryjobs_rate_silenced | Groups to mute for IoT Hub failed query jobs monitor | map | `<map>` | no | | iothub_failed_queryjobs_rate_silenced | Groups to mute for IoT Hub failed query jobs monitor | map | `<map>` | no |
| iothub_failed_queryjobs_rate_threshold_critical | QueryJobs Failed rate limit (critical threshold) | string | `90` | no | | iothub_failed_queryjobs_rate_threshold_critical | QueryJobs Failed rate limit (critical threshold) | string | `90` | no |
| iothub_failed_queryjobs_rate_threshold_warning | QueryJobs Failed rate limit (warning threshold) | string | `50` | no | | iothub_failed_queryjobs_rate_threshold_warning | QueryJobs Failed rate limit (warning threshold) | string | `50` | no |
| iothub_fallback_d2c_telemetry_egress_message | Custom message for IoT Hub fallback d2c telemetry monitor | string | `` | no |
| iothub_fallback_d2c_telemetry_egress_silenced | Groups to mute for IoT Hub fallback d2c telemetry monitor | map | `<map>` | no |
| iothub_fallback_d2c_telemetry_egress_threshold_critical | D2C Telemetry Fallback limit (critical threshold) | string | `1000` | no |
| iothub_fallback_d2c_telemetry_egress_threshold_warning | D2C Telemetry Fallback limit (warning threshold) | string | `500` | no |
| iothub_invalid_d2c_telemetry_egress_message | Custom message for IoT Hub invalid d2c telemetry monitor | string | `` | no | | iothub_invalid_d2c_telemetry_egress_message | Custom message for IoT Hub invalid d2c telemetry monitor | string | `` | no |
| iothub_invalid_d2c_telemetry_egress_rate_threshold_critical | D2C Telemetry Invalid limit (critical threshold) | string | `90` | no |
| iothub_invalid_d2c_telemetry_egress_rate_threshold_warning | D2C Telemetry Invalid limit (warning threshold) | string | `50` | no |
| iothub_invalid_d2c_telemetry_egress_silenced | Groups to mute for IoT Hub invalid d2c telemetry monitor | map | `<map>` | no | | iothub_invalid_d2c_telemetry_egress_silenced | Groups to mute for IoT Hub invalid d2c telemetry monitor | map | `<map>` | no |
| iothub_invalid_d2c_telemetry_egress_threshold_critical | D2C Telemetry Invalid limit (critical threshold) | string | `1000` | no |
| iothub_invalid_d2c_telemetry_egress_threshold_warning | D2C Telemetry Invalid limit (warning threshold) | string | `500` | no |
| iothub_orphaned_d2c_telemetry_egress_message | Custom message for IoT Hub orphaned d2c telemetry monitor | string | `` | no | | iothub_orphaned_d2c_telemetry_egress_message | Custom message for IoT Hub orphaned d2c telemetry monitor | string | `` | no |
| iothub_orphaned_d2c_telemetry_egress_rate_threshold_critical | D2C Telemetry Orphaned limit (critical threshold) | string | `90` | no |
| iothub_orphaned_d2c_telemetry_egress_rate_threshold_warning | D2C Telemetry Orphaned limit (warning threshold) | string | `50` | no |
| iothub_orphaned_d2c_telemetry_egress_silenced | Groups to mute for IoT Hub orphaned d2c telemetry monitor | map | `<map>` | no | | iothub_orphaned_d2c_telemetry_egress_silenced | Groups to mute for IoT Hub orphaned d2c telemetry monitor | map | `<map>` | no |
| iothub_orphaned_d2c_telemetry_egress_threshold_critical | D2C Telemetry Orphaned limit (critical threshold) | string | `1000` | no |
| iothub_orphaned_d2c_telemetry_egress_threshold_warning | D2C Telemetry Orphaned limit (warning threshold) | string | `500` | no |
| iothub_status_message | Custom message for IoT Hub status monitor | string | `` | no | | iothub_status_message | Custom message for IoT Hub status monitor | string | `` | no |
| iothub_status_silenced | Groups to mute for IoT Hub status monitor | map | `<map>` | no | | iothub_status_silenced | Groups to mute for IoT Hub status monitor | map | `<map>` | no |
| iothub_too_many_d2c_telemetry_ingress_nosent_message | Custom message for IoT Hub unsent d2c telemetry monitor | string | `` | no | | iothub_too_many_d2c_telemetry_ingress_nosent_message | Custom message for IoT Hub unsent d2c telemetry monitor | string | `` | no |

2
cloud/azure/app-services/monitors-app_services.tf
View File

@ -13,7 +13,7 @@ resource "datadog_monitor" "appservices_response_time" {
message = "${coalesce(var.response_time_message, var.message)}" message = "${coalesce(var.response_time_message, var.message)}"
query = <<EOF query = <<EOF
avg(last_5m): ( min(last_5m): (
avg:azure.app_services.average_response_time{${data.template_file.filter.rendered}} by {resource_group,region,name} avg:azure.app_services.average_response_time{${data.template_file.filter.rendered}} by {resource_group,region,name}
) > ${var.response_time_threshold_critical} ) > ${var.response_time_threshold_critical}
EOF EOF

27
cloud/azure/eventhub/monitors-eventhub.tf
View File

@ -38,11 +38,11 @@ resource "datadog_monitor" "eventhub_failed_requests" {
query = <<EOF query = <<EOF
sum(last_5m): ( sum(last_5m): (
avg:azure.eventhub_namespaces.failed_requests{${data.template_file.filter.rendered}} by {resource_group,region,name}.as_count() ) / default(
( avg:azure.eventhub_namespaces.failed_requests{${data.template_file.filter.rendered}} by {resource_group,region,name}.as_count() /
avg:azure.eventhub_namespaces.successful_requests{${data.template_file.filter.rendered}} by {resource_group,region,name}.as_count() + avg:azure.eventhub_namespaces.incoming_requests{${data.template_file.filter.rendered}} by {resource_group,region,name}.as_count(),
avg:azure.eventhub_namespaces.failed_requests{${data.template_file.filter.rendered}} by {resource_group,region,name}.as_count() 0) * 100
) * 100 > ${var.failed_requests_rate_thresold_critical} ) > ${var.failed_requests_rate_thresold_critical}
EOF EOF
type = "metric alert" type = "metric alert"
@ -58,7 +58,7 @@ resource "datadog_monitor" "eventhub_failed_requests" {
evaluation_delay = "${var.delay}" evaluation_delay = "${var.delay}"
renotify_interval = 0 renotify_interval = 0
notify_audit = false notify_audit = false
timeout_h = 1 timeout_h = 0
include_tags = true include_tags = true
locked = false locked = false
require_full_window = false require_full_window = false
@ -69,20 +69,21 @@ resource "datadog_monitor" "eventhub_failed_requests" {
} }
resource "datadog_monitor" "eventhub_errors" { resource "datadog_monitor" "eventhub_errors" {
name = "[${var.environment}] Event Hub too manny errors {{comparator}} {{#is_alert}}{{threshold}}%{{/is_alert}}{{#is_warning}}{{warn_threshold}}%{{/is_warning}} ({{value}}%)" name = "[${var.environment}] Event Hub too many errors {{comparator}} {{#is_alert}}{{threshold}}%{{/is_alert}}{{#is_warning}}{{warn_threshold}}%{{/is_warning}} ({{value}}%)"
message = "${coalesce(var.errors_rate_message, var.message)}" message = "${coalesce(var.errors_rate_message, var.message)}"
query = <<EOF query = <<EOF
sum(last_5m): ( sum(last_5m): (
default(
(
avg:azure.eventhub_namespaces.internal_server_errors{${data.template_file.filter.rendered}} by {resource_group,region,name}.as_count() + avg:azure.eventhub_namespaces.internal_server_errors{${data.template_file.filter.rendered}} by {resource_group,region,name}.as_count() +
avg:azure.eventhub_namespaces.server_busy_errors{${data.template_file.filter.rendered}} by {resource_group,region,name}.as_count() + avg:azure.eventhub_namespaces.server_busy_errors{${data.template_file.filter.rendered}} by {resource_group,region,name}.as_count() +
avg:azure.eventhub_namespaces.other_errors{${data.template_file.filter.rendered}} by {resource_group,region,name}.as_count() avg:azure.eventhub_namespaces.other_errors{${data.template_file.filter.rendered}} by {resource_group,region,name}.as_count()
) / ( ) / (
avg:azure.eventhub_namespaces.successful_requests{${data.template_file.filter.rendered}} by {resource_group,region,name}.as_count() + avg:eventhub_namespaces.incoming_requests{${data.template_file.filter.rendered}} by {resource_group,region,name}.as_count()
avg:azure.eventhub_namespaces.internal_server_errors{${data.template_file.filter.rendered}} by {resource_group,region,name}.as_count() + ),
avg:azure.eventhub_namespaces.server_busy_errors{${data.template_file.filter.rendered}} by {resource_group,region,name}.as_count() + 0) * 100
avg:azure.eventhub_namespaces.other_errors{${data.template_file.filter.rendered}} by {resource_group,region,name}.as_count() ) > ${var.errors_rate_thresold_critical}
) * 100 > ${var.errors_rate_thresold_critical}
EOF EOF
type = "metric alert" type = "metric alert"
@ -98,7 +99,7 @@ resource "datadog_monitor" "eventhub_errors" {
evaluation_delay = "${var.delay}" evaluation_delay = "${var.delay}"
renotify_interval = 0 renotify_interval = 0
notify_audit = false notify_audit = false
timeout_h = 1 timeout_h = 0
include_tags = true include_tags = true
locked = false locked = false
require_full_window = false require_full_window = false

46
cloud/azure/inputs.tf
View File

@ -522,14 +522,14 @@ variable "iothub_dropped_d2c_telemetry_egress_message" {
default = "" default = ""
} }
variable "iothub_dropped_d2c_telemetry_egress_threshold_warning" { variable "iothub_dropped_d2c_telemetry_egress_rate_threshold_warning" {
description = "D2C Telemetry Dropped limit (warning threshold)" description = "D2C Telemetry Dropped limit (warning threshold)"
default = 500 default = 50
} }
variable "iothub_dropped_d2c_telemetry_egress_threshold_critical" { variable "iothub_dropped_d2c_telemetry_egress_rate_threshold_critical" {
description = "D2C Telemetry Dropped limit (critical threshold)" description = "D2C Telemetry Dropped limit (critical threshold)"
default = 1000 default = 90
} }
variable "iothub_orphaned_d2c_telemetry_egress_silenced" { variable "iothub_orphaned_d2c_telemetry_egress_silenced" {
@ -544,14 +544,14 @@ variable "iothub_orphaned_d2c_telemetry_egress_message" {
default = "" default = ""
} }
variable "iothub_orphaned_d2c_telemetry_egress_threshold_warning" { variable "iothub_orphaned_d2c_telemetry_egress_rate_threshold_warning" {
description = "D2C Telemetry Orphaned limit (warning threshold)" description = "D2C Telemetry Orphaned limit (warning threshold)"
default = 500 default = 50
} }
variable "iothub_orphaned_d2c_telemetry_egress_threshold_critical" { variable "iothub_orphaned_d2c_telemetry_egress_rate_threshold_critical" {
description = "D2C Telemetry Orphaned limit (critical threshold)" description = "D2C Telemetry Orphaned limit (critical threshold)"
default = 1000 default = 90
} }
variable "iothub_invalid_d2c_telemetry_egress_silenced" { variable "iothub_invalid_d2c_telemetry_egress_silenced" {
@ -566,36 +566,14 @@ variable "iothub_invalid_d2c_telemetry_egress_message" {
default = "" default = ""
} }
variable "iothub_invalid_d2c_telemetry_egress_threshold_warning" { variable "iothub_invalid_d2c_telemetry_egress_rate_threshold_warning" {
description = "D2C Telemetry Invalid limit (warning threshold)" description = "D2C Telemetry Invalid limit (warning threshold)"
default = 500 default = 50
} }
variable "iothub_invalid_d2c_telemetry_egress_threshold_critical" { variable "iothub_invalid_d2c_telemetry_egress_rate_threshold_critical" {
description = "D2C Telemetry Invalid limit (critical threshold)" description = "D2C Telemetry Invalid limit (critical threshold)"
default = 1000 default = 90
}
variable "iothub_fallback_d2c_telemetry_egress_silenced" {
description = "Groups to mute for IoT Hub fallback d2c telemetry monitor"
type = "map"
default = {}
}
variable "iothub_fallback_d2c_telemetry_egress_message" {
description = "Custom message for IoT Hub fallback d2c telemetry monitor"
type = "string"
default = ""
}
variable "iothub_fallback_d2c_telemetry_egress_threshold_warning" {
description = "D2C Telemetry Fallback limit (warning threshold)"
default = 500
}
variable "iothub_fallback_d2c_telemetry_egress_threshold_critical" {
description = "D2C Telemetry Fallback limit (critical threshold)"
default = 1000
} }
# Azure Redis specific variables # Azure Redis specific variables

16
cloud/azure/iothubs/README.md
View File

@ -40,9 +40,9 @@ Inputs
|------|-------------|:----:|:-----:|:-----:| |------|-------------|:----:|:-----:|:-----:|
| delay | Delay in seconds for the metric evaluation | string | `600` | no | | delay | Delay in seconds for the metric evaluation | string | `600` | no |
| dropped_d2c_telemetry_egress_message | Custom message for IoT Hub dropped d2c telemetry monitor | string | `` | no | | dropped_d2c_telemetry_egress_message | Custom message for IoT Hub dropped d2c telemetry monitor | string | `` | no |
| dropped_d2c_telemetry_egress_rate_threshold_critical | D2C Telemetry Dropped limit (critical threshold) | string | `90` | no |
| dropped_d2c_telemetry_egress_rate_threshold_warning | D2C Telemetry Dropped limit (warning threshold) | string | `50` | no |
| dropped_d2c_telemetry_egress_silenced | Groups to mute for IoT Hub dropped d2c telemetry monitor | map | `<map>` | no | | dropped_d2c_telemetry_egress_silenced | Groups to mute for IoT Hub dropped d2c telemetry monitor | map | `<map>` | no |
| dropped_d2c_telemetry_egress_threshold_critical | D2C Telemetry Dropped limit (critical threshold) | string | `1000` | no |
| dropped_d2c_telemetry_egress_threshold_warning | D2C Telemetry Dropped limit (warning threshold) | string | `500` | no |
| environment | Architecture Environment | string | - | yes | | environment | Architecture Environment | string | - | yes |
| failed_c2d_methods_rate_message | Custom message for IoT Hub failed c2d method monitor | string | `` | no | | failed_c2d_methods_rate_message | Custom message for IoT Hub failed c2d method monitor | string | `` | no |
| failed_c2d_methods_rate_silenced | Groups to mute for IoT Hub failed c2d methods monitor | map | `<map>` | no | | failed_c2d_methods_rate_silenced | Groups to mute for IoT Hub failed c2d methods monitor | map | `<map>` | no |
@ -76,20 +76,16 @@ Inputs
| failed_queryjobs_rate_silenced | Groups to mute for IoT Hub failed query jobs monitor | map | `<map>` | no | | failed_queryjobs_rate_silenced | Groups to mute for IoT Hub failed query jobs monitor | map | `<map>` | no |
| failed_queryjobs_rate_threshold_critical | QueryJobs Failed rate limit (critical threshold) | string | `90` | no | | failed_queryjobs_rate_threshold_critical | QueryJobs Failed rate limit (critical threshold) | string | `90` | no |
| failed_queryjobs_rate_threshold_warning | QueryJobs Failed rate limit (warning threshold) | string | `50` | no | | failed_queryjobs_rate_threshold_warning | QueryJobs Failed rate limit (warning threshold) | string | `50` | no |
| fallback_d2c_telemetry_egress_message | Custom message for IoT Hub fallback d2c telemetry monitor | string | `` | no |
| fallback_d2c_telemetry_egress_silenced | Groups to mute for IoT Hub fallback d2c telemetry monitor | map | `<map>` | no |
| fallback_d2c_telemetry_egress_threshold_critical | D2C Telemetry Fallback limit (critical threshold) | string | `1000` | no |
| fallback_d2c_telemetry_egress_threshold_warning | D2C Telemetry Fallback limit (warning threshold) | string | `500` | no |
| filter_tags | Tags used for filtering | string | `*` | no | | filter_tags | Tags used for filtering | string | `*` | no |
| invalid_d2c_telemetry_egress_message | Custom message for IoT Hub invalid d2c telemetry monitor | string | `` | no | | invalid_d2c_telemetry_egress_message | Custom message for IoT Hub invalid d2c telemetry monitor | string | `` | no |
| invalid_d2c_telemetry_egress_rate_threshold_critical | D2C Telemetry Invalid limit (critical threshold) | string | `90` | no |
| invalid_d2c_telemetry_egress_rate_threshold_warning | D2C Telemetry Invalid limit (warning threshold) | string | `50` | no |
| invalid_d2c_telemetry_egress_silenced | Groups to mute for IoT Hub invalid d2c telemetry monitor | map | `<map>` | no | | invalid_d2c_telemetry_egress_silenced | Groups to mute for IoT Hub invalid d2c telemetry monitor | map | `<map>` | no |
| invalid_d2c_telemetry_egress_threshold_critical | D2C Telemetry Invalid limit (critical threshold) | string | `1000` | no |
| invalid_d2c_telemetry_egress_threshold_warning | D2C Telemetry Invalid limit (warning threshold) | string | `500` | no |
| message | Message sent when an alert is triggered | string | - | yes | | message | Message sent when an alert is triggered | string | - | yes |
| orphaned_d2c_telemetry_egress_message | Custom message for IoT Hub orphaned d2c telemetry monitor | string | `` | no | | orphaned_d2c_telemetry_egress_message | Custom message for IoT Hub orphaned d2c telemetry monitor | string | `` | no |
| orphaned_d2c_telemetry_egress_rate_threshold_critical | D2C Telemetry Orphaned limit (critical threshold) | string | `90` | no |
| orphaned_d2c_telemetry_egress_rate_threshold_warning | D2C Telemetry Orphaned limit (warning threshold) | string | `50` | no |
| orphaned_d2c_telemetry_egress_silenced | Groups to mute for IoT Hub orphaned d2c telemetry monitor | map | `<map>` | no | | orphaned_d2c_telemetry_egress_silenced | Groups to mute for IoT Hub orphaned d2c telemetry monitor | map | `<map>` | no |
| orphaned_d2c_telemetry_egress_threshold_critical | D2C Telemetry Orphaned limit (critical threshold) | string | `1000` | no |
| orphaned_d2c_telemetry_egress_threshold_warning | D2C Telemetry Orphaned limit (warning threshold) | string | `500` | no |
| status_message | Custom message for IoT Hub status monitor | string | `` | no | | status_message | Custom message for IoT Hub status monitor | string | `` | no |
| status_silenced | Groups to mute for IoT Hub status monitor | map | `<map>` | no | | status_silenced | Groups to mute for IoT Hub status monitor | map | `<map>` | no |
| too_many_d2c_telemetry_ingress_nosent_message | Custom message for IoT Hub unsent d2c telemetry monitor | string | `` | no | | too_many_d2c_telemetry_ingress_nosent_message | Custom message for IoT Hub unsent d2c telemetry monitor | string | `` | no |

46
cloud/azure/iothubs/inputs.tf
View File

@ -244,14 +244,14 @@ variable "dropped_d2c_telemetry_egress_message" {
default = "" default = ""
} }
variable "dropped_d2c_telemetry_egress_threshold_warning" { variable "dropped_d2c_telemetry_egress_rate_threshold_warning" {
description = "D2C Telemetry Dropped limit (warning threshold)" description = "D2C Telemetry Dropped limit (warning threshold)"
default = 500 default = 50
} }
variable "dropped_d2c_telemetry_egress_threshold_critical" { variable "dropped_d2c_telemetry_egress_rate_threshold_critical" {
description = "D2C Telemetry Dropped limit (critical threshold)" description = "D2C Telemetry Dropped limit (critical threshold)"
default = 1000 default = 90
} }
variable "orphaned_d2c_telemetry_egress_silenced" { variable "orphaned_d2c_telemetry_egress_silenced" {
@ -266,14 +266,14 @@ variable "orphaned_d2c_telemetry_egress_message" {
default = "" default = ""
} }
variable "orphaned_d2c_telemetry_egress_threshold_warning" { variable "orphaned_d2c_telemetry_egress_rate_threshold_warning" {
description = "D2C Telemetry Orphaned limit (warning threshold)" description = "D2C Telemetry Orphaned limit (warning threshold)"
default = 500 default = 50
} }
variable "orphaned_d2c_telemetry_egress_threshold_critical" { variable "orphaned_d2c_telemetry_egress_rate_threshold_critical" {
description = "D2C Telemetry Orphaned limit (critical threshold)" description = "D2C Telemetry Orphaned limit (critical threshold)"
default = 1000 default = 90
} }
variable "invalid_d2c_telemetry_egress_silenced" { variable "invalid_d2c_telemetry_egress_silenced" {
@ -288,34 +288,12 @@ variable "invalid_d2c_telemetry_egress_message" {
default = "" default = ""
} }
variable "invalid_d2c_telemetry_egress_threshold_warning" { variable "invalid_d2c_telemetry_egress_rate_threshold_warning" {
description = "D2C Telemetry Invalid limit (warning threshold)" description = "D2C Telemetry Invalid limit (warning threshold)"
default = 500 default = 50
} }
variable "invalid_d2c_telemetry_egress_threshold_critical" { variable "invalid_d2c_telemetry_egress_rate_threshold_critical" {
description = "D2C Telemetry Invalid limit (critical threshold)" description = "D2C Telemetry Invalid limit (critical threshold)"
default = 1000 default = 90
}
variable "fallback_d2c_telemetry_egress_silenced" {
description = "Groups to mute for IoT Hub fallback d2c telemetry monitor"
type = "map"
default = {}
}
variable "fallback_d2c_telemetry_egress_message" {
description = "Custom message for IoT Hub fallback d2c telemetry monitor"
type = "string"
default = ""
}
variable "fallback_d2c_telemetry_egress_threshold_warning" {
description = "D2C Telemetry Fallback limit (warning threshold)"
default = 500
}
variable "fallback_d2c_telemetry_egress_threshold_critical" {
description = "D2C Telemetry Fallback limit (critical threshold)"
default = 1000
} }

72
cloud/azure/iothubs/monitors-iothubs.tf
View File

@ -336,15 +336,20 @@ resource "datadog_monitor" "too_many_d2c_telemetry_egress_dropped" {
query = <<EOF query = <<EOF
sum(last_5m): ( sum(last_5m): (
avg:azure.devices_iothubs.d2c.telemetry.egress.dropped{${var.filter_tags}} by {resource_group,region,name}.as_count() avg:azure.devices_iothubs.d2c.telemetry.egress.dropped{${var.filter_tags}} by {resource_group,region,name}.as_count() /
) > ${var.dropped_d2c_telemetry_egress_threshold_critical} (avg:azure.devices_iothubs.d2c.telemetry.egress.dropped{${var.filter_tags}} by {resource_group,region,name}.as_count() +
avg:azure.devices_iothubs.d2c.telemetry.egress.orphaned{${var.filter_tags}} by {resource_group,region,name}.as_count() +
avg:azure.devices_iothubs.d2c.telemetry.egress.invalid{${var.filter_tags}} by {resource_group,region,name}.as_count() +
avg:azure.devices_iothubs.d2c.telemetry.egress.success{${var.filter_tags}} by {resource_group,region,name}.as_count())
* 100
) > ${var.dropped_d2c_telemetry_egress_rate_threshold_critical}
EOF EOF
type = "metric alert" type = "metric alert"
thresholds { thresholds {
warning = "${var.dropped_d2c_telemetry_egress_threshold_warning}" warning = "${var.dropped_d2c_telemetry_egress_rate_threshold_warning}"
critical = "${var.dropped_d2c_telemetry_egress_threshold_critical}" critical = "${var.dropped_d2c_telemetry_egress_rate_threshold_critical}"
} }
silenced = "${var.dropped_d2c_telemetry_egress_silenced}" silenced = "${var.dropped_d2c_telemetry_egress_silenced}"
@ -369,15 +374,20 @@ resource "datadog_monitor" "too_many_d2c_telemetry_egress_orphaned" {
query = <<EOF query = <<EOF
sum(last_5m): ( sum(last_5m): (
avg:azure.devices_iothubs.d2c.telemetry.egress.orphaned{${var.filter_tags}} by {resource_group,region,name}.as_count() avg:azure.devices_iothubs.d2c.telemetry.egress.orphaned{${var.filter_tags}} by {resource_group,region,name}.as_count() /
) > ${var.orphaned_d2c_telemetry_egress_threshold_critical} (avg:azure.devices_iothubs.d2c.telemetry.egress.dropped{${var.filter_tags}} by {resource_group,region,name}.as_count() +
avg:azure.devices_iothubs.d2c.telemetry.egress.orphaned{${var.filter_tags}} by {resource_group,region,name}.as_count() +
avg:azure.devices_iothubs.d2c.telemetry.egress.invalid{${var.filter_tags}} by {resource_group,region,name}.as_count() +
avg:azure.devices_iothubs.d2c.telemetry.egress.success{${var.filter_tags}} by {resource_group,region,name}.as_count())
* 100
) > ${var.orphaned_d2c_telemetry_egress_rate_threshold_critical}
EOF EOF
type = "metric alert" type = "metric alert"
thresholds { thresholds {
warning = "${var.orphaned_d2c_telemetry_egress_threshold_warning}" warning = "${var.orphaned_d2c_telemetry_egress_rate_threshold_warning}"
critical = "${var.orphaned_d2c_telemetry_egress_threshold_critical}" critical = "${var.orphaned_d2c_telemetry_egress_rate_threshold_critical}"
} }
silenced = "${var.orphaned_d2c_telemetry_egress_silenced}" silenced = "${var.orphaned_d2c_telemetry_egress_silenced}"
@ -402,15 +412,20 @@ resource "datadog_monitor" "too_many_d2c_telemetry_egress_invalid" {
query = <<EOF query = <<EOF
sum(last_5m): ( sum(last_5m): (
avg:azure.devices_iothubs.d2c.telemetry.egress.invalid{${var.filter_tags}} by {resource_group,region,name}.as_count() avg:azure.devices_iothubs.d2c.telemetry.egress.invalid{${var.filter_tags}} by {resource_group,region,name}.as_count() /
) > ${var.invalid_d2c_telemetry_egress_threshold_critical} (avg:azure.devices_iothubs.d2c.telemetry.egress.dropped{${var.filter_tags}} by {resource_group,region,name}.as_count() +
avg:azure.devices_iothubs.d2c.telemetry.egress.orphaned{${var.filter_tags}} by {resource_group,region,name}.as_count() +
avg:azure.devices_iothubs.d2c.telemetry.egress.invalid{${var.filter_tags}} by {resource_group,region,name}.as_count() +
avg:azure.devices_iothubs.d2c.telemetry.egress.success{${var.filter_tags}} by {resource_group,region,name}.as_count())
* 100
) > ${var.invalid_d2c_telemetry_egress_rate_threshold_critical}
EOF EOF
type = "metric alert" type = "metric alert"
thresholds { thresholds {
warning = "${var.invalid_d2c_telemetry_egress_threshold_warning}" warning = "${var.invalid_d2c_telemetry_egress_rate_threshold_warning}"
critical = "${var.invalid_d2c_telemetry_egress_threshold_critical}" critical = "${var.invalid_d2c_telemetry_egress_rate_threshold_critical}"
} }
silenced = "${var.invalid_d2c_telemetry_egress_silenced}" silenced = "${var.invalid_d2c_telemetry_egress_silenced}"
@ -429,39 +444,6 @@ resource "datadog_monitor" "too_many_d2c_telemetry_egress_invalid" {
tags = ["env:${var.environment}", "resource:iothub", "team:azure", "provider:azure"] tags = ["env:${var.environment}", "resource:iothub", "team:azure", "provider:azure"]
} }
resource "datadog_monitor" "too_many_d2c_telemetry_egress_fallback" {
name = "[${var.environment}] IOT Hub Too many d2c telemetry egress fallback {{comparator}} {{#is_alert}}{{threshold}}{{/is_alert}}{{#is_warning}}{{warn_threshold}}{{/is_warning}} ({{value}})"
message = "${coalesce(var.fallback_d2c_telemetry_egress_message, var.message)}"
query = <<EOF
sum(last_5m): (
avg:azure.devices_iothubs.d2c.telemetry.egress.fallback{${var.filter_tags}} by {resource_group,region,name}.as_count()
) > ${var.fallback_d2c_telemetry_egress_threshold_critical}
EOF
type = "metric alert"
thresholds {
warning = "${var.fallback_d2c_telemetry_egress_threshold_warning}"
critical = "${var.fallback_d2c_telemetry_egress_threshold_critical}"
}
silenced = "${var.fallback_d2c_telemetry_egress_silenced}"
notify_no_data = false
evaluation_delay = "${var.delay}"
renotify_interval = 0
notify_audit = false
timeout_h = 1
include_tags = true
locked = false
require_full_window = false
new_host_delay = "${var.delay}"
no_data_timeframe = 20
tags = ["env:${var.environment}", "resource:iothub", "team:azure", "provider:azure"]
}
resource "datadog_monitor" "too_many_d2c_telemetry_ingress_nosent" { resource "datadog_monitor" "too_many_d2c_telemetry_ingress_nosent" {
name = "[${var.environment}] IOT Hub Too many d2c telemetry ingress not sent {{comparator}} {{#is_alert}}{{threshold}}{{/is_alert}}{{#is_warning}}{{warn_threshold}}{{/is_warning}} ({{value}})" name = "[${var.environment}] IOT Hub Too many d2c telemetry ingress not sent {{comparator}} {{#is_alert}}{{threshold}}{{/is_alert}}{{#is_warning}}{{warn_threshold}}{{/is_warning}} ({{value}})"
message = "${coalesce(var.too_many_d2c_telemetry_ingress_nosent_message, var.message)}" message = "${coalesce(var.too_many_d2c_telemetry_ingress_nosent_message, var.message)}"

16
cloud/azure/monitors.tf
View File

@ -98,8 +98,8 @@ module "iothub" {
too_many_d2c_telemetry_ingress_nosent_silenced = "${var.iothub_too_many_d2c_telemetry_ingress_nosent_silenced}" too_many_d2c_telemetry_ingress_nosent_silenced = "${var.iothub_too_many_d2c_telemetry_ingress_nosent_silenced}"
dropped_d2c_telemetry_egress_silenced = "${var.iothub_dropped_d2c_telemetry_egress_silenced}" dropped_d2c_telemetry_egress_silenced = "${var.iothub_dropped_d2c_telemetry_egress_silenced}"
dropped_d2c_telemetry_egress_message = "${var.iothub_dropped_d2c_telemetry_egress_message}" dropped_d2c_telemetry_egress_message = "${var.iothub_dropped_d2c_telemetry_egress_message}"
dropped_d2c_telemetry_egress_threshold_critical = "${var.iothub_dropped_d2c_telemetry_egress_threshold_critical}" dropped_d2c_telemetry_egress_rate_threshold_critical = "${var.iothub_dropped_d2c_telemetry_egress_rate_threshold_critical}"
dropped_d2c_telemetry_egress_threshold_warning = "${var.iothub_dropped_d2c_telemetry_egress_threshold_warning}" dropped_d2c_telemetry_egress_rate_threshold_warning = "${var.iothub_dropped_d2c_telemetry_egress_rate_threshold_warning}"
failed_c2d_methods_rate_silenced = "${var.iothub_failed_c2d_methods_rate_silenced}" failed_c2d_methods_rate_silenced = "${var.iothub_failed_c2d_methods_rate_silenced}"
failed_c2d_methods_rate_message = "${var.iothub_failed_c2d_methods_rate_message}" failed_c2d_methods_rate_message = "${var.iothub_failed_c2d_methods_rate_message}"
failed_c2d_methods_rate_threshold_critical = "${var.iothub_failed_c2d_methods_rate_threshold_critical}" failed_c2d_methods_rate_threshold_critical = "${var.iothub_failed_c2d_methods_rate_threshold_critical}"
@ -132,18 +132,14 @@ module "iothub" {
failed_queryjobs_rate_message = "${var.iothub_failed_queryjobs_rate_message}" failed_queryjobs_rate_message = "${var.iothub_failed_queryjobs_rate_message}"
failed_queryjobs_rate_threshold_critical = "${var.iothub_failed_queryjobs_rate_threshold_critical}" failed_queryjobs_rate_threshold_critical = "${var.iothub_failed_queryjobs_rate_threshold_critical}"
failed_queryjobs_rate_threshold_warning = "${var.iothub_failed_queryjobs_rate_threshold_warning}" failed_queryjobs_rate_threshold_warning = "${var.iothub_failed_queryjobs_rate_threshold_warning}"
fallback_d2c_telemetry_egress_silenced = "${var.iothub_fallback_d2c_telemetry_egress_silenced}"
fallback_d2c_telemetry_egress_message = "${var.iothub_fallback_d2c_telemetry_egress_message}"
fallback_d2c_telemetry_egress_threshold_critical = "${var.iothub_fallback_d2c_telemetry_egress_threshold_critical}"
fallback_d2c_telemetry_egress_threshold_warning = "${var.iothub_fallback_d2c_telemetry_egress_threshold_warning}"
invalid_d2c_telemetry_egress_silenced = "${var.iothub_invalid_d2c_telemetry_egress_silenced}" invalid_d2c_telemetry_egress_silenced = "${var.iothub_invalid_d2c_telemetry_egress_silenced}"
invalid_d2c_telemetry_egress_message = "${var.iothub_invalid_d2c_telemetry_egress_message}" invalid_d2c_telemetry_egress_message = "${var.iothub_invalid_d2c_telemetry_egress_message}"
invalid_d2c_telemetry_egress_threshold_critical = "${var.iothub_invalid_d2c_telemetry_egress_threshold_critical}" invalid_d2c_telemetry_egress_rate_threshold_critical = "${var.iothub_invalid_d2c_telemetry_egress_rate_threshold_critical}"
invalid_d2c_telemetry_egress_threshold_warning = "${var.iothub_invalid_d2c_telemetry_egress_threshold_warning}" invalid_d2c_telemetry_egress_rate_threshold_warning = "${var.iothub_invalid_d2c_telemetry_egress_rate_threshold_warning}"
orphaned_d2c_telemetry_egress_silenced = "${var.iothub_orphaned_d2c_telemetry_egress_silenced}" orphaned_d2c_telemetry_egress_silenced = "${var.iothub_orphaned_d2c_telemetry_egress_silenced}"
orphaned_d2c_telemetry_egress_message = "${var.iothub_orphaned_d2c_telemetry_egress_message}" orphaned_d2c_telemetry_egress_message = "${var.iothub_orphaned_d2c_telemetry_egress_message}"
orphaned_d2c_telemetry_egress_threshold_critical = "${var.iothub_orphaned_d2c_telemetry_egress_threshold_critical}" orphaned_d2c_telemetry_egress_rate_threshold_critical = "${var.iothub_orphaned_d2c_telemetry_egress_rate_threshold_critical}"
orphaned_d2c_telemetry_egress_threshold_warning = "${var.iothub_orphaned_d2c_telemetry_egress_threshold_warning}" orphaned_d2c_telemetry_egress_rate_threshold_warning = "${var.iothub_orphaned_d2c_telemetry_egress_rate_threshold_warning}"
} }
module "redis" { module "redis" {