Generate a bash script to add Datadog monitor settings

Disclaimer

--We do not guarantee that the content is correct. ――Since the content is slightly different from what you are actually using, we do not guarantee the operation. -(I feel like I can do this) ――I cry if the same function is provided by Datadog. ――The usage at the end is almost delusional.

Purpose

Using a JSON file that describes the Datadog monitor settings Generate a bash script to add monitor settings.

Actually, instead of setting it from the GUI like sashimi dandelion The monitor settings with similar contents can be applied all at once by simply rewriting a part of the text.

Maybe if you make full use of Shell art, it will be completed with a shell. I don't have the shell power and the feeling of maintenance, so I make it with python.

Contents

Directory structure

Directory structure

./
 ├ module/ 
 │   └ monitoring_script_builder.py (Script to render template)
 ├ template/
 │   └ create_monitor.j2
 ├ script/ (Script output directory)
 ├ json/ (Datadog monitor settings json file storage location)
 └ builder.py (Main script)

Template file

create_monitor.j2

#!/bin/bash
api_key={{api_key}}
app_key={{app_key}}

curl -X POST -H "Content-type: application/json" \
-d '{{json}}' "https://app.datadoghq.com/api/v1/monitor?api_key=${api_key}&application_key=${app_key}"

This template itself is a template made by partially modifying Datadog official sample.

script

Template rendering script (monitoring_script_builder.py)

Datadog Application Key and API Key, After that, read the information (JSON file) to be embedded in the script, Renders and returns the template (create_monitor.j2).

monitoring_script_builder.py

from jinja2 import Template, Environment, FileSystemLoader


class MonitoringScriptBuilder():

    def __init__(self, **kwargs):
        env = Environment(loader=FileSystemLoader('./template/'))
        self.template = env.get_template('create_monitor.j2')

    def render(self, data, app_key, api_key):
        """
        """
        data['app_key'] = app_key
        data['api_key'] = api_key
        rendered = self.template.render(data)
        return rendered

Main script (builder.py)

While reading the json file in the json directory Read the Datadog API key and Datadog APP key from the environment variables.

builder.py

from module.monitoring_script_builder import MonitoringScriptBuilder
import os
import glob
import re
builder = MonitoringScriptBuilder()

#Read part of JSON file
file_list = glob.glob(os.path.join("json", "*.json"))


#Read key information
api_key = os.getenv('DD_API_KEY')
app_key = os.getenv('DD_APP_KEY')

#Generate file from template
for file in file_list:
    f = open(file)
    json = f.read()
    data = {
        'json': json, 
    } 

    rendered = builder.render(data, app_key, api_key)
    print(rendered)

    f.close()

    file_out = file.replace("json", "")


    #Open the file for writing
    fout = open("script/" + file_out +"sh",'w')
    fout.write(rendered)
    fout.close()

Execution sample

This time, we will use the following sample.

json/sample.json

{
	"name": "{{host.name}}Cron is stopped at.",
	"type": "service check",
	"query": "\"process.up\".over(\"process:cron\").by(\"host\",\"process\").last(2).count_by_status()",
	"message": "{{host.name}}of{{process.name}}Is stopped.@Notification destination",
	"options": {
		"notify_audit": false,
		"locked": false,
		"timeout_h": 0,
		"no_data_timeframe": 2,
		"new_host_delay": 300,
		"notify_no_data": false,
		"renotify_interval": 10,
		"escalation_message": "@Notification destination\n\n10 minutes have passed,{{host.name}}of{{process.name}} \n is stopped.",
		"thresholds": {
			"warning": 1,
			"ok": 1,
			"critical": 1
		}
}

Set like this (key information is of course dummy)

Execution sample

$ export APP_KEY="piyopiyo"
$ export API_KEY="hogehoge"
$ python ./builder.py

If the execution is successful, under the ``` script directory` ``

sample.sh should be output.

#### **`sample.Confirmation of sh`**
```bash

#!/bin/bash
api_key=hogehoge
app_key=piyopiyo

curl -X POST -H "Content-type: application/json" \
-d '{
	"name": "{{host.name}}Cron is stopped at.",
	"type": "service check",
	"query": "\"process.up\".over(\"process:cron\").by(\"host\",\"process\").last(2).count_by_status()",
	"message": "{{host.name}}of{{process.name}}Is stopped.@Notification destination",
	"options": {
		"notify_audit": false,
		"locked": false,
		"timeout_h": 0,
		"no_data_timeframe": 2,
		"new_host_delay": 300,
		"notify_no_data": false,
		"renotify_interval": 10,
		"escalation_message": "@Notification destination\n\n10 minutes have passed,{{host.name}}of{{process.name}} \n is stopped.",
		"thresholds": {
			"warning": 1,
			"ok": 1,
			"critical": 1
		}
}' "https://app.datadoghq.com/api/v1/monitor?api_key=${api_key}&application_key=${app_key}"

How to use this script

--Large amount of sashimi dandelion monitor settings

It is used when you want to input a large number of monitor settings that have almost the same conditions but are slightly different.

--Recovering monitor settings from backup

When the monitor setting is saved in json and the monitor setting of Datadog is broken Used for recovery. This is different because a script is generated for each single monitor setting. It may be used.

--Diversion of monitor settings of other accounts

When migrating monitor settings used by other accounts Available via json export.

References

Dadadog API Reference Basic usage of Jinja2