Batch startup monitoring (Java x PushGateway x Prometheus)

Summary of this article

--I found something called Push Gateway when I was wondering what to do with batch startup monitoring. ――I tried to summarize from the introduction to the place where you can see the metrics in Prometheus

Constitution

--Push Gateway and Prometheus start with Docker ――As a mechanism --Batch sends metrics to Push Gateway --Prometheus gets metrics from Push Gateway --Proceed as if everything exists locally

Push Gateway construction

• It is a prerequisite that the environment where Docker can be used is in place.

Start-up

docker pull prom/pushgateway
docker run -d \
    -p 9091:9091 \
    --name pushgateway \
    prom/pushgateway

-d: Start in detached mode. I'm not very familiar with it, so it's recognized that it can be executed in the background -p: Port forwarding settings -name: Specifying the container name

Operation check

--Access http: // localhost: 9091 --The screen is still blank, but when the batch notifies you of the metrics, it will be displayed on this screen. --By the way, the endpoint used to get the metrics from Prometheus is / metrics - http://localhost:9091/metrics

Creating batches (miscellaneous)

build.gradle

apply plugin: 'java'

repositories {
    mavenCentral()
}

dependencies {
    compile('io.prometheus:simpleclient_pushgateway:0.6.0')
}

SampleBatch.java

import java.io.IOException;
import java.util.HashMap;
import java.util.Map;

public class SampleBatch {
    public static void main(String[] args) {
        String url = "localhost:9091";
        PushGateway pushGateway = new PushGateway(url);

        long now = System.currentTimeMillis();

        //Collect metrics (here set system milliseconds as startup time)
        CollectorRegistry registry = new CollectorRegistry();
        Gauge lastStartAt = Gauge.build()
                .name("batch_start_time")
                .help("batch start time")
                .register(registry);
        lastStartAt.set(now);

        //Set tags for metrics
        Map<String, String> key = new HashMap<>();
        key.put("tagName", "tagValue");

        try {
            //Register metrics
            pushGateway.pushAdd(registry, "sample-batch", key);
        } catch (IOException e) {
            //Exception handling
            e.printStackTrace();
        }
    }
}

Run!

--After execution, if you look at the UI of Push Gateway, ... ――It is certainly registered!

Building Prometheus

Start-up

prometheus.yml

# /etc/prometheus/prometheus.yml

global:
  scrape_interval:     15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
  evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.
alerting:
  alertmanagers:
  - static_configs:
    - targets:

rule_files:

scrape_configs:
- job_name: 'pushgateway'
  scrape_interval: 1m
  metrics_path: /metrics
  static_configs:
  - targets: ['localhost:9091']

docker pull prom/prometheus
docker run -d \
  -p 9090:9090 \
  -v prometheus.yml:/etc/prometheus/prometheus.yml \
  --name prometheus \
  prom/prometheus

Operation check

--Check with UI (http: // localhost: 9090) --There seems to be a better way, but you can see when the batch started with PromQL called changes (batch_start_time [2m])!

Actual operation

--In the field, we use Grafana to skip visualizations and alerts.