Prometheus client_python tips-How to create metrics for multiple targets

Overview

Prometheus has various exporters, but I made APIs If you want to generate your own metrics, you can use the client library.

https://prometheus.io/docs/instrumenting/clientlibs/

So python also has a library for it, which is here. Basically, you can make it using Gauge, so the code will be as follows.

#!/usr/bin/env python3
# import os
import sys
import re
from logging import getLogger
from prometheus_client import generate_latest
from prometheus_client import Gauge
from prometheus_client import REGISTRY
logger = getLogger(__name__)

class MetricsGenetator(object):

    def __init__(self):
        self.g = Gauge('my_inprogress_requests', 'Description of gauge')

    def run(self):
        self.g.inc()      # Increment by 1
        self.g.dec(10)    # Decrement by given value
        self.g.set(4.2)   # Set to a given value
        metrics = generate_latest(self.registry)
        return metrics



def main():
    app = MetricsGenerator()
    metrics = app.run()
    print(metrics)

if __name__ == '__main__':
    main()

It would be nice if this was done each time, For example, if the value 4.2 passed to self.g.set is used as follows, problems will occur.

--When the code itself is reused just by dividing the target like snmpget cpu_usage from --The above code is modularized and keeps running as a constant process. --Returns when a value can be obtained, but does not enter a dummy value when it cannot be obtained

Specifically, what will happen?

Get CPU usage from server A-set to -50
Attempting to get CPU usage from server B fails (None)

At this time, if None is returned, skip self.g.set () itself

Then the CPU usage of server B is returned as 50.

In other words, the previous value is used as it is. In the case of the above structure, CPU usage = None etc. should be explicitly entered. The number of CPUs of A server and B server is fluid, and it is in the form of for cpu in cpu_num :. Dummy values cannot be entered when labels are dynamically generated.

what to do

By using the newly committed Collector Registry target_info in September 2019 You will be able to separate the Registry. Like this. However, as of November 2019, target_info added to registry.py It is not packaged, so you need to replace it yourself.

#!/usr/bin/env python3
# import os
import sys
import re
from logging import getLogger
from prometheus_client import generate_latest
from prometheus_client import Gauge
# from prometheus_client import REGISTRY
from prometheus_client.core import CollectorRegistry
logger = getLogger(__name__)

class MetricsGenetator(object):

    def __init__(self):
        registry = CollectorRegistry(target_info={"target": server_name})
        self.g = Gauge(
                     'my_inprogress_requests',
                     'Description of gauge',
                     registry=registry)

    def run(self):
        self.g.inc()      # Increment by 1
        self.g.dec(10)    # Decrement by given value
        self.g.set(4.2)   # Set to a given value
        metrics = generate_latest(self.registry)
        return metrics



def main():
    app = MetricsGenerator()
    metrics = app.run()
    print(metrics)

if __name__ == '__main__':
    main()

Note that the above does not work as it is because no server_name is entered. (Since it is an explanation of Collector Registry, the consistency of such places is omitted.)