Write your blackbird plugin # 003

Let's write the blackbird plugin The third bullet, where I could not write in detail in the second bullet, mainly

Let's die.

About each member properties of JobBase class

self.options

I explained this in the second bullet, but interpret the contents described in the config file as key and value, It is in dictionary format. In particular

[SECTION_NAME_IS_ANYTHING_OK]
hostname = hogehoge.com
hogehoge = fugafuga

If there is a config file like

self.options = {
    'hostname': 'hogehoge.com',
    'hogehoge': 'fugafuga'
}

You will have a dictionary that looks like this. I intend to write about this in quite detail in the second installment, so It is like this. More kwsk! If there is something like that, I think I'll write it in the comments or if you contact me.

self.queue

The Main process is a Queue for exchanging messages between Threads. I didn't do anything particularly difficult I'm using the Python-made Queue.Queue as it is. So think of the self.queue.put method as being the same as reading Queue.Queue.put.

notes:

Regarding the put method (I think that plugin writers will mainly use the put method) When there is an argument called block and you are doing a put operation, you can perform operations from other Consumers and Producers. It's about blocking. However, this is under the condition that key, clock, host do not cover one item. ** block = False is highly recommended **.

self.logger

An instance for logging. Since this instance is just a Python-made logging module with only the output format changed, The methods of the logging module can be used as they are.

self.logger.debut('This is Debug log!! HOGEHOGE!!!!!!!!!!')

Therefore, it is also possible to write as above. The log output level is WARN by default (this can be changed in the global section).

self.invalid_key_list

//It's new, but I feel like I made a mistake in the property name.
//I feel like it's not invalid w ignore??

This is a blacklist that if the Item key exists in this list, it cannot be put in the Queue. For example

class ConcreteJob(blackbird.plugins.base.JobBase):

    def __init__(self, options, queue=None, logger=None):
        super(ConcreteJob, self).__init__(options, queue, logger)
        self.invalid_key_list = [
            'host.IGNORE_KEY_NAME001',
            'host.IGNORE_KEY_NAME002'
        ]

By doing so, it is possible to cut out the logic to another method without writing the distribution process in the place where it is fetched completely like the stats command of a certain memcached. In this case, of course, you can't just add it to the list.

#..snip..

def filter_item(item):
    if not item.data['key'] in self.invalid_key_list:
        return item

def filter_items(items):
    filtered_items = list()
    for entry in items:
        if not items.data.['key'] in self.invalid_key_list:
            filtered_items.append(entry)

    return filtered_items

I hope you can implement it with a good feeling.

About convenience functions implemented on each base class side

Next, I would like to talk about the convenience functions implemented on each base class side. Well, to be honest, there aren't many, but I think they will help you create plugins.

blackbird.plugins.base.ValidatorBase.detect_hostname

As the name suggests, it is a method that fetches the hostname. I'm using low-layer functions (I hope Python can use this kind of thing lightly), but it's really simple,

return socket.getfqdn() or socket.gethostname() or 'localhost'

I'm just doing it.

  1. If there is a result of socket.getfqdn (), return it as it is
  2. If not, return socket.gethostname () (short hostname, the result of hostname -s)
  3. If that doesn't work, you can return localhost as a string.

As a messenger

class Validator(base.ValidatorBase):
    def __init__(self):
        self.__spec = None

    @property
    def spec(self):
        self.__spec = (
            "[{0}]".format(__name__),
            "hostname = string(default={0})".format(self.detect_hostname()),
        )
        return self.__spec

Is it such a place? By the way, this was implemented at the suggestion of a big senior, and thank you for taking this opportunity. M (\ _ \ _) m

Timer context

Timer context is a context that measures the processing time in the context enclosed in with. We use it to write and read keys to redis and memcached to measure ratency and make it Metric. The usage is no different from ordinary Context,

import blackbird.plugins.base.Timer

with Timer() as timer:
    for entry in xrange(0, 9):
        print entry

    return [timer.sec, timer.msec]

In other words, the sec property of the timer class returns the processing time in seconds. The msec property returns the processing time in milliseconds.

About Blabckbird Exception

Exception (the real meaning here is Exception when an unexpected error occurs in your plugin) To distinguish whether it was unexpected when it occurred or was intentionally generated like a raise Value Error. It implements its own Exception. However, since you only need to specify the error string and raise it, raise it in exactly the same way as ValueError. For example

import blackbird.plugins.base.BlackbirdPluginError

try:
    value = self.options['key']
except KeyError as exception:
    raise blackbird.plugins.base.BlackbirdPluginError(
        exception.__str__
    )

This makes it easier to debug the behavior when an Exception is thrown.


blackbird.plugins.base This is what the module does, but I don't know what this is, or it's different here, and I think I want a method like this on the base side as well. If there is, I would like to know more and more m (\ _ \ _) m

Recommended Posts

Write your blackbird plugin # 002
Write your blackbird plugin # 001
Write your blackbird plugin # 003
Write a vim plugin in Python
Write a simple Vim Plugin in Python 3