Create a custom search command in Splunk (Streaming Command)

Reverse strings in Splunk. near the end

If you use it often, you may want to have an external python script.

I wrote that, so I made it. It was really hard: sweat_smile:

@uneyamauneko @ msi's article really helped me.

The copy source is countmatches.py@splunk-sdk-python

Preparation

You can do pip install splunk-sdk in Install the Splunk Enterprise SDK for Python.

But my environment

Environment variables, etc.


% uname -v 
Darwin Kernel Version 19.6.0: Tue Nov 10 00:10:30 PST 2020
% which python
/opt/anaconda3/bin/python
% python --version
Python 3.7.6
% $SPLUNK_HOME/bin/splunk --version
Splunk 8.1.1 (build 08187535c166)

With pip install splunk-sdk

pip result


 % pip install splunk-sdk
Requirement already satisfied: splunk-sdk in /opt/anaconda3/lib/python3.7/site-packages (1.6.14)

It is included. Certainly, I remember brew install splunk-sdk.

Here, set commands.conf in $ SPLUNK_HOME/etc/<< MYAPPS >>/default/as shown in Example. Https://docs.splunk.com/Documentation/Splunk/8.1.1/Admin/Commandsconf Is incorrect. Reported.

commands.conf


[rev]
chunked = true
filename = rev.py

important subjects

Create a link in / Applications/Splunk/lib/python3.7/site-packages.

ln -s /Applications/Splunk/lib/python3.7/site-packages $SPLUNK_HOME/lib/python3.7/site-packages/splunklib 

result


% pwd
/Applications/Splunk/lib/python3.7/site-packages
% ls -la
lrwxr-xr-x    1 XXXXX  wheel     52 12 31 10:53 splunklib -> /opt/anaconda3/lib/python3.7/site-packages/splunklib

Reason

--No matter how many times I try, import splunk lib does not work -- import splunklib works fine with python started with / usr/bin/env splunk. --import splunklib fails with python in $ SPLUNK_HOME/bin/splunk cmd python. ――It can't be helped, so I created a symbolic link and it worked.

Code

rev.py


#!/usr/bin/env python

import sys
from splunklib.searchcommands import dispatch, StreamingCommand, Configuration, Option, validators

@Configuration()
class reverseCommand(StreamingCommand):
    """ Reverse the string.

    ##Syntax

    .. code-block::
        rev output=<field> <field>

    ##Description
    Outputs the string of the input field in reverse order.

    ##Example

    .. code-block::
        index=_internal clientip=* | head 1 | rev output=r_clientip clientip

    """

    output = Option(
        doc='''
        **Syntax:** **output=***<fieldname>*
        **Description:** Name of the field that will hold the revesed text''',
        require=True, validate=validators.Fieldname())

    def stream(self, records):
        self.logger.debug('revCommand: %s', self)  # logs command line
        for record in records:
            for fieldname in self.fieldnames:
                pass
            record[self.output]=record[fieldname][-1::-1]
            yield record

dispatch(reverseCommand, sys.argv, sys.stdin, sys.stdout, __name__)

How to use

| rev output=<output field> <field>

result

rev.spl


index=_internal clientip=* | head 1
| rev clientip output=clientip_s

rev.png

A difficult story named commentary

--Almost the original copy and paste -Shebang has no effect (likely).

First I put commands.conf and rev.py in place (default and bin) and restarted Splunk. At that time, such a file.

rev.py


#!/usr/bin/env python

import sys
from splunklib.searchcommands import dispatch, StreamingCommand, Configuration, Option, validators

@Configuration()
class reverseCommand(StreamingCommand):
    """ Reverse the string.
    """
    def stream(self, records):
        self.logger.debug('revCommand: %s', self)  # logs command line
        for record in records:
            yield record

dispatch(reverseCommand, sys.argv, sys.stdin, sys.stdout, __name__)

A command that does nothing. For the time being, I confirmed the operation with this. Once I've confirmed that | rev client ip works without error.

After that, I made various changes, executed it, saw the result, corrected it, and repeated the execution.

The main processing of this command is as follows.

main


    def stream(self, records):
        self.logger.debug('revCommand: %s', self)  # logs command line
        for record in records:
            for fieldname in self.fieldnames:
                pass
            record[self.output]=record[fieldname][-1::-1]
            yield record

Is it the Streaming Command that returns with yield without returning the return value with return?

After trying various things, it seems that the field name that is the argument of the command can only be taken with the fieldname created byfor fieldname in self.fieldnames:. I get the error unhashable type:'list'.

I tried merging the

lists and it was okay without them.

I tried to remove it.


    def stream(self, records):
        self.logger.debug('revCommand: %s', self)  # logs command line
        for record in records:
            record[self.output]=record[''.join(self.fieldnames)][-1::-1]
            yield record

Click here for reference StreamingCommand.fieldnames

If you simply overwrite record [fieldname] = record [fieldname] [-1 :: -1] is fine, but I think that the usability is subtle, so I set the option and changed to the above

Summary

It was very difficult because I made it for the first time. It's been a while since I couldn't import ...

Looking at the * Job Survey *, I feel that it is taking some time. For the time being, I think that you can make this as a template with python that simply returns.

Recommended Posts

Create a custom search command in Splunk (Streaming Command)
Create Splunk custom search command Part 2
Create a Splunk custom search command-the first step-
Create a dictionary in Python
Write a binary search in Python
Hit a command in Python (Windows)
Create a CSV reader in Flask
Create a DI Container in Python
Create a binary file in Python
Write a depth-first search in Python
Create a Kubernetes Operator in Python
Create a random string in Python
Create a LINE Bot in Django
Create a custom field where enum can be specified in choices
[LLDB] Create your own command in Python
Create a simple GUI app in Python
Create a JSON object mapper in Python
Implement a custom View Decorator in Pyramid
Create a command to delete all temporary files generated in a specific folder
Create a Python-GUI app in Docker (PySimpleGUI)
[GPS] Create a kml file in Python
Creating custom search commands for Splunk EventingCommand
Implement a Custom User Model in Django
Create a web service in Flask-SQLAlchemy + PostgreSQL
Create a plugin that allows you to search Sublime Text 3 tabs in Python
Launch a simple password-protected search service in 5 minutes
Create a Vim + Python test environment in 1 minute
Create a GIF file using Pillow in Python
Create an executable file in a scripting language
I want to create a window in Python
Create a standard normal distribution graph in Python
How to create a JSON file in Python
Create a virtual environment with conda in Python
Create a protein sequence mutation library in pandas
Use click to create a sub-sub command --netsted sub-sub command -
Create a simple momentum investment model in Python
Try to create a new command on linux
How to create a shortcut command for LINUX
Create a new page in confluence with Python
Create a datetime object from a string in Python (Python 3.3)
Implement similar face search in half a day
Create a package containing global commands in Python
Use a custom error page in python / tornado
How to create a Rest Api in Django
Until you create a new app in Django
Create a MIDI file in Python using pretty_midi
Create a loop antenna pattern in Python in KiCad
Create a command to get the work log
[Docker] Create a jupyterLab (python) environment in 3 minutes!
Create a web server in Go language (net/http) (2)
[Linux] Search command
I tried to create a class to search files with Python's Glob method in VBA
Heppoko develops web service in a week # 2 Domain search
Create a data collection bot in Python using Selenium
Specify a subcommand as a command line argument in Python
[LINE Messaging API] Create a rich menu in Python
Create a plugin to run Python Doctest in Vim (2)
[Go] How to create a custom error for Sentry
Create a plugin to run Python Doctest in Vim (1)
In Python, create a decorator that dynamically accepts arguments Create a decorator
Create a web server in Go language (net / http) (1)