A story that reduces the effort of operation / maintenance

Premise

The work terminal is Windows. There are dozens of Unix-like servers for operation / maintenance. The purpose is to generate the necessary data in CSV format on this Unix server and squeeze it on Windows. The following languages are ksh93 and Python

The situation at that time

I logged in to each server, hit the command, and took the data as text. There can be no such inefficiency. I'll improve it!

Improvement

First move

Spread a script (hereinafter moge.sh) that converts the required data into CSV on each Unix server. This shortens it to a few minutes per unit. After executing the script with dozens of units, fetch it with Filezilla etc. Then go crazy on Windows. Still troublesome.

2nd move

Let's make it possible to start scripts on all servers on one server, and then copy them. Like this

kicker.ksh


cat servers.list | while read HOST ; do
    ssh ${HOST} -n /hoge/moge.sh
    scp ${HOST}:/hoge/moge.csv ./data/${HOST}.csv
done

When finished, fetch all CSV from one with Filezilla etc. But ** serial **, so it takes ** 1 hour ** to finish the script.

3rd move

Since ssh is slow, I should put this in the background. That's why it looks like this

kicker2.ksh


cat servers.list | while read HOST ; do
    ssh ${HOST} /hoge/moge.sh &
    PID[${HOST}]=$!
done

while [[ ${#PID[*]} -ne 0 ]] ; do
    for HOST in ${!PID[*]} ; do
        if ! ps -ef | grep -w ${PID[${HOST}]} ; then
            scp ${HOST}:/hoge/moge.csv ./data/${HOST}.csv
            unset PID[${LPAR}]
        fi
    done
    sleep 1
done

It took only a few minutes, but after all, Filezilla-like things from Windows were troublesome.

4th move

The language is decided because Python that can make excel is convenient for messing up CSV.

kicker3.py



with open("server.lst", "r", encoding='shift-jis') as fd:
    for host in fd:
        proc = subprocess.Popen(["ssh", host, "/hoge/moge.sh"], stdout=DEVNULL, stderr=DEVNULL)
        proclist[proc.pid] = proc

while proclist:
    for pid in list(proclist):
        if proclist[pid].poll() is not None:
            del proc[pid]
            subprocess.Popen(["scp", host + ":/hoge/moge.csv", "./" + host + ".csv"], stdout=DEVNULL, stderr=DEVNULL)
    sleep(1)

It works with WSL, but ** stops in the middle ** for the command prompt. It works if you reduce the contents of server.list (up to 5 to 6). why?

4 hands extra time

I didn't understand at all, so for the time being, it is better to execute it asynchronously with asyncio of Python. It may be good. So I started research. After many twists and turns, it looks like this.

kicker4.py


async def _run(host):
    ssh = "ssh %s /hoge/moge.sh" % host
    p = await asyncio.create_subprocess_exec(ssh, stdout=DEVNULL, stderr=DEVNULL)
    await p.wait()
    scp = ["scp", host + ":/hoge/moge.csv", "./" + host + "%s.csv"]
    p = subprocess.run(scp, stdoutDEVNULL, stderr=DEVNULL)

loop = asyncio.ProactorEventLoop()
asyncio.set_event_loop(loop)

with open("server.lst", "r", encoding='shift-jis') as fd:
    funcs = asyncio.gather(*[_run(host) for host in fd])

loop.run_until_complete(funcs)

This also doesn't work. The event does not change at all. What I noticed here. ** Move when you hit * Enter * when you stop **. why? Is there any problem with standard input?

Solution

The correct answer is written in the first ksh script. This is due to the ** "-n" option ** of ssh. In ksh, read and ssh conflict with standard input, so I thought that "-n" was necessary, so I was wearing it from the beginning. However, Python has a standard input other than ssh. It seems that the prejudice that it is okay to do nothing was a problem because there is no such thing. It seems that standard input needs control when launching multiple ssh in the background. (stdin = subprocess.DEVNULL was useless) It's still a mystery that it worked on WSL, but I won't investigate it because I don't use it anymore.

Why I stopped using WSL

In my Python script, I also use openpyxl for gonyogonyo on Windows. If this is on WSL, it sometimes eats BSOD, so it works from the command prompt It was repaired as follows.

Simple question

Why are there multiple ways to give commands to subprocess? I wish I could unify it.

Recommended Posts

A story that reduces the effort of operation / maintenance
The story of making a package that speeds up the operation of Juman (Juman ++) & KNP
A story that analyzed the delivery of Nico Nama.
The story of writing a program
A story that struggled to handle the Python package of PocketSphinx
A story that stumbled upon a comparison operation
The story of creating a site that lists the release dates of books
The story of blackjack A processing (python)
The story of making a module that skips mail with python
A story that visualizes the present of Qiita with Qiita API + Elasticsearch + Kibana
The story of developing a web application that automatically generates catchphrases [MeCab]
The story of making a lie news generator
The story of making a mel icon generator
The story of sys.path.append ()
The story of IPv6 address that I want to keep at a minimum
The story of making a box that interconnects Pepper's AL Memory and MQTT
The story of making a web application that records extensive reading with Django
The story of Django creating a library that might be a little more useful
The story of making a Line Bot that tells us the schedule of competitive programming
The story of launching a Minecraft server from Discord
[Python] A program that counts the number of valleys
Make a BOT that shortens the URL of Discord
# Function that returns the character code of a string
The story of making a music generation neural network
A story that struggled with the common set HTTP_PROXY = ~
Generate that shape of the bottom of a PET bottle
A story about changing the master name of BlueZ
The story that the return value of tape.gradient () was None
Zip 4 Gbyte problem is a story of the past
[Python] A program that compares the positions of kangaroos.
A python script that generates a sample dataset for checking the operation of a classification tree
The story of Linux that I want to teach myself half a year ago
The story of building Zabbix 4.4
[Apache] The story of prefork
A tool that automatically turns the gacha of a social game
The story of creating a VIP channel for in-house chatwork
The story of a Django model field disappearing from a class
The story of creating a database using the Google Analytics API
The story of making a question box bot with discord.py
A Python script that compares the contents of two directories
A memo that reproduces the slide show (gadget) of Windows 7 on Windows 10.
When incrementing the value of a key that does not exist
A story stuck with the installation of the machine learning library JAX
The story that the version of python 3.7.7 was not adapted to Heroku
The story of Python and the story of NaN
pandas Fetch the name of a column that contains a specific character
The story of participating in AtCoder
The story that a hash error came out when using Pipenv
A formula that simply calculates the age from the date of birth
A story that verified whether the number of coronas is really increasing rapidly among young people
The story of making a standard driver for db with python.
A function that measures the processing time of a method in python
The story of the release work of the application that Google does not tell
The story of the "hole" in the file
A story that is a little addicted to the authority of the directory specified by expdp (for beginners)
I made a slack bot that notifies me of the temperature
A story that supports electronic scoring of exams with image recognition
The story of making a tool that runs on Mac and Windows at the game development site
The story of remounting the application server
[python] A note that started to understand the behavior of matplotlib.pyplot
The story of creating a bot that displays active members in a specific channel of slack with python