Execute a script from Jupyter to process

Since I received a comment in the Last time article, I will show you how to execute the script from Jupyter. Your shell art will shine even more with Jupyter. : star2:

Use cell magic %% script to execute the script passed as an argument. For example, %% script python3.7 runs Python3.7, and %% script bash runs bash. Enter the contents of the script directly in the cell.

I will try running bash.

%%script bash

for i in 1 2 3 ; do
  echo $i
done
1
2
3

I was able to execute the bash I wrote in the Jupyter cell. The advantage of running with Jupyter is that you can process and visualize the results.

As an example, let's run a script to get the number of processes and visualize the result in pandas.

%%script bash --out bash_out

echo "date,procs"
for i in `seq 10` ; do
  echo "`date +%Y-%m-%d-%H:%M:%S`,`ps -ef | grep -cv grep`"
  sleep 5
done

You can store it in a Python object with the name you passed in the --out option.

print(bash_out)
date,procs
2019-12-06-10:49:21,337
2019-12-06-10:49:26,340
2019-12-06-10:49:31,340
2019-12-06-10:49:36,342
2019-12-06-10:49:41,340
2019-12-06-10:49:46,341
2019-12-06-10:49:51,344
2019-12-06-10:49:56,343
2019-12-06-10:50:01,342
2019-12-06-10:50:06,341

Let's load this into a pandas DataFrame.

from io import StringIO
import pandas as pd

df = pd.read_csv(StringIO(bash_out), index_col=["date"], parse_dates=["date"])
print(df)
                     procs
date                      
2019-12-06 10:49:21    337
2019-12-06 10:49:26    340
2019-12-06 10:49:31    340
2019-12-06 10:49:36    342
2019-12-06 10:49:41    340
2019-12-06 10:49:46    341
2019-12-06 10:49:51    344
2019-12-06 10:49:56    343
2019-12-06 10:50:01    342
2019-12-06 10:50:06    341

Visualize quickly with the plot method.

%matplotlib inline
df.plot()

image.png

In this way, Jupyter can execute arbitrary scripts and process the results with Python objects. Have a good Jupyter life!

Recommended Posts

Execute a script from Jupyter to process
[Ubuntu] How to execute a shell script
Python script to create a JSON file from a CSV file
How to run a Python program from within a shell script
[Linux] Copy data from Linux to Windows with a shell script
I made you to execute a command from a web browser
Execute Python script from batch file
Script to generate directory from json file
Try to make a kernel of Jupyter
Send a message from Python to Slack
How to execute commands in jupyter notebook
A simple IDAPython script to name a function
I made a script to display emoji
How to create a repository from media
Use Django from a local Python script
Script to create a Mac dictionary file
Creating a shell script to write a diary
How to run a Maya Python script
Verification of how to periodically execute a script on a Linux server on Windows
Send a message from Slack to a Python server
Run a Python script from a C # GUI application
Edit Excel from Python to create a PivotTable
How to open a web browser from python
Easy script migration from CentOS to Oracle Linux
To execute a Python enumerate function in JavaScript
How to create a function object from a string
Jupyter process management
Script to create FlatBuffers binaries from SQL database
Create a New Todoist Task from Python Script
How to generate a Python object from JSON
Sum from 1 to 10
How to extract coefficients from a fractional formula
A memorandum of how to execute the! Sudo magic command in Jupyter Notebook
Connect to Supercomputer ITO Jupyter from client browser
"Python Kit" that calls a Python script from Swift
I wrote a script to upload a WordPress plugin
Call a Python script from Embedded Python in C ++ / C ++
[Python] How to easily drop a child process started by multiprocess from another process
How to Git from GCP's Jupyter Lab to GSR
Write a script to convert a MySQL dump to TSV
Try to write a ping confirmation script appropriately
Run a python script from excel (using xlwings)
How to execute a query from psycopg2 built with SAM to Postgres launched with Docker
I was soberly addicted to calling awscli from a Python 2.7 script registered in crontab
Steps to attach and debug from VS Code to Jupyter Lab on a remote server
How to use NUITKA-Utilities hinted-compilation to easily create an executable file from a Python script
I made a script to put a snippet in README.md
A memorandum to run a python script in a bat file
How to hold a hands-on seminar using Jupyter using docker
Steps from installing Python 3 to creating a Django app
From buying a computer to running a program with python
How to instantly launch Jupyter Notebook from the terminal
Consider a conversion from a Python recursive function to a non-recursive function
Execute Python function from Powershell (how to pass arguments)
How to create a simple TCP server / client script
How to post a ticket from the Shogun API
How to take a captured image from a video (OpenCV)
How to execute a command using subprocess in Python
Generate a bash script to add Datadog monitor settings
[Python] How to call a c function from python (ctypes)
How to create a kubernetes pod from python code