Python standard module that can be used on the command line

Launch a python module from the command line

$ python --help
usage: python [option] ... [-c cmd | -m mod | file | -] [arg] ...
...
-m mod : run library module as a script (terminates option list)
...

-m mod : run library module as a script (terminates option list)

If you start python with the -m option, it will search for the module in $ PYTHONPATH and execute it. Just by advocating Battery Included, you can use a wide range of functions without writing code. Even if you're not interested in the Python language itself, it's helpful to remember it when working with the shell. However, it seems that there are many things that are not well documented, so I will introduce what I know.

Introduction of modules that can be used

Both can be used with python2 series (which is included as standard for OSX / Linux). No need to install special packages.

json.tool

Receives JSON data from standard input, formats it and displays it.

$ echo '{"spam": ["ham", "egg"]}' | python -m json.tool
{
    "spam": [
        "ham",
        "egg"
    ]
}

You can't use jq or jsonlint without installing it, but python can be used in many environments.

The weak point is that the indent is fixed at 4 (others are usually 2), so it's good to pipe to sed -e's / / / g'.

SimpleHTTPServer

Publish under the current directory via HTTP.

$ python -m SimpleHTTPServer 8080
Serving HTTP on 0.0.0.0 port 8080 ...

If you access http: // localhost: 8080 in this state, you can see the file list.

SimpleHTTPServer.png

It can be used for debugging web applications and checking HTML files. With no arguments, port 8000 is used by default.

CGIHTTPServer

Almost the same as SimpleHTTPServer, but with CGI enabled.

$ python -m CGIHTTPServer
Serving HTTP on 0.0.0.0 port 8000 ...

Very useful for publishing small CGI scripts on your LAN.

smtpd

Set up an SMTP proxy server locally. It can be used in combination with smtplib described later.

$ sudo python -m smtpd -d -n localhost:25

smtplib

Send an email to localhost. Prompts will appear in the order of From, To, Body, so write an email from standard input.

$ python -m smtplib

ftplib

Become an ftp client.

$ python -m ftplib ftp://localhost

It's a degraded copy of the ftp command, but it's possible that there is no ftp and there is a python.

telnetlib

Become a telnet client.

$ python -m telnetlib towel.blinkenlights.nl

You can watch the AA version of the famous Star Wars Episode 4.

poplib

Become a pop3 mail client.

$ python -m poplib localhost user password

htmllib

Become a text browser. Since you cannot refer to the URL directly, you need to download it locally in advance.

$ python -m htmllib index.html

It's not a great substitute for w3m or lynx, such as not interpreting inline javascript. If you are forced to think about how to use it, it will be a watershed between parsing non-well-formatted HTML with htmllib or bringing out BeautifulSoup (if this command fails, it is HTML that cannot be parsed by Python standard, so BeautifulSoup You can use to determine that it should be analyzed).

xmllib

Parse XML.

$ python -m xmllib egg.xml
xml: encoding = ISO-8859-1 standalone = None
data: '\n'
...

It doesn't seem to be usable by itself ... It may be used a little if combined with ʻawk`. But it's better to write a script than to do that.

filecmp

Take the difference between the two directories.

python -m filecmp dir1/ dir2/
diff dir1 dir2
Only in dir1 : ['foo.txt']
Only in dir2 : ['bar.txt']
Differing files : ['baz.txt']

Recursively traverse directories with the -r option (default is only one level). You can see the file name and whether it has been changed, but note that the contents are not displayed.

gzip

GZIP compression / decompression tool.

$ python -m gzip baz.tar   # compress
$ python -m gzip -d baz.tar.gz # decompress

Slightly less functional than the zipfile described below. I don't even help. You can't expand tar, so you may not use it.

zipfile

ZIP compression / decompression tool.

$ python -m zipfile -c foo.zip bar.txt baz.png

List with the -l option, test with the -t option, expand with the -e option, and compress with the -c option.

You can do most of the operations with the zip command, but it is subtle if you have a chance to use it (in an environment where this module can be used, the zip command is likely to be used as well).

mimetype

Takes a file name as an argument and returns the MIMETYPE of that file.

$ python -m mimetype ham.png
type: image/png encoding: None

Since it is judged from the file name, it works even if the file itself does not exist. Is it almost equivalent to the --mimetype option of the file command?

base64

Base64 encode / decode standard input and display it on standard output.

$ echo 'python' | python -m base64
cHl0aG9uCg==
$ echo 'cHl0aG9uCg==' | python -m base64 -d
python

Almost equivalent to the ʻopenssl base64command. There seems to be a decent environment withpython without ʻopenssl.

uu

Encode / decode standard input with uuencode and display it on standard output.

$ python -m uu < baz.png

There are few opportunities to use it compared to base64.

mimify

Encode / decode the standard input into a format that can be handled by e-mail and display it on the standard output. Similar to base64, but with a mail header.

$ python -m mimify -e foo.jpg  # encode
$ python -m mimify -d bar.eml  # decode

It is recommended to redirect to a file as control characters can corrupt the terminal.

quopri

Encode / decode standard input with Quoted-printable and display it on standard output.

$ python -m quopri < quux.png

Base64 is the de facto standard for these binary <=> ascii conversion codecs, and there are few opportunities to use them.

timeit

Measure the execution time of a piece of Python code. It seems to be more precise than the POSIX time command. It is quite smart, such as automatically determining the number of loops so that the error is small.

$ python -m timeit 'pow(2, 10000)'
10000 loops, best of 3: 25 usec per loop

There are many options. See python -m timeit -h for more information.

profile

Measure the profile of a Python script.

$ python -m profile ham.py
          473 function calls in 0.009 seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.000    0.000    0.000    0.000 :0(range)
        1    0.006    0.006    0.006    0.006 :0(setprofile)
...

Good for finding bottlenecks when tuning scripts.

modulefinder

Recursively search for the module you are importing.

$ python -m modulefinder foo.py

  Name          File
  ----          ----
m __main__      foo.py
m array
m re            /usr/lib/python2.7/re.py

You can use it when you are addicted to a dependency ... Isn't it?

unittest

A test runner for Python's standardized test library ʻunittest`. Since it is described in detail in Reference, the explanation is omitted.

It's good to remember the idiom of python -m unittest discover -v.

doctest

This is also a test runner for Python's standardized test library doctest. See Reference.

After all it is OK if you only remember python -m doctest -v foo.py.

Personally, I think it's a very Python-like module.

site

Display information such as module loading destination, sys.path.

$ python -m site
sys.path = [
    '/usr/lib/python2.7',
    '/usr/lib/python2.7/plat-x86_64-linux-gnu',
...

You can list environment variables related to Python, so it seems to be useful.

platform

Display OS information.

$ python -m platform
Linux-3.13.0-48-generic-x86_64-with-Ubuntu-14.04-trusty

The output information is like adding ʻuname -aand (in Linux)cat / etc / issue`. It is easy to use because it is normalized by hyphen delimiter.

shlex

Become a shell lexical analyzer. Cut out tokens using Bourne shell style grammar.

$ echo 'spam ham egg' | python -m shlex
Token: 'spam'
Token: 'ham'
Token: 'egg'

The purpose is unknown, but there seems to be no similar command.

locale

Display locale settings.

$ python -m locale
Locale aliasing:

Locale defaults as determined by getdefaultlocale()
-----------------------------------------------------------------
Language: ja_JP
Encoding: UTF-8

Locale settings on startup:
-----------------------------------------------------------------
LC_NUMERIC ...
   Language:  (undefined)
   Encoding:  (undefined)
...

The display is easy to read (redundant), which is almost the same as the locale command.

webbrowser

A module that just opens a browser from the command line.

$ python -m webbrowser -n http://qiita.com

Open a window with the -n option and a new tab with the -t option. It's sober, but there aren't many similar commands, so it might be useful if you have a messenger. The browser to open is automatically selected as specified by the system and cannot be changed.

tabnanny

Check the indentation of the Python script, correct any ambiguity, and rewrite.

$ python -m tabnanny my_script.py

The directory can also be taken as an argument, but be careful as it will recursively overwrite the files inside.

compileall

Compile a Python script.

python -m compileall my_project/

Recursively search directories and compile .py files into .pyc. Since it has a decompiler, it cannot be used for obfuscation, Usually, the performance improvement is not so big, so I don't have much chance to use it. Click here for Reference.

dis

Python disassembler. Analyze the code in the * .py script.

$ python -m dis /usr/lib/python2.7/abc.py
  4           0 LOAD_CONST               0 ('Abstract Base Classes (ABCs) according to PEP 3119.')
              3 STORE_NAME               0 (__doc__)

  6           6 LOAD_CONST               1 (-1)
              9 LOAD_CONST               2 (None)
             12 IMPORT_NAME              1 (types)
             15 STORE_NAME               1 (types)

  8          18 LOAD_CONST               1 (-1)
             21 LOAD_CONST               3 (('WeakSet',))

See Reference for a list of instructions.

bonus

code

Open the interpreter. Equivalent to the python command with no arguments.

$ python -m code
Python 2.7.6 (default, Mar 22 2014, 22:59:56)
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>>

trace

Trace Python function calls.

Reference describes how to use it, but honestly, I feel that pdb is enough.

pdb

Start Python's interactive debugger pdb. See Reference for details.

It is usually provided as a pdb command, so you don't have to call it as a module.

pydoc

This is also the familiar pydoc command, but it can also be started as a module.

turtle

Demonstration of the standard GUI module Tkinter. Although it is attached as standard, please note that it is not included depending on the environment such as OSX.

$ python -m turtle

turtle.png

encodings.rot_13

Encrypt the standard input with rot13 and display it on the standard output.

$ echo 'hamegg' | python -m encodings.rot_13
unzrtt

Originally a story module. I don't usually use it, but it's interesting that rot13 is included in the standard library.

antigravity

$ python -m antigravity

Neta module part 2. I will not explain it.

Recommended Posts

Python standard module that can be used on the command line
Correspondence memo when the direction key cannot be used on the python command line
Python standard input summary that can be used in competition pro
The problem that the ifconfig command cannot be used
Python note: When the pip command cannot be used
Can be used in competition pros! Python standard library
[Django] About users that can be used on template
Goroutine (parallel control) that can be used in the field
Goroutine that can be used in the field (errgroup.Group edition)
Scripts that can be used when using bottle in Python
Install Mecab and CaboCha on ubuntu16.04LTS so that it can be used from python3 series
Which octal literals can be used depends on the programming language
A timer (ticker) that can be used in the field (can be used anywhere)
Keep getting RSS on the command line
Understand the probabilities and statistics that can be used for progress management with a python program
[Python] I tried to make a simple program that works on the command line using argparse.
You search commandlinefu on the command line
About the matter that the re.compiled object can be used for the re.match pattern
I created a template for a Python project that can be used universally
Acoustic signal processing module that can be used with Python-Sounddevice ASIO [Application]
Mathematical optimization that can be used for free work with Python + PuLP
Acoustic signal processing module that can be used with Python-Sounddevice ASIO [Basic]
The --security option cannot be used with the dnf command on CentOS Linux
File types that can be used with Go
Functions that can be used in for statements
Quickly display the QR code on the command line
Frequently used (personally) notes on the tar command
Arduino development on the command line: vim + platformio
Syntax highlighting on the command line using Pygments
My thoughts on python2.6 command line app template
Operate Route53 on the command line using AWS-CLI.
Think about the selective interface on the command line
[Python] The movement of the decorator that can be understood this time ② The decorator that receives the argument
I made a familiar function that can be used in statistics with Python
Linux command that can be used from today if you know it (Basic)
"Gazpacho", a scraping module that can be used more easily than Beautiful Soup
[Python3] Code that can be used when you want to change the extension of an image at once
Introduced "Glances" command, a monitoring tool that can be understood at a glance, on Mac
Basic algorithms that can be used in competition pros
Japanese can be used with Python in Docker environment
ANTs image registration that can be used in 5 minutes
How to filter foreign keys that can be selected on the Django admin screen
[Python] Introduction to web scraping | Summary of methods that can be used with webdriver
How to pass arguments when invoking python script from blender on the command line
Qiita can now be viewed on the console (shell)
Deep learning course that can be crushed on site
Steps to use the AWS command line interface (Python / awscli) on Mac OS X
List the classes that can be referenced by ObjCClass
[Redash] Standard library cannot be used in python function
Simple statistics that can be used to analyze the effect of measures on EC sites and codes that can be used in jupyter notebook
[Django] Field names, user registration, and login methods that can be used in the User model
[Python3] Code that can be used when you want to resize images in folder units
[Python] A program to find the number of apples and oranges that can be harvested
How to set variables that can be used throughout the Django app-useful for templates, etc.-
Why can I use the module by importing with python?
Search for large files on Linux from the command line
[Python] Variadic arguments can be used when unpacking iterable elements
SSD 1306 OLED can be used with Raspberry Pi + python (Note)
I investigated the pretreatment that can be done with PyCaret
About the Python module venv
Run unix command on python