tse --Introduction to Text Stream Editor in Python

Introduction

This article is the 12th day article of Shell Script Advent Calendar 2015.

Here, the text formatting tool "tse" by Python created by Mr. Ishimoto @atsuoishimoto I would like to introduce.

Execution environment

• Mac OSX 10.11.1
• Python 3.5.0

What is tse?

You can understand the details by looking at the slides announced at this year's PyCon JP 2015. tse --Text Formatting Utility with Python

Super simply What is tse? "Python is said to be unsuitable for one-liner, but I still want to do text processing or Python." In response to the request "CLI tools for doing things like ** sed ** and ** awk ** in Python" is.

You can install it with pip.

pip install tse

Good points of tse

• ** sys **, ** re **, ** os **, ** os.path ** are automatically imported
• If you specify the required modules, import them before execution
• Input / output encoding specification

After that, I think it's faster to see the actual processing, so I'll cover some of them.

• I used Past Questions of Shell Art Study Group.

Challenge the shell art study session problem

The 17th Junkudo is a study session that shell art has taken over

https://blog.ueda.asia/?p=6454

Q1

The following data

$ cat data1
a 1
b 4
a 2
a 3
b 5

Let's convert it as follows.

a 1 2 3
b 4 5

If you have the spare capacity, try the following JSON format.

{a:[1,2,3],b:[4,5]}

answer

• Initialize dictionary with --begin option
• With the --end option, you can output the dictionary key and value respectively.

$cat data1 | tse -b 'd={"a": "", "b": ""}' -s '' 'd[L1] = d[L1] + " " + L2' -e 'for k, v in sorted(d.items()):print(k + v)'
a 1 2 3
b 4 5

• json format is OK if you do json.dumps () the dictionary at the end
• Import json with -module option

$cat data1 | tse -m json -b 'd={"a": [], "b": []}' -s '' 'd[L1].append(int(L2))' -e 'print(json.dumps(sorted(d.items())))'
[["a", [1, 2, 3]], ["b", [4, 5]]]

18th Do you want to add garlic? Shell art study session

https://blog.ueda.asia/?p=6877

Q1 In the following file, the first column is the key and the second column is the value, but look for a key that has both "Oton" and "Okan" values.

$ cat text 
 001 Oton
 001 Oton
 001 Akan
 002 Okan
 003 Oton
 003 Kettle
 003 Okan
 004 Okan
 005 Oton
 005 oranges
 005 Akan

answer

• With the -s option, you can specify a regular expression that includes" Oton "or" Okan ".

$uniq text | tse -s 'Oton|Okan' 'print L1' | uniq -d
003

19th Shell Art 3rd Anniversary Study Session

https://blog.ueda.asia/?p=7146

Q2 Two natural numbers

$ echo 1 4

After outputting with echo like

4
3
2
1
2
3
4

Try filling in the numbers in between.

answer

• It's a bit verbose, but you can simply repeat it with the for statement.

$echo 1 4 | tse -s '' 'for s in range(int(L1), int(L2)+1)[::-1]: print(s)' 'for s in range(int(L1)+1, int(L2)+1): print(s)'

in conclusion

I tried to use tse roughly, but it became easy to write one liner with the syntax of Python that I am accustomed to. I feel like I was able to get used to the one-liner field that I was scared to avoid ^^

Tomorrow's turn is @zayarwinttun!