Measures to be taken when garbled characters when trying to redirect / pipe the result of aws-cli

If you transfer the command from aws cli like this, it will look like this (command and output are dummy).

$ aws s3 ls --recursive bucket/photo/1/
2014-05-06 16:45:30      11979 photo/1/20140506164529_Screenshot 2014-01-28 15.27.48.png

However, if you change it to the following command, for example, it becomes annoying.

$ aws s3 ls --recursive bucket/photo/1/ | tee /dev/null
2014-05-06 16:45:30      11979 photo/1/20140506164529_????????? 2014-01-28 15.27.48.pn

Haunted!

The same phenomenon occurs when the file is output by redirect.

Countermeasures

The cause is in the Python implementation that implements the ʻaws` command. Python adjusts the character code according to the output destination, but in the case of redirect or pipe, since the character code of the output destination is not known, it is forced to output in ASCII! !! !! Extra imitation! !! !! </ del>

As a countermeasure, you can define an environment variable / shell variable called PYTHONIOENCODING.

$ PYTHONIOENCODING=UTF-8 aws s3 ls --recursive bucket/photo/1/ | tee /dev/null
2014-05-06 16:45:30      11979 photo/1/20140506164529_Screenshot 2014-01-28 15.27.48.png

Recommended Posts

Measures to be taken when garbled characters when trying to redirect / pipe the result of aws-cli
Measures to be taken when Suspicious Operation occurs in HttpResponse Redirect
Record of actions to be taken when google_image_download cannot be used
When you want to save the result of the callback function somewhere
The story of trying to reconnect the client
A memo of misunderstanding when trying to load the entire self-made module with Python3
I want to be notified of the connection environment when the Raspberry Pi connects to the network
I want to grep the execution result of strace
Be careful when differentiating the eigenvectors of a matrix