Python's GzipFile class is read via BufferedReader
That's why I'm talking about what happened, but I noticed that I was writing code using GzipFile, so it's a memo.
What is GzipFile?
A class that handles gzip in Python.
You can read and write a local Gzip file by passing the file name to the constructor, and a "like" Gzip object by passing fileobj.
Where BufferedReader is used
In in the constructor of GzipFile, as shown in the figure below.
- Actual ioClass (argument fileobj or file obtained from file name)
- Wrap with _GzipReader to actually read
- In addition, wrap it with BufferedReader
When reading a GzipFile, it goes through the BufferedReader (*).
- To be more precise, a class called _PaddedFile is sandwiched between _GzipReader and IoClass.
When you want to change the size of the buffer
GzipFile uses the default (io.DEFAULT_BUFFER_SIZE) for the BufferedReader's buffer size.
Depending on the io to be read, you may want to buffer in larger units (*). However, it seems that the buffer size cannot be changed by the function of GzipFile, so let's pass BufferedReader to fileobj of GzipFile.
- For example, if the unit to download GCS is small (several KB units), [performance will drop](https://cloud.google.com/blog/ja/products/gcp/optimizing-your-cloud- storage-performance-google-cloud-performance-atlas).