Save the binary file in Python

I often download wallpaper images of a certain wallpaper site in a batch with ** Python **, or download material images of a certain free material site in a batch, but if it is a UNIX environment at that time,

import os
os.system("wget -O sample.jpg http://sample.org/sample.img")

It was easy to do with ** wget **, but wget cannot be used in a windows environment. So I tried to find out how to save the binaries. It seems to do as follows.


import shutil
import requests

URL = "http://sample.org/sample.img"

res = requests.get(URL,stream=True)
with open(filepath,"wb") as fp:
    shutil.copyfileobj(res.raw,fp)

As mentioned above, the raw (HTTPResponse object using urllib3) in the Response object of the ** requests ** module, The format is * save by copying to the target file object using the copyfileobj function of the ** shutil ** module.

If you use this, for example, when you want to download image files from a certain wallpaper site at once,

import shutil
import requests
from BeautifulSoup import BeautifulSoup

URL = "http://sample.org/"
targets = []

soup = BeautifulSoup(requests.get(URL).text)
for link in soup.findAll("a"):
    if link.get("href").endswith(".jpg "):
        targets.append(link.get("href"))

for target in targets:
    res = requests.get(target)
    with open(target.split("/")[-1], "wb") as fp:
        shutil.copyfileobj(res.raw,fp)

You can do it by writing it like this. It's very convenient, so I think it's a good idea to script it.

Recommended Posts

Save the binary file in Python
Download the file in Python
Create a binary file in Python
Read the file line by line in Python
Read the file line by line in Python
[Python] Read the specified line in the file
File operations in Python
File processing in Python
Binary search in Python
File operations in Python
Binary search in Python (binary search)
Find the difference in Python
Binary search in Python / C ++
Algorithm in Python (binary search)
Download the file while viewing the progress in Python 3.x
Write a binary search in Python
Extract the xz file with python
Save the pystan model and results in a pickle file
Getting the arXiv API in Python
File / folder path manipulation in Python
[Python] Open the csv file in the folder specified by pandas
Python in the browser: Brython's recommendation
Hit the Sesami API in Python
Linebot creation & file sharing in Python
Change the standard output destination to a file in Python
Get the desktop path in Python
Get the MIME type in Python and determine the file format
Get the script path in Python
In the python command python points to python3.8
Implement the Singleton pattern in Python
[Note] Import of a file in the parent directory in Python
The story of the "hole" in the file
Hit the web API in Python
After enabling the python virtual environment in the batch file, run the python file
Extract the targz file using python
I wrote the queue in Python
Calculate the previous month in Python
Examine the object's class in python
Get the desktop path in Python
ORC, Parquet file operations in Python
Save a YAML-formatted file in PyYAML
Get the host name in Python
Google search for the last line of the file in Python
Access the Twitter API in Python
The first step in Python Matplotlib
I wrote the stack in Python
Master the weakref module in Python
Save the Pydrive authentication file in a different directory from the script
Sort the file names obtained by Python glob in numerical order
Read a file in Python with a relative path from the program
[Python3] Save the mean and covariance matrix in json with pandas
Get the formula in an excel file as a string in Python
Trial of writing the configuration file in Python instead of .ini etc.
Learn the design pattern "Prototype" in Python
Load the remote Python SDK in IntelliJ
Try using the Wunderlist API in Python
Exclusive control with lock file in Python
Learn the design pattern "Flyweight" in Python
Try using the Kraken API in Python
Learn the design pattern "Memento" in Python
Learn the design pattern "Proxy" in Python