How to end Python's infinite scroll scraping

While implementing scraping of a web page with infinite scrolling in Python, I stumbled because I didn't know how to finish it.

sample

>>> t = 0
>>> start_at = time.time()
>>> t = time.time() - start_at
>>> t == 5
>>> break

Commentary

start_at is the time that has elapsed since it was executed. t is the difference When t reaches 5 (5 seconds have passed), the process ends.

reference https://note.nkmk.me/python-datetime-timedelta-measure-time/ https://ai-inter1.com/python-selenium/#st-toc-h-14

Recommended Posts

How to end Python's infinite scroll scraping
Scraping 2 How to scrape
How to use Python's logging module
How to use Python's Context Manager
How to crawl pages that scroll infinitely
How to implement Scroll View in pythonista 1
How to use TouchDesigner Python's external module
How to read an array with Python's ConfigParser
Scraping with Python-Selenium is old! ?? ・ ・ ・ How to use Pyppeteer
How to use xml.etree.ElementTree
How to use Python-shell
How to use tf.data
How to use virtualenv
How to use Seaboan
How to use image-match
How to use shogun
How to install Python
How to use Pandas 2
How to install pip
How to use Virtualenv
How to use numpy.vectorize
How to update easy_install
How to install archlinux
How to restart gunicorn
How to install python
How to virtual host
How to debug selenium
How to use partial
How to use Bio.Phylo
How to read JSON
How to use SymPy
How to use x-means
How to use WikiExtractor.py
How to update Spyder
How to use IPython
How to install BayesOpt
How to use virtualenv
How to use Matplotlib
How to use iptables
How to use numpy
How to use TokyoTechFes2015
How to use venv
How to use dictionary {}
How to use Pyenv
How to grow dotfiles
How to use list []
How to use python-kabusapi
"How to count Fukashigi"
How to install Nbextensions
How to use OptParse
How to use return
How to install Prover9
How to use dotenv
How to operate NumPy
How to use pyenv-virtualenv
How to use Go.mod
How to use imutils
How to use import
How to support x-nullable for swagger in python's jsonschema library
How to kill a process instantly with Python's Process Pool Executor
How to print characters as a table with Python's print function