Python + Selenium Frequently used operation method summary

Introduction

We use Selenium for web scraping. At the time of development, I forgot to ask, "What? How do you write this?", And decided to cover only the items I'm investigating in this article.

ChromeOptions In rare cases, the following timeout error will occur if the required options are not set.

Timed out receiving message from renderer: 600.000

About this, Stack Overflow Answerでは、以下のオプション設定を行えば問題ないそうです。(ちなみに「役に立たない増え続ける引数オプション」と呼ばれており、まさにその通り・・・)

options.addArguments("start-maximized"); 
options.addArguments("enable-automation"); 
options.addArguments("--headless"); 
options.addArguments("--no-sandbox");
options.addArguments("--disable-infobars"); 
options.addArguments("--disable-dev-shm-usage"); 
options.addArguments("--disable-browser-side-navigation"); 
options.addArguments("--disable-gpu");
driver = new ChromeDriver(options);

Explicit wait processing

The WebDriverWait.until method allows you to set an explicit wait time for any HTML element to wait for a particular state. Click here for details (https://kurozumi.github.io/selenium-python/api.html#module-selenium.webdriver.support.expected_conditions).

Judgment if clickable
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.ID, "cnfm_btn")))
Judgment whether it can be selected
WebDriverWait(driver, 10).until(EC.element_located_to_be_selected((By.ID, "cnfm_btn")))

Recommended Posts

Python + Selenium Frequently used operation method summary
Selenium webdriver Summary of frequently used operation methods
Python directory operation summary
[Python/Django] Summary of frequently used commands (3) <Operation of PostgreSQL>
Summary of frequently used Python arrays (for myself)
Note the frequently used options in Python + Selenium + Chrome
[Python] Frequently used library code
Python frequently used code snippets
8 Frequently Used Commands in Python Django
selenium case study summary python pyCharm
[Linux] Frequently used Linux commands (file operation)
[Anaconda3] Summary of frequently used commands
[Linux] Frequently used Linux commands (folder operation)
Python Summary
Python summary
Summary of frequently used commands of django (beginner)
Summary of frequently used commands in matplotlib
[python] Frequently used techniques in machine learning
[Python/Django] Summary of frequently used commands (4) -Part 2- <Production operation: Amazon EC2 (Amazon Linux 2)>
[Python/Django] Summary of frequently used commands (4) -Part 1- <Production operation: Amazon EC2 (Amazon Linux 2)>
[Python / Selenium] XPath
Correspondence summary of array operation of ruby and python
Automatic operation of Chrome with Python + Selenium + pandas
Python tutorial summary
Python OS operation
Frequently used methods of Selenium and Beautiful Soup
[Python] Matrix operation
python related summary
[Python/Django] Summary of frequently used commands (2) <Installing packages>
Summary of frequently used commands (with petit commentary)
Johnson method (python)
Python basics summary
[Python] Semi-Lagrange method
Summary of Pandas methods used when extracting data [Python]
[Python] Summary of table creation method using DataFrame (pandas)
Comparison table of frequently used processes of Python and Clojure
Automated testing method combining Beautiful Soup and Selenium (Python)
pyenv Frequently used commands
[Python] Operation of enumerate
Summary about Python scraping
Python Django tutorial summary
Frequently used tmux commands
Frequently used Linux commands
Kernel Method with Python
Python installation method Windows
Frequently used Linux commands
Frequently used linux commands
Summary about Python3 + OpenCV3
Python function argument summary
ScreenShot with Selenium (Python)
Simplex method (simplex method) in Python
Python AI framework summary
Python iteration related summary
Private method in python
Python logical operation stumbling
Summary of Python arguments
Scraping with Selenium [Python]
Python web scraping selenium
Python decorator operation memo
Summary of test method
[python] Array slice operation