[Python / Selenium] XPath

Here's some useful knowledge to keep in mind when scraping with Selenium.

XPath You will need it to get the Element. The image below is easy to understand.

image.png Required for crawler creation! XPATH notation summary --Qiita

XPath Helper There is an extension that is very useful for identifying XPATHs, so I'll list it here. XPath Helper

You can get the XPath by holding down the Shift key and moving the cursor. image.png Chrome Extension! Installing and Using XPath Helper \ | WATLAB -Python, Signal Processing, AI-

Omit

XPath can also be omitted as shown below.

driver.find_elements(By.XPATH,'//div[@class="products"]')

Webdriver method

If you look at the article below, I think there is no problem. Selenium webdriver Summary of frequently used operation methods --Qiita

The methods I often use are summarized below.

driver.get

#Specify the URL to use
driver.get('url')

driver.back

#Return to the previous page
driver.back()

driver.quit It is possible to close the window.

driver.quit()

driver.execute_script I also use this quite a lot.

driver.execute_script('javascript:smarty.product.pager(2);')

Working with Selenium in Python and executing JavaScript embedded in html --Qiita

driver.find_elements As introduced above, it can be used as follows.

driver.find_elements(By.XPATH,'//div[@class="p-products"]')

Recommended Posts

[Python / Selenium] XPath
ScreenShot with Selenium (Python)
Scraping with Selenium [Python]
Python web scraping selenium
python selenium chromedriver beautifulsoup
Scraping with selenium in Python
Scraping with Selenium + Python Part 1
Python
Python: Working with Firefox with selenium
Python Selenium Dynamic download wait
selenium
Scraping with Selenium in Python
Start to Selenium using python
Selenium WebDriver + Firefox49 (provisional) (Python)
Web scraping using Selenium (Python)
Scraping with Selenium + Python Part 2
[Python + Selenium] Tips for scraping
Challenge Python3 and Selenium Webdriver
Scraping with Selenium in Python (Basic)
Scraping with Python, Selenium and Chromedriver
selenium case study summary python pyCharm
ElasticSearch + Kibana + Selenium + Python for SEO
Write selenium test code in python
Selenium and python to open google
kafka python
Python basics ⑤
python + lottery 6
Python Summary
Built-in python
Get html from element with Python selenium
Python comprehension
Python technique
Studying python
Python 2.7 Countdown
Python memorandum
Python FlowFishMaster
Python service
python tips
python function ①
Python basics
Python memo
ufo-> python (3)
WebUI test with Python2.6 + Selenium 2.44.0 --profile setting
Python + Selenium Frequently used operation method summary
Python comprehension
install python
Python Singleton
Python basics ④
Python Memorandum 2
python memo
Python Jinja2
Python increment
atCoder 173 Python
[Python] function
Python installation
python tips
Installing Python 3.4.3.
Try python
Python memo
Python iterative
Python algorithm