How to scrape at speed per second with Python Selenium

Get selector with XPATH

Quickly specify the XPATH on the page using the Chrome extension xPath Finder.

image.png

And send the input items there with send_keys etc.

The following is an excerpt of the SBI SECURITIES login script.

try:
    browser.get('https://www.sbisec.co.jp/ETGate')
    username_field = browser.find_element_by_xpath('/html/body/table/tbody/tr[1]/td[2]/form/div/div/div/dl/dd[1]/div/input')
    username_field.clear()
    username_field.send_keys(username)
    password_field = browser.find_element_by_xpath("/html/body/table/tbody/tr[1]/td[2]/form/div/div/div/dl/dd[2]/div/input")
    password_field.send_keys(password)
    login_button = browser.find_element_by_xpath("/html/body/table/tbody/tr[1]/td[2]/form/div/div/div/p[2]/a/input")
    login_button.click()#click login
    time.sleep(5)

With this pattern, projects using Selenium will progress at once.

Change log

--2020/06/06 Newly created

Recommended Posts

How to scrape at speed per second with Python Selenium
How to scrape image data from flickr with python
How to switch to smartphone mode with Python + Selenium + Chrome
How to get started with Python
How to use FTP with Python
How to speed up Python calculations
How to calculate date with python
[Python] How to get divisors of natural numbers at high speed
How to automatically install Chrome Driver for Chrome version with Python + Selenium + Chrome
Convert memo at once with Python 2to3
How to work with BigQuery in Python
How to display python Japanese with lolipop
How to scrape websites created with SPA
How to enter Japanese with Python curses
[Python] How to deal with module errors
How to install python3 with docker centos
[Python] How to set the (client) window size inside the browser with Selenium
How to upload with Heroku, Flask, Python, Git (4)
How to create large files at high speed
How to enjoy programming with Minecraft (Ruby, Python)
[REAPER] How to play with Reascript in Python
How to do multi-core parallel processing with python
How to crop an image with Python + OpenCV
How to scrape horse racing data with BeautifulSoup
How to deal with SessionNotCreatedException when using Selenium
Log in to Yahoo Business with Selenium Python
How to measure execution time with Python Part 1
How to use tkinter with python in pyenv
[Python] How to handle Japanese characters with openCV
[Python] How to compare datetime with timezone added
How to measure execution time with Python Part 2
[Python] How to save images on the Web at once with Beautiful Soup
Scraping 2 How to scrape
How to install Python
How to debug selenium
ScreenShot with Selenium (Python)
Scraping with Selenium [Python]
How to convert / restore a string with [] in python
How to do hash calculation with salt in Python
[Introduction to Python] How to iterate with the range function?
How to upload with Heroku, Flask, Python, Git (Part 3)
How to run tests in bulk with Python unittest
[Python] How to specify the download location with youtube-dl
Easy way to scrape with python using Google Colab
How to measure mp3 file playback time with python
How to use python interactive mode with git bash
I was addicted to scraping with Selenium (+ Python) in 2020
How to upload with Heroku, Flask, Python, Git (Part 1)
[Python] How to deal with pandas read_html read error
How to download files from Selenium in Python in Chrome
How to upload with Heroku, Flask, Python, Git (Part 2)
How to get mouse wheel verdict with Python curses
[Python] How to rewrite the table style with python-pptx [python-pptx]
[Python] How to create a 2D histogram with Matplotlib
Calculate Gaussian kernel at explosive speed even with python
How to operate Discord API with Python (bot registration)
[Python] How to draw a scatter plot with Matplotlib
How to operate Firefox with selenium on Windows Memo
Perform half-width / full-width conversion at high speed with Python
Connect to BigQuery with Python
I tried to simulate how the infection spreads with Python