[Python] Introduction to web scraping | Summary of methods that can be used with webdriver

[Python] Introduction to web scraping | Summary of methods that can be used with webdriver

List of major methods available in webdriver for selenium module

** ■ Methods that can be used without specifying elements ** --get (opens the specified page) --back --forward --refresh --current_url (get current URL) --title (Get page title) --page_source (Get page source) --close (closes the current window) --quit (close all windows) --maximize_window


** ■ Element specification method **

** ▼ 1 element **

** ▼ Multiple elements **


** ■ Method used in combination with element specification ** --click (click an element) --send_keys (text input) --getText --get_attribute --is_enabled (determines if the element is enabled) --is_selected (whether the element is selected)

How to use

Install webdriver for selenium module.

Used as a set with webdriver.Chrome ('chromedriver.exe'). (* The chromedriver.exe file is in the same hierarchy as the source code)

python


from selenium import webdriver
browser = webdriver.Chrome('chromedriver.exe')

### ■ Methods that can be used without specifying elements Execute for browser object.

Example: browser.get (https: // ~) └ Open the specified URL


### ■ Method of element specification Example: `browser.find_element_by_id ('headline')` └ Get the element that contains the specified element └ In the above example id ='headline'
### ■ Method used in combination with element specification Example 1: `driver.find_element_by_id ('btn'). click ()` └ Execute method for specified element └ In the above example, click the element with id ='btn'

Example 2: driver.find_element_by_id('useName').send_keys("myname") └ In the above example, enter "myname" in text in the element of id ='useName'

Recommended Posts

[Python] Introduction to web scraping | Summary of methods that can be used with webdriver
Summary of statistical data analysis methods using Python that can be used in business
Basic summary of scraping with Requests that beginners can absolutely understand [Python]
Python knowledge notes that can be used with AtCoder
Format summary of formats that can be serialized with gensim
[Python] Introduction to scraping | Program to open web pages (selenium webdriver)
[Introduction to Python] Summary of functions and methods that frequently appear in Python [Problem format]
List of tools that can be used to easily try sentiment analysis of Japanese sentences in Python (try with google colab)
[Raspberry Pi] Scraping of web pages that cannot be obtained with python requests + Beautiful Soup
Python standard input summary that can be used in competition pro
Comparison of 4 styles that can be passed to seaborn with set_context
Mathematical optimization that can be used for free work with Python + PuLP
File types that can be used with Go
Introduction to Tornado (1): Python web framework started with Tornado
Selenium webdriver Summary of frequently used operation methods
Introduction to Web Scraping
Summary of scikit-learn data sources that can be used when writing analysis articles
I made a familiar function that can be used in statistics with Python
How to install a Python library that can be used by pharmaceutical companies
[Python3] Code that can be used when you want to change the extension of an image at once
Japanese can be used with Python in Docker environment
Summary of Pandas methods used when extracting data [Python]
[Chapter 3] Introduction to Python with 100 knocks of language processing
[Chapter 2] Introduction to Python with 100 knocks of language processing
[Chapter 4] Introduction to Python with 100 knocks of language processing
Convert images from FlyCapture SDK to a form that can be used with openCV
Introduction of automatic image collection package "icrawler" (0.6.3) that can be used during machine learning
Web scraping with python + JupyterLab
Web scraping beginner with python
Processing of python3 that seems to be usable in paiza
20200329_Introduction to Data Analysis with Python Second Edition Personal Summary
An introduction to Python that even monkeys can understand (Part 3)
SSD 1306 OLED can be used with Raspberry Pi + python (Note)
Scripts that can be used when using bottle in Python
[Raspi4; Introduction to Sound] Stable recording of sound input with python ♪
An introduction to Python that even monkeys can understand (Part 1)
An introduction to Python that even monkeys can understand (Part 2)
[Introduction to Udemy Python3 + Application] 51. Be careful with default arguments
Module summary that automates and assists WebDriver installation with Python
How to start a simple WEB server that can execute cgi of php and python
Can be used with AtCoder! A collection of techniques for drawing short code in Python!
[Python3] Code that can be used when you want to resize images in folder units
Summary of Python implementation know-how and tips that AI engineers want to be careful about
[Python] A program to find the number of apples and oranges that can be harvested
[Python] Make a graph that can be moved around with Plotly
IPynb scoring system made with TA of Introduction to Programming (Python)
Investigation of DC power supplies that can be controlled by Python
Web scraping with Python ① (Scraping prior knowledge)
I tried to expand the database so that it can be used with PES analysis software
Web scraping with Python First step
I tried web scraping with python.
Here's a summary of things that might be useful when dealing with complex numbers in Python
Understand the probabilities and statistics that can be used for progress management with a python program
Easy-to-understand explanation of Python Web application (Django) even for beginners (5) [Introduction to DB operation with Django shell]
Install packages that need to be compiled with Python3 with pip [Windows]
About the matter that torch summary can be really used when building a model with Pytorch
[Python] A program that finds the maximum number of toys that can be purchased with your money
From the introduction of JUMAN ++ to morphological analysis of Japanese with Python
[Introduction to Udemy Python 3 + Application] Summary
Python standard module that can be used on the command line
Introduction of "scikit-mobility", a library that allows you to easily analyze human flow data with Python (Part 1)