Scraping with Selenium

I wanted to automate it, so I did a lot of research, but it seems that some people made it in advance, but honestly, I'm still a beginner in programming, so I don't know what I'm writing.

So, this time I studied using selenium so that I could do the automation myself.

Execution environment

Articles referred to this time

[Selenium] python beginners have modified the Instagram automatic like tool without permission [[Python x Selenium] We have created an automation tool that allows you to search for tags on Instagram and like all the acquired photos](https://review-of-my-life.blogspot.com/2017/12/selenium- instagram.html) [Python] Selenium usage memo Complete automatic operation of Chrome with Python + Selenium Scraping with Selenium in Python (Basic)

Clogged place

I downloaded the chromedriver and tried to use it on python, but I couldn't use it because I didn't know the developer.

So, this time I used chromedriver_binary to make it usable.

$ pip install chromedriver-binary
import chromedriver_binary

You can now use it.

A memo of knowledge I did not know

What is selenium

selenium is a tool for automating screen operations of web applications. It is also used for automated web testing and scraping.

Here, the difference between scraping and crawling is that they both collect data on the web. Scraping is processing to make it easier to process and collecting data on the web, and crawling is just collecting data on the web.

Summary

I haven't reached automation at all yet, but I would like to study and be able to do automation myself.

Recommended Posts

Scraping with selenium
Scraping with selenium ~ 2 ~
Scraping with Selenium
Successful scraping with Selenium
Scraping with Selenium [Python]
Scraping with selenium in Python
Scraping with Selenium + Python Part 1
Scraping with Selenium in Python
Scraping with Selenium + Python Part 2
I-town page scraping with selenium
Scraping with Python
Scraping with Python
Scraping with Selenium in Python (Basic)
Scraping with Python, Selenium and Chromedriver
Beginning with Selenium
Practice web scraping with Python and Selenium
Scraping with Python (preparation)
Try scraping with Python.
Scraping with Python + PhantomJS
Scraping with scrapy shell
ScreenShot with Selenium (Python)
Python web scraping selenium
Scraping with Python + PyQuery
Scraping with Beautiful Soup
Scraping RSS with Python
Serverless scraping using selenium with [AWS Lambda] -Part 1-
I tried scraping with Python
Automatically download images with scraping
Web scraping with python + JupyterLab
Scraping with chromedriver in python
Save images with web scraping
Python: Working with Firefox with selenium
I was addicted to scraping with Selenium (+ Python) in 2020
Easy web scraping with Scrapy
Scraping with Tor in Python
Web scraping using Selenium (Python)
Scraping weather forecast with python
Memories of fighting with Selenium
scraping the Nikkei 225 with playwright-python
Try Selenium Grid with Docker
[Python + Selenium] Tips for scraping
I tried scraping with python
Web scraping beginner with python
Scraping 1
Table scraping with Beautiful Soup
Try scraping with Python + Beautiful Soup
Scraping multiple pages with Beautiful Soup
Scraping with Node, Ruby and Python
Web scraping with Python ① (Scraping prior knowledge)
Web scraping with BeautifulSoup4 (layered page)
Scraping Alexa's web rank with pyQuery
Summary of scraping relations (selenium, pyautogui)
Scraping pages with pagination with Beautiful Soup
Scraping with Beautiful Soup in 10 minutes
Make testing with Selenium more accessible
Let's do image scraping with Python
Get Qiita trends with Python scraping
"Scraping & machine learning with Python" Learning memo
Website scraping with Python's Beautiful Soup
Get weather information with Python & scraping
[Python, Selenium, PhantomJS] A story when scraping a website with lazy load