scraping the Nikkei 225 with playwright-python

Installation

sudo apt update
sudo apt install -y libwoff1 libopus0 libwebp6 libwebpdemux2 libenchant1c2a libgudev-1.0-0 libsecret-1-0 libhyphen0 libgdk-pixbuf2.0-0 libegl1 libnotify4 libxslt1.1 libevent-2.1-7 libgles2 libxcomposite1 libatk1.0-0 libatk-bridge2.0-0 libepoxy0 libgtk-3-0 libharfbuzz-icu0 libgstreamer-gl1.0-0 libgstreamer-plugins-bad1.0-0 gstreamer1.0-plugins-good gstreamer1.0-libav libnss3 libxss1 libasound2 fonts-noto-color-emoji libxtst6 libdbus-glib-1-2 libxt6 ffmpeg xvfb

Install Japanese font

sudo apt install fonts-ipafont-gothic fonts-ipafont-mincho

Install playwright

sudo apt install python3-pip
pip3 install playwright

#Install chromium / Firefox / WebKit
python3 -m playwright install

screenshot

from playwright import sync_playwright

with sync_playwright() as p:
    for browser_type in [p.chromium, p.firefox, p.webkit]:
        browser = browser_type.launch()
        page = browser.newPage()
        page.goto("http://whatsmyuseragent.org/")
        page.screenshot(path=f"example-{browser_type.name}.png ")
        browser.close()

Scraping the Nikkei 225

from playwright import sync_playwright

with sync_playwright() as p:
    browser = p.chromium.launch()
    page = browser.newPage()
    page.goto("http://www.nikkei.com/markets/kabu/")
    # html = page.content()
    print(page.innerText("span.mkc-stock_prices"))
    browser.close()

Recommended Posts

scraping the Nikkei 225 with playwright-python
Scraping with selenium
Scraping with selenium ~ 2 ~
Scraping with Python
Scraping with Python
Scraping with Selenium
Scraping the holojour and displaying it with CLI
Successful scraping with Selenium
Scraping with Python (preparation)
Try scraping with Python.
Scraping with Python + PhantomJS
Scraping with scrapy shell
Scraping with Selenium [Python]
Scraping with Python + PyQuery
Scraping with Beautiful Soup
Scraping RSS with Python
I tried scraping with Python
Automatically download images with scraping
Try scraping the data of COVID-19 in Tokyo with Python
Web scraping with python + JupyterLab
Scraping with selenium in Python
Scraping with Selenium + Python Part 1
Scraping with chromedriver in python
Festive scraping with Python, scrapy
Kill the process with sudo kill -9
Save images with web scraping
Scraping with Selenium in Python
Easy web scraping with Scrapy
Scraping with Tor in Python
Guess the password with klee
gethostbyaddr () communicates with the outside
Prediction of Nikkei 225 with Pytorch 2
Scraping weather forecast with python
Check the code with flake8
Scraping with Selenium + Python Part 2
Calibrate the model with PyCaret
Prediction of Nikkei 225 with Pytorch
Call the API with python3.
I tried scraping with python
Web scraping beginner with python
I-town page scraping with selenium
Table scraping with Beautiful Soup
Stock price plummeted with "new corona"? I tried to get the Nikkei Stock Average by web scraping
I tried scraping the ranking of Qiita Advent Calendar with Python
Decrypt the QR code with CNN
Try scraping with Python + Beautiful Soup
Scraping multiple pages with Beautiful Soup
Scraping with Node, Ruby and Python
Color the integration interval with matplotlib.pyplot
Web scraping with Python ① (Scraping prior knowledge)
Follow the file hierarchy with fts
Specifying the date with the Twitter API
Scraping with Python, Selenium and Chromedriver
Pave the road with combinatorial optimization
Scraping Alexa's web rank with pyQuery
Web scraping with Python First step
I tried web scraping with python.
Run the app with Flask + Heroku
Scraping with Python and Beautiful Soup
Get the weather with Python requests
Get the weather with Python requests 2