A script that opens the URLs written in CSV in order and takes a full screen screenshot

I wrote a little bit when I needed it.

environment

module

code

I'm assuming this kind of configuration.

.
├── data
│   └── urls.csv
├── utilities
│   └── read_csv.py
└── test_screenshot.py

Main processing

It is a process to open the URL written in the url column of csv and take a screenshot, and open it to take a screenshot.

The width is fixed at 1920px, but it is possible to support scrolling as well as height.

test_screenshot.py


import time

import pytest
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from webdriver_manager.chrome import ChromeDriverManager

from utilities.read_csv import read_csv_data


class TestScreenshot():

    datalist = read_csv_data("./data/urls.csv")

    @classmethod
    def setup_class(cls):
        options = Options()
        options.add_argument('--headless')
        options.add_argument('--hide-scrollbars')
        cls.driver = webdriver.Chrome(executable_path=ChromeDriverManager().install(), options=options)
        cls.driver.maximize_window()

    @pytest.mark.parametrize("id, url", datalist)
    def test_reserve_multi(self, id, url):
        driver = self.driver
        driver.get(url)
        time.sleep(3)
        page_height = driver.execute_script('return document.body.scrollHeight')
        driver.set_window_size(1920, page_height)
        driver.save_screenshot(id + '.png')

CSV

It has two columns, the ʻid column and the ʻurl column. ʻId` is also used in the screenshot file name.

urls.csv


id,url
1,https://www.hoge.co.jp/
2,https://www.hoge.co.jp/pageA/
3,https://www.hoge.co.jp/pageB/
4,https://www.hoge.co.jp/pageC/
5,https://www.hoge.co.jp/pageD/

CSV reading process

read_csv.py


import csv

def read_csv_data(csv_path):
    rows = []
    with open(str(csv_path), encoding="utf-8") as csv_data:
        content = csv.reader(csv_data)
        next(content, None)
        for row in content:
            rows.append(row)
        print(rows)
        return rows

How to use

> pytest test_screenshot.py

important point

I actually tried using it, but I couldn't get it in full screen on a site that was moving by making full use of JS.

Probably not limited to this method, I think it is quite strict.

reference

-Python: Take a screenshot of the entire web page with Selenium + Headless Chrome --CUBE SUGAR CONTAINER ――The process of taking a full-screen screenshot is almost imitated.

Recommended Posts

A script that opens the URLs written in CSV in order and takes a full screen screenshot
A Python script that reads a SQL file, executes BigQuery and saves the csv
Process the files in the folder in order with a shell script
I wrote a script that splits the image in two
Process the contents of the file in order with a shell script
The one that divides the csv file, reads it, and processes it in parallel
[Note] Based on the latitude and longitude of the CSV file, we created a script that extracts data in the target range and adds a mesh code.
Run the Python interpreter in a script
Define a division value in Django and easily reflect it on the screen
A Python script that crawls RSS in Azure Status and posts it to Hipchat
A note that runs an external program in Python and parses the resulting line
A solution to the problem that files containing [and] are not listed in glob.glob ()