[Selenium] I want to display the browser by hitting the driver on the host OS from WSL

Introduction

WSL is very convenient in that you can use Linux commands while using Windows, but when it comes to Selenium, there is a drawback that it is difficult to check the behavior of the browser because the Ubuntu GUI cannot be used.

You can check the screen display at that point by launching the virtual display and taking a screenshot [^ 1], but it seems like there is nothing like seeing it actually moving.

As a result of various trials and errors, I succeeded in displaying the browser by hitting the driver on the host OS from WSL2 by using Selenium Grid, so I will share it.

environment

Host OS: Windows 10 Home 20H2 WSL: Ubuntu 18.04.5 on WSL2 Python: 3.7.5

Previous research

[Operating a web browser on the Windows side using Selenium from Python on WSL (Ubuntu16.04.4 LTS) – Laboradian](https://laboradian.com/operate-browser-on-win-by-python- of-wsl-via-selenium /) ⇒Since it was an article more than two years ago, I could not reproduce it in my environment. I get the error message Can not connect to the Service. It may be possible if the environment is prepared.

I wanted to operate the Chrome window on the Windows side from Selenium Webdriver or Puppeteer started on WSL2, but I can't-- Corredor ⇒Although it has not been solved yet, it may be helpful for those who are developing in languages other than Python because there are abundant references to previous research.

Method 1: Selenium3 (reliability: medium)

① Windows: Install Java (11 or higher).

See below for the procedure. If it is Java 10 or less, an error may occur. Installation method memo of OpenJDK11 --Qiita

② Windows: Download Selenium Server (Grid).

Go to the here page and download 3.141.59. The location of the downloaded file is not specified.

③ Windows: Download the browser driver.

Chrome Driver from here, geckodriver (Firefox) from here. Please download the driver with the same version ** as the browser installed on Windows **. Extract the downloaded zip and other files, and place the executable file (chromedriver.exe or geckodriver.exe) in the same folder ** as the jar file downloaded in ** ②.

④ Windows: Start Selenium Server.

In Selenium Grid 3, we need to set up two processes, Hub and Node, so we will launch two command prompts. First, start Hub by executing the following command at one command prompt.

java -jar [Storage location of jar file]\selenium-server-standalone-3.141.59.jar -role hub

If the following output is output and the process does not end, there is no problem.

13:11:40.423 INFO [GridLauncherV3.parse] - Selenium server version: 3.141.59, revision: e82be7d358
13:11:40.483 INFO [GridLauncherV3.lambda$buildLaunchers$5] - Launching Selenium Grid hub on port 4444
2020-11-23 13:11:40.848:INFO::main: Logging initialized @636ms to org.seleniumhq.jetty9.util.log.StdErrLog
13:11:41.814 INFO [Hub.start] - Selenium Grid hub is up and running
13:11:41.816 INFO [Hub.start] - Nodes should register to http://192.168.21.1:4444/grid/register/
13:11:41.816 INFO [Hub.start] - Clients should connect to http://192.168.21.1:4444/wd/hub

Then run the following command at the other command prompt to start Node. For [Hub URL], assign the URL that follows Clients should connect to in the above output.

java -jar [Storage location of jar file]\selenium-server-standalone-3.141.59.jar -role node -hub [Hub URL]:4444

If the following output is output and the process does not end, there is no problem.

13:11:59.609 INFO [GridLauncherV3.parse] - Selenium server version: 3.141.59, revision: e82be7d358
13:11:59.688 INFO [GridLauncherV3.lambda$buildLaunchers$7] - Launching a Selenium Grid node on port 31688
2020-11-23 13:12:00.999:INFO::main: Logging initialized @1598ms to org.seleniumhq.jetty9.util.log.StdErrLog
13:12:01.177 INFO [WebDriverServlet.<init>] - Initialising WebDriverServlet
13:12:01.231 INFO [SeleniumServer.boot] - Selenium Server is up and running on port 31688
13:12:01.231 INFO [GridLauncherV3.lambda$buildLaunchers$7] - Selenium Grid node is up and ready to register to the hub
13:12:01.654 INFO [SelfRegisteringRemote$1.run] - Starting auto registration thread. Will try to register every 5000 ms.
13:12:01.883 INFO [SelfRegisteringRemote.registerToHub] - Registering the node to the hub: http://192.168.21.1:4444/grid/register
13:12:02.317 INFO [SelfRegisteringRemote.registerToHub] - The node is registered to the hub and ready to use

⑤ WSL: Install Selenium 3.141.0.

pip install selenium==3.141.0

⑥ WSL: Create sample code.

sample.py


from selenium import webdriver

chrome_options = webdriver.ChromeOptions()
driver = webdriver.Remote(
    command_executor="http://192.168.21.1:4444/wd/hub",
    options=chrome_options
    )
driver.get("https://www.google.com/search?q=qiita")

For command_executor, specify the URL that followsClients should connect to. If you change ChromeOptions () to FirefoxOptions (), you will use geckodriver.

⑦ WSL: Execute the sample code.

python sample.py

If successful, it will look like this: selenium_demo.gif

Method 2: Selenium4 (reliability: high)

To tell the truth, the above method with Selenium 3 did not work well from the beginning, and I found a loophole to succeed by playing with the Selenium source, but I thought it would be a good idea to introduce it, so I put it in storage once. I was there. After that, I discovered how to implement it in Selenium 4, and when I wrote this article, I verified it again with Selenium 3 and it worked. There is a good opinion that "then I should introduce the one with higher certainty first", but since Selenium 4 is still a pre-release, I thought that it would be better if the stable version of Selenium 3 works well. It depends on the priority. Excuse me for the long introduction.

① Windows: Install Java (11 or higher).

Omitted because it is the same as Selenium3. If it is not Java 11 or higher, an error will definitely occur.

② Windows: Download Selenium Server (Grid) 4.

Go to the here page and download from the link labeled Latest Selenium 4 Alpha version.

③ Windows: Download the browser driver.

Same as Selenium3. The difference is that the executable does not have to be in the same location as the jar file.

④ Windows: Add the driver path to the system environment variable PATH.

This procedure is required only for Selenium 4. Open the setting screen by selecting [Start] → [Settings] and enter "environment variables" in the search box. Two candidates, "Edit environment variables" and "Edit system environment variables", will appear. Select "Edit Variable". スクリーンショット 2020-11-23 165701.png

Then the following screen will open, so press "Environment Variables". スクリーンショット 2020-11-23 165510.png When the "Environment Variables" screen opens, select the Path row in the" System Variables "table and press" Edit ". スクリーンショット 2020-11-23 165853.png When the "Edit Environment Variable Name" screen opens, click "New" and enter the path of the folder where the browser driver is stored. When the input is completed, confirm with the Enter key and press "OK". スクリーンショット 2020-11-23 170220.png Click "OK" on the "Environment Variables" screen and "System Properties" screen, close the "Settings" screen, and you're done.

⑤ Windows: Start Selenium Server.

Selenium Grid 4 has a mode called standalone, which allows you to launch all components from one command prompt.

Start a command prompt and execute the following command. Please note that the file name depends on the downloaded version.

java -jar [Storage location of jar file]\selenium-server-4.0.0-alpha-7.jar standalone

If the last line of the command output is as follows and the process does not end, there is no problem.

17:08:44.674 INFO [Standalone.execute] - Started Selenium standalone 4.0.0-alpha-7 (revision Unknown): http://172.26.240.1:4444

⑥ WSL: Install Selenium4.

pip install selenium==4.0.0a7

If you have Selenium installed, you can upgrade with the following command.

pip install -U --pre selenium

If the Python version is 3.8 or higher, Selenium == 4.0.0a7 installation will fail. In that case, you will either have to lower the Python version or install Selenium == 4.0.0a6. ** Please note that if you lower the version of Selenium, you also need to lower the version of Selenium Server **.

⑦ WSL: Create sample code.

Same as Selenium3. The URL set in command_executor may be different between Selenium 3 and 4, so change it if necessary.

⑧ WSL: Execute the sample code.

python sample.py

If successful, it will look like this:

selenium4_demo.gif

Afterword

I'm not familiar with Java at all, so when I ran the jar file and got an error, I was so confused that I almost cried, but as a result I was relieved that it worked. I think the points were ** Java version ** and ** system environment variable PATH **. We look forward to helping you.

Thank you for watching until the end.

References

Remote WebDriver client :: Documentation for Selenium Grid :: Documentation for Selenium

[^ 1]: The following page was helpful for this method. Python: Take a screenshot of the entire web page with Selenium + Headless Chrome-CUBE SUGAR CONTAINER

Recommended Posts

[Selenium] I want to display the browser by hitting the driver on the host OS from WSL
I want to display the progress bar
I want to display the progress in Python!
[Writing] I want to display the variables used in the program on the server side in real time on the browser and update them.
I want to change the symbolic link destination of / lib64 from / usr / lib64 to / my-lib64 on CentOS
I want to save the photos sent by LINE to S3
I want to calculate the allowable downtime from the operating rate
I tried to summarize the languages that beginners should learn from now on by purpose
I tried changing the python script from 2.7.11 to 3.6.0 on windows10
For the time being using FastAPI, I want to display how to use API like that on swagger
I want to send a signal only from the sub thread to the main thread
[Python] Warning came out when I raised from Selenium 3 to 4 [Web Driver]
I want to display an image on Jupyter Notebook using OpenCV (mac)
[Google Colab] I want to display multiple images side by side in tiles
I want to pin Spyder to the taskbar
I want to output to the console coolly
I want to handle the rhyme part1
I want to handle the rhyme part3
I want to use jar from python
I want to use Linux on mac
I want to prevent the speaker connected to the Raspberry Pi (jessie) from bouncing when the OS is restarted (Python script)
I want to handle the rhyme part2
I want to develop Android apps on Android
I want to handle the rhyme part5
I want to handle the rhyme part4
[LINE Messaging API] I want to send a message from the program to everyone's LINE
I want to use Python in the environment of pyenv + pipenv on Windows 10
I tried to rescue the data of the laptop by booting it on Ubuntu
I tried to pass the G test and E qualification by training from 50
I tried to display the infection condition of coronavirus on the heat map of seaborn
I want to be notified when the command operation is completed on linux!
I want to connect to PostgreSQL from various languages
I tried hitting the Qiita API from go
I want to email from Gmail using Python.
[Python] I want to manage 7DaysToDie from Discord! 1/3
I want to perform SageMaker inference from PHP
I want to display multiple images with matplotlib.
When you want to update the chrome driver.
I want to handle the rhyme part7 (BOW)
I want to make fits from my head
I want to use ceres solver from python
I want to do pyenv + pipenv on Windows
I want to sell Mercari by scraping python
[Python] I want to manage 7DaysToDie from Discord! 2/3
I want to log file I / O on Linux
I want to make C ++ code from Python code!
I want to customize the appearance of zabbix
I want to use the activation function Mish
How to set a shared folder with the host OS in CentOS7 on VirtualBOX
I want to plot the location information of GTFS Realtime on Jupyter! (With balloon)
[Pyhton] I want to solve the problem that tkinter does not work on MacOS11
I want to take a screenshot of the site on Docker using any font
I want to add my own structure to the structure created by Python's C extension module!