Follow the steps below to build an environment for debugging using Jupyter Lab and VS Code.
If you watch the video below, you can get an idea of what you can do.
--In VS Code, by using a mechanism called remote development, you can edit folders and files on the remote server almost the same as on your local PC by connecting with ssh. See here for details. --VS Code allows you to attach and debug Python processes on the remote server by installing a package called ptvsd on the remote server. See here for details. --In JupyterLab, it is possible to use any process (not necessarily Python) as an interpreter by using a mechanism called kernel. ――In this paper, by combining the above, the above explanation is realized.
--In this article, we built the environment based on the Docker image "tensorflow / tensorflow: 1.14.0-gpu-py3-jupyter". ――However, if you have already installed Jupyter Lab with anaconda etc., you can use it as it is. (I'll explain it below, but the point is that it works if ptvsd is installed on the remote server and set as the kernel of Jupyter Lab.)
# apt-get update
# apt-get install -y wget
# apt-get install -y language-pack-ja
# apt-get install -y ssh
# apt-get install -y git
# update-locale LANG=ja_JP.UTF-8
Required to install the JupyterLab Extension. This step is not necessary if you just want to set the kernel, but it is better to support it for convenient use of Jupyter Lab.
# curl -sL https://deb.nodesource.com/setup_12.x | bash -
# apt-get install -y nodejs
--In particular, you need jupyterlab and ptvsd. --This time, I will install numpy, matplotlib, scikit_learn. --As of April 2020, version 2 of jupyterlab is out, but since there are many extensions that are not yet supported, we will install 1.0.2 this time.
# pip install -r requirements.txt
requirements.txt
numpy==1.16.3
matplotlib==3.0.3
scikit_learn==0.20.3
jupyterlab==1.0.2
ptvsd==4.3.2
Required to connect to a remote server from VS Code for remote development. This time, open the ssh port on port 19205.
# cp /etc/ssh/sshd_config /etc/ssh/sshd_config.org
# sed -e s/#Port/Port/g /etc/ssh/sshd_config.org | sed -e s/22/19205/g > /etc/ssh/sshd_config
Next, start ssh.
root@25fa703a90c4:/tf# service ssh start
* Starting OpenBSD Secure Shell server sshd [ OK ]
Next, set up Jupyter Lab. This time, we will start the jupyterLab server as a jupyter user.
First, in preparation, change ownership of the / usr / local / share / jupyter / lab directory to jupyter. If you do not do this, you will not be able to install Extensions from Jupyter Lab under startup as a jupyter user, which is inconvenient. I think there is a better way (for example, keep these files in your home directory). Here, I will simply change the ownership below.
# chown -R jupyter:jupyter /usr/local/share/jupyter/lab
First, save the following file with the file name /usr/local/share/jupyter/kernels/ptvsd/kernel.json so that you can call the debug interpreter from JupyterLab. (The method of calling the interpreter set here from JupyterLab will be explained later.)
kernel.json
{
"argv": [
"/usr/bin/python3",
"-m",
"ptvsd",
"--host",
"0.0.0.0",
"--port",
"19204",
"-m",
"ipykernel_launcher",
"-f",
"{connection_file}"
],
"display_name": "Python3 Debug",
"language": "python"
}
It is like this.
root@25fa703a90c4:/usr/local/share/jupyter/kernels# pwd
/usr/local/share/jupyter/kernels
root@25fa703a90c4:/usr/local/share/jupyter/kernels# mkdir ptvsd
root@25fa703a90c4:/usr/local/share/jupyter/kernels# cd ptvsd
root@25fa703a90c4:/usr/local/share/jupyter/kernels/ptvsd# cat > kernel.json <<EOL
{
"argv": [
"/usr/bin/python3",
"-m",
"ptvsd",
"--host",
"0.0.0.0",
"--port",
"19204",
"-m",
"ipykernel_launcher",
"-f",
"{connection_file}"
],
"display_name": "Python3 Debug",
"language": "python"
}
EOL
root@25fa703a90c4:/usr/local/share/jupyter/kernels/ptvsd# ls kernel.json
kernel.json
This completes the Jupyter Lab settings.
Next, make settings for remote development with VS Code.
First of all, if you haven't done so already, you need to prepare for key exchange so that you can connect to the remote server from your local PC with ssh. I will omit the detailed procedure, but I think that here etc. will be helpful.
Follow these steps (https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.vscode-remote-extensionpack) to install the Remote Development extension pack in Visual Studio.
Follow the steps below to connect to a remote server from VS Code 1 and edit folders and files. (From the second time onward, you can also connect using the saved settings.)
ssh user@host name-p port name
5. Specify the save location of ssh settings (usually, specify the one under the personal folder)
6. Click the "Connect" button in the dialog that appears at the bottom right
Next, install the Python extension for Visual Studio Code from here to debug Python on VS Code.
This completes the connection from VS Code to the remote server. This procedure allows you to edit files and folders on the remote server with VS Code as if they were on your local PC.
It's very easy, but I will use the sample program below. First, on the remote server, save the following files in the same directory where you save the notebook.
sample.py
import numpy as np
def sample(x):
y = np.square(x)
return y
Next, start JupyterLab on the remote server and access JupyterLab from the browser on the local PC. (It is the same as accessing Jupyter Lab normally.)
jupyter lab --ip=0.0.0.0
You can find the following message in the log when jupyterLab is started, so you can open JupyterLab by accessing the URL from your local PC. * At the time of access, the IP address will be changed to that of the remote PC.
To access the notebook, open this file in a browser:
file:///home/jupyter/.local/share/jupyter/runtime/nbserver-13-open.html
Or copy and paste one of these URLs:
http://(da9742c1f7ef or 127.0.0.1):8888/?token=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
You can access JupyterLab by accessing the URL output when 5.2.1 JupyterLab starts from a browser on your local PC.
When you access JupyterLab, you should see a kernel called "Python3 debug" in the Notebook column as shown below. (This is the kernel using ptvsd set in step 4.2.2.) Click this button to create a new notebook.
The upper right of the screen is "Python3 Debug", and you can see that the Kernel for Debug created earlier is used.
For the time being, create a notebook like the one below and run it.
Follow the steps in 4.3 to connect to the remote server with VS Code. Once connected, open the sample program. It is like this.
Follow the steps below to configure debug settings.
The settings are as follows.
--port: Specify the port set in the kernel settings of ptvsd. --host: Specify the IP address of the remote server. --pathMappings: localRoot should be $ {workspaceFolder} as shown in the example below. remoteRoot specifies the folder on the remote server where the files are stored.
{
//You can use IntelliSense to learn the available attributes.
//Hover and display the description of existing attributes.
//Check the following for more information: https://go.microsoft.com/fwlink/?linkid=830387
"version": "0.2.0",
"configurations": [
{
"name": "Python:attach",
"type": "python",
"request": "attach",
"port": 19204,
"host": "192.168.2.200",
"pathMappings": [
{
"localRoot": "${workspaceFolder}",
"remoteRoot": "/home/jupyter/proj/docker"
}
]
}
]
}
This completes the debug settings. Next, let's actually debug.
The debugging procedure is as follows. See also the demo video at the beginning for details.
After setting breakpoints on VS Code as appropriate, execute the "Python attach" set earlier.
Run notebook from JupyterLab
Debugging is performed because the processing is interrupted at the breakpoint.
The result of the command executed in "Debug Console" on VS Code is also displayed on JupyterLab. Especially, it is very convenient because you can display internal variables with matplotlib etc. and display them as a graph!
Summary So far, we have explained how to debug a Python process executed from JupyterLab by remotely attaching it from VS Code. It's very convenient, so please try it. It's a little scribbled, so there may be some deficiencies. If you have any questions, please comment and I would like to add or correct as appropriate.
Recommended Posts