Operate Jupyter with REST API to extract and save Python code

I wanted to execute the Python code of jupyter notebook on another machine, so I decided to extract and save it using the API of jupyter.

environment

• Jupyter (docker image)
  jupyter/scipy-notebook:19db0c85c56d

Examine Jupyter tokens

Since you need a token to operate jupyter from the outside, first check the token.

I entered it when I started jupyter for the first time (should I?), But I think that I usually forgot it, so check the jupyter container in the order of search → log display → token confirmation.

• Find jupyter container In the example below, the ID is 022298a2039a and the container name is jupyter.

  $ docker ps
  CONTAINER ID     IMAGE                           COMMAND                  CREATED         STATUS       PORTS                    NAMES
  022298a2039a     jupyter/scipy-notebook:latest   "tini -g -- start-no…"   11 days ago     Up 2 days    0.0.0.0:8888->8888/tcp   jupyter
  

• Display log Display the log by specifying the ID or container name.

  #For container ID
  $ docker logs 022298a2039a
  #For container name
  $ docker logs jupyter
  # docker-For compose
  $ docker-compose logs
  

• Token confirmation Probably there is a token near the end of the log. Error logs such as API will appear, so if you make a few trials and errors, the logs will flow ...

  jupyter    |     Or copy and paste one of these URLs:
  jupyter    |         http://022298a2039a:8888/?token=xxxxxxxxxxxxxxxxxxxxxxxxxxxx
  jupyter    |      or http://127.0.0.1:8888/?token=xxxxxxxxxxxxxxxxxxxxxxxxxxxx
  

Hit the Jupyter API from the outside

Notebook status

I created a notebook for testing. The URL is below. http://localhost:8888/notebooks/work/test1.ipynb

Execution environment on the side that hits the API

Since API operation is performed with python, the requests package is required. It doesn't have to be Docker because it only puts requests, but I think it will be easy to make it a service later, so I will put it in a container. Details of the trial environment (Dockerfile) below.

FROM python:3.6-buster
RUN pip install requests

Build.

$ docker build -t jupyter-api .

Start the container. You don't have to mount (-v), but just mount the current to retrieve the created file.

$ docker run -it --net host -v $(pwd):/tmp jupyter-api bash

API trial

First, check if the jupyter API can be used.

import requests
url_api = 'http://localhost:8888/api'
response = requests.get(url_api)
print(response.status_code, response.text)  # 200 {"version": "6.0.3"}

It seems that I can hit the API safely, so I get a Notebook.

notebook_path = '/work/test1.ipynb'
url_file = url_api + '/contents' + notebook_path

token = 'xxxxxxxxxxxxxxxxx'   #Set token
headers =  {'Authorization': 'token ' + token}

response = requests.get(url_file, headers=headers)

Notebook information is returned in json format in response.text. Below, the execution result of print (response.text) is formatted for display.

{
  "name": "test1.ipynb",
  "path": "work/test1.ipynb",
  "last_modified": "2020-03-07T21:08:58.897321Z",
  "created": "2020-03-07T21:08:57.594298Z",
  "content": {
    "cells": [
      {
        "cell_type": "code",
        "execution_count": 1,
        "metadata": {
            "trusted": true
        },
        "outputs": [
          {
            "data": {
              "text/plain": "array([0.83709745, 0.46685874, 0.94285637, 0.03938868, 0.79617107,\n       0.98784776, 0.27798577, 0.96118447, 0.5253161 , 0.0690074 ])"
            },
            "execution_count": 1,
            "metadata": {},
            "output_type": "execute_result"
          }
        ],
        "source": "import numpy as np\nnp.random.random(10)"
      }
    ],
    "metadata": {
      "kernelspec": {
        "display_name": "Python 3",
        "language": "python",
        "name": "python3"
      },
      "language_info": {
        "codemirror_mode": {
          "name": "ipython",
          "version": 3
        },
        "file_extension": ".py",
        "mimetype": "text/x-python",
        "name": "python",
        "nbconvert_exporter": "python",
        "pygments_lexer": "ipython3",
        "version": "3.7.4"
      }
    },
    "nbformat": 4,
    "nbformat_minor": 4
  },
  "format": "json",
  "mimetype": null,
  "size": 921,
  "writable": true,
  "type": "notebook"
}

This time, I want to extract the code part of the notebook, so I can extract the source of the block of content.cells.cell_type ='code'. If cell_type is a markdown block, cell_type ='markdown'.

import json
json_src = json.loads(response.text)
src = [cell['source'] for cell in json_src['content']['cells'] if cell['cell_type'] == 'code']

The code of each cell is stored in src as an array element, and since there is only one cell in the source code this time, it is an array of one element.

['import numpy as np\nnp.random.random(10)']                                      

Output to a file. For the time being, output it with a UTF8 declaration.

with open('/tmp/test1.py', 'w') as f:
    print("# -*- coding: utf-8 -*-", file=f)
    for block in src:
        print(block, file=f)

Check the output file. Perfect.

# -*- coding: utf-8 -*-
import numpy as np
np.random.random(10)

I was able to output it safely from the Notebook, so it seems that I can boil it, bake it, or execute it.

Reference material

-Command to check token after starting jupyter with docker I referred to how to check the token.

• stack overflow: Interact with Jupyter Notebooks via API I referred to the syntax.
• Jupyter Notebook - Contents API
  I saw the data structure here.
• Jupyter Notebook Server API
  I checked the API.