[NNabla] How to remove the middle tier of a pre-built network

Introduction

This is the second post to qiita. (article2) Continuing from last time, while I was using nnabla, I managed to feel like "I wish I had this kind of information in qiita" Summary of what I found in nnabla reference and dir () (a standard python function that returns member variables and functions of arguments) I will.

1. Requirements

· OS: macOS Catalina (version 10.15.1) ・ Python: 3.5.4 ・ Nnabla: 1.3.0

2. Network construction

The sample network is defined below. (Same as Last time so far)

article2_rewire_on.py

import nnabla as nn
import nnabla.functions as F

# [define network]
x = nn.Variable()
y = F.add_scalar(x, 0.5)  # <-- (1)far
y = F.mul_scalar(y, 2.0)

It is simply in the form of $ y = (x + 0.5) \ times2 $. At this time, the variable y is a variable that has the result of the above formula, and the result of F.add_scalar (x, 0.5), which is an intermediate calculation, is called (1).

3. Remove the middle layer

I will explain how to delete (1) above and simply set $ y = x \ times2 $. It uses the member variable rewire_on of nnabla.Variable in the nnabla reference. This is also (relatively) easy to understand in reference. I practiced it below.

article2_rewire_on.py

h1 = y.parent.inputs[0]   # = (1)
x.rewire_on(h1)

The operation check was done below.

article2_rewire_on.py

# [check func for visit]
def get_func_name(f):
    print(f.name)
print('--- before ---')
y.visit(get_func_name)
print('')

# [rewire_on]
h1 = y.parent.inputs[0]   # = (1)
x.rewire_on(h1)

print('--- after ---')
y.visit(get_func_name)
print('')

output

--- before ---
AddScalar
MulScalar

--- after ---
MulScalar

Commentary

• First, I got (1) as the above h1. See Last for a detailed explanation of this.
• h1 disappears withx.rewire_on (h1). To be precise, it seems that h1 is replaced by x on the calculation graph. The description can be seen from the code, but when the calculation graph is divided into a subgraph that connects to the input with the node that rewire_on as the start point and the end point, and a subgraph that is transmitted to the outputEnd point on the side]). (Since it is difficult to understand in words, I would like to post a figure soon.)
• As a confirmation method, using the visit used previous, the layer names of all layers are displayed byget_func_name (f). As a result, after rewire_on, ʻAddScalar` that is calculating (1) disappears, so the desired operation can be confirmed.

4. Next notice (?)

You can use rewire_on not only to remove the middle layer, but also to insert a new layer. I managed to do this myself, but I thought "I wanted someone to write something like qiita", so I'll write it next time.