Create AND / OR / NAND / XOR circuits with FPGA that may be necessary for deep learning with Polyphony

Read Deep Learning from scratch

The book Deep Learning from scratch is wonderful. Learning with Python As the subtitle of Deep Learning shows, this book is about deepening your understanding by creating Deep Learning apps (? Algorithms?) From scratch with Python.

This book first makes AND, OR, NAND, XOR. I feel like I'm overwhelmed, but let's compile these with Polyphony and make them into hardware.

The source is taken from the following URL on github. https://github.com/ryos36/polyphony-tutorial/

If you want to try it easily, install polyphony and iverilog, clone the above URL and run each source with simu.py.

> pip3 install polyphony
<Install iverilog properly>
> git clone https://github.com/ryos36/polyphony-tutorial/
> cd polyphony-tutorial/DeepLearning
> ../bin/simu.py and.py 

The environment construction using pyvenv is summarized here. http://qiita.com/ryos36/items/7e7fce9078a79f782380

From Chapter 2 Perceptron

Let's create the AND gate in the example of 2.3.1.

and.py


from polyphony import testbench

def AND(x1, x2):
    w1, w2, theta = 5, 5, 7
    tmp = x1*w1 + x2*w2
    if tmp <= theta:
        return 0
    elif tmp > theta:
        return 1

@testbench
def test():
    print(AND(0, 0))
    print(AND(1, 0))
    print(AND(0, 1))
    print(AND(1, 1))

test()

The parameters are 0.5, 0.5, 0.7 in the book, but change them to integer values of 5, 5, 7 for Polyphony. (If you think about it now, maybe 2,2,3 would have been better)

The result is as follows.

[test-0.2.2] Persimmon:polyphony-tutorial> cd DeepLearning/
[test-0.2.2] Persimmon:DeepLearning> ../bin/simu.py and.py
    0:AND_0_in_x1=   x, AND_0_in_x2=   x, AND_0_out_0=   x
  110:AND_0_in_x1=   0, AND_0_in_x2=   0, AND_0_out_0=   x
  160:AND_0_in_x1=   0, AND_0_in_x2=   0, AND_0_out_0=   0
0
  180:AND_0_in_x1=   1, AND_0_in_x2=   0, AND_0_out_0=   0
0
  250:AND_0_in_x1=   0, AND_0_in_x2=   1, AND_0_out_0=   0
0
  320:AND_0_in_x1=   1, AND_0_in_x2=   1, AND_0_out_0=   0
  370:AND_0_in_x1=   1, AND_0_in_x2=   1, AND_0_out_0=   1
1

As expected. The number to the left of the colon of mysterious information that is output without permission is the time (clock number). It is a rough guide for performance.

Use a Python list

Now let's use a Python list.

and2.py


from polyphony import testbench

def list_mul(lst_r, lst_a, lst_b):
    for i in range(len(lst_r)):
        lst_r[i] = lst_a[i] * lst_b[i]

def sum(lst):
    tmp = 0
    for i in range(len(lst)):
        tmp = tmp + lst[i]

    return tmp

def AND(x1, x2):
    lst_r = [0, 0]
    lst_a = [x1, x2]
    lst_b = [5, 5]
    b = -7

    list_mul(lst_r, lst_a, lst_b)
    tmp = sum(lst_r) + b

    if tmp <= 0:
        return 0
    else:
        return 1

@testbench
def test():
    print(AND(0, 0))
    print(AND(1, 0))
    print(AND(0, 1))
    print(AND(1, 1))

test()

Polyphony doesn't have list multiplication or sum, so we're defining a function.

[test-0.2.2] Persimmon:DeepLearning> ../bin/simu.py and2.py
    0:AND_0_in_x1=   x, AND_0_in_x2=   x, AND_0_out_0=   x
  110:AND_0_in_x1=   0, AND_0_in_x2=   0, AND_0_out_0=   x
  550:AND_0_in_x1=   0, AND_0_in_x2=   0, AND_0_out_0=   0
0
  570:AND_0_in_x1=   1, AND_0_in_x2=   0, AND_0_out_0=   0
0
 1030:AND_0_in_x1=   0, AND_0_in_x2=   1, AND_0_out_0=   0
0
 1490:AND_0_in_x1=   1, AND_0_in_x2=   1, AND_0_out_0=   0
 1930:AND_0_in_x1=   1, AND_0_in_x2=   1, AND_0_out_0=   1
1

Using a list added to the abstraction, but it slowed down because I used a for statement for the operation. You can create or.py and nand.py as well. It is a little sad source that it is copied.

Try to make an XOR

Based on these, create an XOR and execute it.

xor.py


from polyphony import testbench

def list_mul(lst_r, lst_a, lst_b):
    for i in range(len(lst_r)):
        lst_r[i] = lst_a[i] * lst_b[i]

def sum(lst):
    tmp = 0
    for i in range(len(lst)):
        tmp = tmp + lst[i]

    return tmp

def AND(x1, x2):
    lst_r = [0, 0]
    lst_a = [x1, x2]
    lst_b = [5, 5]
    b = -7

    list_mul(lst_r, lst_a, lst_b)
    tmp = sum(lst_r) + b

    if tmp <= 0:
        return 0
    else:
        return 1

def OR(x1, x2):
    lst_r = [0, 0]
    lst_a = [x1, x2]
    lst_b = [5, 5]
    b = -2

    list_mul(lst_r, lst_a, lst_b)
    tmp = sum(lst_r) + b

    if tmp <= 0:
        return 0
    else:
        return 1

def NAND(x1, x2):
    lst_r = [0, 0]
    lst_a = [x1, x2]
    lst_b = [-5, -5]
    b = 7

    list_mul(lst_r, lst_a, lst_b)
    tmp = sum(lst_r) + b

    if tmp <= 0:
        return 0
    else:
        return 1

def XOR(x1, x2):
    s1 = NAND(x1, x2)
    s2 = OR(x1, x2)
    y = AND(s1, s2)
    return y

@testbench
def test():
    print(XOR(0, 0))
    print(XOR(1, 0))
    print(XOR(0, 1))
    print(XOR(1, 1))

test()

It's a culmination of the copy and paste program, but it works. It is important to move. The result seems to be correct as follows.

[test-0.2.2] Persimmon:DeepLearning> ../bin/simu.py xor.py
    0:XOR_0_in_x1=   x, XOR_0_in_x2=   x, XOR_0_out_0=   x
  110:XOR_0_in_x1=   0, XOR_0_in_x2=   0, XOR_0_out_0=   x
 1440:XOR_0_in_x1=   0, XOR_0_in_x2=   0, XOR_0_out_0=   0
0
 1450:XOR_0_in_x1=   1, XOR_0_in_x2=   0, XOR_0_out_0=   0
 2780:XOR_0_in_x1=   1, XOR_0_in_x2=   0, XOR_0_out_0=   1
1
 2790:XOR_0_in_x1=   0, XOR_0_in_x2=   1, XOR_0_out_0=   1
1
 4130:XOR_0_in_x1=   1, XOR_0_in_x2=   1, XOR_0_out_0=   1
 5460:XOR_0_in_x1=   1, XOR_0_in_x2=   1, XOR_0_out_0=   0
0

Use python classes

Use classes to avoid copying. The outlook has improved considerably.

c_xor.py


from polyphony import testbench

class BitOp:
    def __init__(self, w0, w1, b):
        self.w0 = w0
        self.w1 = w1
        self.b = b

    def eval(self, x0, x1):
        tmp0 = self.w0 * x0
        tmp1 = self.w1 * x1
        tmp = tmp0 + tmp1 + self.b
        if tmp <= 0:
            return 0
        else:
            return 1

def AND(x1, x2):
    op = BitOp(5, 5, -7)
    return op.eval(x1, x2)

def OR(x1, x2):
    op = BitOp(5, 5, -2)
    return op.eval(x1, x2)

def NAND(x1, x2):
    op = BitOp(-5, -5, 7)
    return op.eval(x1, x2)

def XOR(x1, x2):
    AND = BitOp(5, 5, -7)
    OR = BitOp(5, 5, -2)
    NAND = BitOp(-5, -5, 7)
    s1 = NAND.eval(x1, x2)
    s2 = OR.eval(x1, x2)
    y = AND.eval(s1, s2)
    return y

@testbench
def test():
    print(XOR(0, 0))
    print(XOR(1, 0))
    print(XOR(0, 1))
    print(XOR(1, 1))

test()

I stopped using lists, so the execution time is faster.

[test-0.2.2] Persimmon:DeepLearning> ls
and.py  and2.py  c_xor.py  nand.py  or.py  t_and.py  xor.py
[test-0.2.2] Persimmon:DeepLearning> ../bin/simu.py c_xor.py
    0:XOR_0_in_x1=   x, XOR_0_in_x2=   x, XOR_0_out_0=   x
  110:XOR_0_in_x1=   0, XOR_0_in_x2=   0, XOR_0_out_0=   x
  280:XOR_0_in_x1=   0, XOR_0_in_x2=   0, XOR_0_out_0=   0
0
  290:XOR_0_in_x1=   1, XOR_0_in_x2=   0, XOR_0_out_0=   0
  460:XOR_0_in_x1=   1, XOR_0_in_x2=   0, XOR_0_out_0=   1
1
  470:XOR_0_in_x1=   0, XOR_0_in_x2=   1, XOR_0_out_0=   1
1
  650:XOR_0_in_x1=   1, XOR_0_in_x2=   1, XOR_0_out_0=   1
  820:XOR_0_in_x1=   1, XOR_0_in_x2=   1, XOR_0_out_0=   0
0

Bonus: Tuple version and

If you set Polyphony to 0.3.0 (normally pip3 install will install 0.2.2 at the moment 2017.3.27), the tuple version will also work.

t_and.py


from polyphony import testbench

def t_mul2(t_a, t_b):
    a0, a1 = t_a
    b0, b1 = t_b
    return (a0 * b0, a1 * b0)

def t_sum2(t_a):
    a0, a1 = t_a
    return a0 + a1

def AND(x1, x2):
    para = (5, 5)
    b = -7

    t_r = t_mul2((x1, x2), para)
    tmp = t_sum2(t_r) + b

    if tmp <= 0:
        return 0
    else:
        return 1

@testbench
def test():
    print(AND(0, 0))
    print(AND(1, 0))
    print(AND(0, 1))
    print(AND(1, 1))

test()

I made XOR

I was able to XOR, but is it natural that FPGA can do it in the first place? The road to becoming a deep learning master is still far away

Recommended Posts

Create AND / OR / NAND / XOR circuits with FPGA that may be necessary for deep learning with Polyphony
Overview and useful features of scikit-learn that can also be used for deep learning
Create an environment for "Deep Learning from scratch" with Docker
Try Deep Learning with FPGA
Realize environment construction for "Deep Learning from scratch" with docker and Vagrant
selenium: wait for element with AND / OR
A collection of resources that may be useful for creating and expanding dotfiles