This is a continuation of Previous article. In this article, we will prepare for coding. If you want to implement it yourself, but it's quite ... please take a look. (Basically, I can't use complicated techniques, so don't worry) ** The coding environment is jupyter notebook. ** ** If you want to manage it at the module level elsewhere, uncomment the command such as ʻimport`. If it doesn't work, let me know in the comments ... The next article is here
-[Layer module code preparation](# Layer module code preparation)
-[Layer Manager Code Preparation](# Layer Manager Code Preparation)
-
First, make a base container.
baselayer.py
import numpy as np
class BaseLayer():
"""
All underlying layer classes
Describe the processing common to the intermediate layer and the output layer.
"""
def __init__(self):
pass
def forward(self):
"""
Implementation of forward propagation
"""
pass
def backward(self):
"""
Implementation of backpropagation
"""
pass
def update(self):
"""
Implementation of parameter learning
"""
pass
middlelayer.py
import numpy as np
#import .baselayer import BaseLayer
class MiddleLayer(BaseLayer):
"""
Middle class
The input layer is also treated as one of the intermediate layers in mounting.
"""
pass
outputlayer.py
import numpy as np
#from .baselayer import BaseLayer
class OutputLayer(BaseLayer):
"""
Output layer class
"""
pass
Here, we will prepare a manager to handle the layer module.
** Finally, put the code together. Please note that you may have forgotten to change the code in the middle. ** **
Basically, I will make it conscious of Python's List
type and Dictionary
type.
layermanager.py
import numpy as np
#from .layer import *
class _TypeManager():
"""
Manager class for layer types
"""
N_TYPE = 2 #Number of layer types
MIDDLE = 0 #Middle layer numbering
OUTPUT = 1 #Output layer numbering
class LayerManager(_TypeManager):
"""
Manager class for managing layers
"""
def __init__(self):
pass
The LayerManager
class is aware of the Dictionary
type and allows you to keep a list of layers and add / remove them.
Also, let's inherit _TypeManager
.
By the way, the layer
module is a module for importing all Layer
related modules.
layer.py
from .middlelayer import *
from .outputlayer import *
from ._layererror import *
The _layererror
module will be described later, but we plan to make it a module that summarizes the errors that occur in layers.
First, implement a special method of the LayerManager
class. See Official Documentation for a list of special methods.
__init__
methodNow, implement the __init__
method of the LayerManager
class.
In the first place, the __init__
method is one of the special methods of Python and one of the methods called when instantiating a class.
In other words, describe the action you want to take when the LayerManager
class is generated.
here
--Keep a list of layers --Keep a list of layer names --Keep the number of layers by type
I want you to do it, so implement it that way.
layermanager.py
def __init__(self):
self.__layer_list = [] #List of layers
self.__name_list = [] #Name list for each layer
self.__ntype = np.zeros(self.N_TYPE) #Number of layers by type
__repr__
and __str__
methodsThe __repr__
method is an" official "string representation called from the repr
function, one of Python's built-in functions.
It's called "official", but the point is that it's detailed information.
The __str__
method is an" unofficial "string representation called from one of Python's built-in functions, the str
function, and the built-in functions format
and print
.
The meaning of "informal" is that it is easy to read.
layermanager.py
def __repr__(self):
layerRepr= "layer_list: " + repr(self.__layer_list)
nameRepr = "name_list: " + repr(self.__name_list)
ntypeRepr = "ntype: " + repr(self.__ntype)
return (layerRepr + "\n"
+ nameRepr + "\n"
+ ntypeRepr)
def __str__(self):
layerStr = "layer_list: " + str(self.__layer_list)
nameStr = "name_list: " + str(self.__name_list)
ntypeStr = "ntype: " + str(self.__ntype)
return (layerStr + "\n"
+ nameStr + "\n"
+ ntypeStr)
__len__
methodThe __len__
method describes the behavior when called from the len
function, which is one of Python's built-in functions.
Let's return the number of layers that the LayerManager
class has.
layermanager.py
def __len__(self):
"""
Python built-in functions`len`Describes the operation when called from.
Returns the sum of the number of layers by type.
"""
return np.sum(self.__ntype)
__getitem__
methodThe __getitem__
method is a method called when fetching an element by specifying an index etc. in a Python list or numpy array.
layermanager.py
def __getitem__(self, key):
"""
For example
lm = LayerManager()
+----------------+
| (Add element to lm) |
+----------------+
x = lm[3].~~
Because it is called when an element of a list or array is accessed, like
Describe the operation at that time.
slice and str,Only allow access via int.
"""
if isinstance(key, slice):
#If the key is a slice, refer to the list of layers with slice.
#Unusual value(Index out of range etc.)When is entered
#Python gives me an error.
return self.__layer_list[key]
elif isinstance(key, str):
#If key is a string, get the index from the name list of each layer and
#Returns the elements of the list of applicable layers.
if key in self.__name_list:
index = self.__name_list.index(key)
return self.__layer_list[index]
else:
#If the key does not exist, KeyError is issued.
raise KeyError("{}: No such item".format(key))
elif isinstance(key, int):
#If key is an integer, returns the corresponding element in the list of layers.
#Unusual value(Index out of range etc.)When is entered
#Python gives me an error.
return self.__layer_list[key]
else:
raise KeyError("{}: Not defined such key type.".format(key))
__setitem__
method__setitem__
is a method often used in the Dictionary
type, etc., which is called when setting key
and value
as lm [key] = value
.
Only allow overwriting of elements and give an error otherwise.
The reason is that we don't want users to register their own key
because we'll use the naming rules later.
layermanager.py
def __setitem__(self, key, value):
"""
For example
lm = LayerManager()
+----------------+
| (Add element to lm) |
+----------------+
lm[1] = x
Because it is called when an element of a list or array is accessed, like
Describe the operation at that time.
Only overwriting elements is allowed, and adding new elements is prohibited.
"""
value_type = ""
if isinstance(value, list):
#Specified on the right side'value'But'list'If
#All elements'BaseLayer'Error if class or not inheriting it.
if not np.all(
np.where(isinstance(value, BaseLayer), True, False)):
self.AssignError()
value_type = "list"
elif isinstance(value, BaseLayer):
#Specified on the right side'value'But'BaseLayer'Is it a class?
#Error if it is not inherited.
self.AssignError(type(value))
if value_type == "":
value_type = "BaseLayer"
if isinstance(key, slice):
#If key is a slice, overwrite the element in the list of layers.
#However'value_type'But'list'Otherwise an error.
#Unusual value(Index out of range etc.)When is entered
#Python gives me an error.
if value_type != "list":
self.AssignError(value_type)
self.__layer_list[key] = value
elif isinstance(key, str):
#If key is a string, get the index from the name list of each layer and
#Overwrite the element in the list of applicable layers.
#However'value_type'But'BaseLayer'Otherwise an error.
if value_type != "BaseLayer":
raise AssignError(value_type)
if key in self.__name_list:
index = self.__name_list.index(key)
self.__layer_list[index] = value
else:
#If the key does not exist, KeyError is issued.
raise KeyError("{}: No such item".format(key))
elif isinstance(key, int):
#If key is an integer, overwrite the corresponding element in the layer list.
#However'value_type'But'BaseLayer'Otherwise an error.
#Also, an abnormal value(Index out of range etc.)When is entered
#Python gives me an error.
if value_type != "BaseLayer":
raise AssignError(value_type)
self.__layer_list[key] = value
else:
raise KeyError(key, ": Undefined such key type.")
__delitem__
method__delitem__
is a method called by del lm [key]
etc. This just deletes the specified element.
However, the processing after deletion is a little troublesome.
layermanager.py
def __delitem__(self, key):
"""
For example
lm = LayerManager()
+----------------+
| (Add element to lm) |
+----------------+
del lm[2]
Because it is called when the element of the list or array is accessed by the del statement like
Describe the operation at that time.
If the specified element exists, it will be deleted and renamed.
"""
if isinstance(key, slice):
#If the key is a slice, delete the specified element as it is
#Unusual value(Index out of range etc.)When is entered
#Python gives me an error.
del self.__layer_list[slice]
del self.__name_list[slice]
elif isinstance(key, str):
#If key is a string, get the index from the name list of each layer and
#Delete the relevant element.
if key in self.__name_list:
del self.__layer_list[index]
del self.__name_list[index]
else:
#If the key does not exist, KeyError is issued.
raise KeyError("{}: No such item".format(key))
elif isinstance(key, int):
#If key is an integer, delete the corresponding element in the layer list.
#Unusual value(Index out of range etc.)When is entered
#Python gives me an error.
del self.__layer_list[key]
else:
raise KeyError(key, ": Undefined such key type.")
#Rename
self._rename()
By the way, _
(single underscore) means a weaker private
attribute than __
(double underscore). This can be accessed from outside as class name._rename
, but double underscores can only be accessed with class name ._ module name __ method name
, for example. ..
Also, module functions that do not belong to a class cannot be imported by from module name import *
if private
is specified by _
. You need to call it from module name import _method name
properly.
From here, we will implement user-defined (that is, we define) functions. Here, I will basically implement the famous points of the List
type and the Dictionary
type.
_rename
methodFor the time being, implement the _rename
method that appeared in the previous implementation.
When is this necessary?
--Elements are added in the middle --Elements in the middle are deleted
There are two ways. If you do your best and think about the logic, it seems that you can reduce the number of changes, but it is troublesome, so I will recount from the beginning. Because it is easy to implement. ** Maintenance is important. ** **
layermanager.py
def _rename(self):
"""
When the name list naming violates the rules due to list operations
Rename the naming list and each layer to meet the rules again.
The naming rule is[Layer type][What number]will do.
If the layer type is Middle Layer, Middle
Output for Output Layer
It is abbreviated as.
The number is counted by type.
Also, here again__Counts ntypes.
"""
#Initialize the number of layers by type
self.__ntype = np.zeros(self.N_TYPE)
#Recount and rename each layer
for i in range(len(self)):
if "Middle" in self.__name_list[i]:
self.__ntype[self.MIDDLE] += 1
self.__name_list[i] = "Middle{}".format(
self.__ntype[self.MIDDLE])
self.__layer_list[i].name = "Middle{}".format(
self.__ntype[self.MIDDLE])
elif "Output" in self.__name_list[i]:
self.__ntype[self.OUTPUT] += 1
self.__name_list[i] = "Output{}".format(
self.__ntype[self.OUTPUT])
self.__layer_list[i].name = "Output{}".format(
self.__ntype[self.OUTPUT])
else:
raise UndefinedLayerType(self.__name_list[i])
Implement the familiar ʻappend` method in the method of adding an element to a list. Since it is added at the end, there is no complicated processing.
layermanager.py
def append(self, *, name="Middle", **kwds):
"""
Implementation of the familiar append method, which is a method for adding elements to a list.
"""
if "prev" in kwds:
# 'prev'Is included in the keyword
#This means that the number of elements in the previous layer is specified.
#Basically it is supposed to be the time to insert the first layer, so
#Other than that, it is basically determined automatically and is not specified.
if len(self) != 0:
if kwds["prev"] != self.__layer_list[-1].n:
#Error if it does not match the number of units at the end.
raise UnmatchUnitError(self.__layer_list[-1].n,
kwds["prev"])
else:
if len(self) == 0:
#The first layer must always specify the number of input units.
raise UnmatchUnitError("Input units", "Unspecified")
else:
#The number of units in the last layer'kwds'Add to
kwds["prev"] = self.__layer_list[-1].n
#Read the layer type and change the name according to the naming rule
if name == "mid" or "m":
name = "Middle"
elif name == "out" or "o":
name = "Output"
else:
raise UndefinedLayerError(name)
#Add a layer.
if name == "Middle":
#Increment the layer by type
self.__ntype[self.MIDDLE] += 1
#Add to name
name += str(self.__ntype[self.MIDDLE])
#Add to name list
self.__name_list.append(name)
#Finally, create a layer and add it to the list.
self.__layer_list.append(
MiddleLayer(name=name, **kwds))
elif name == "Output":
#This is also the same.
self.__ntype[self.OUTPUT] += 1
name += str(self.__ntype[self.OUTPUT])
self.__name_list.append(name)
self.__layer_list.append(
OutputLayer(name=name, **kwds))
#If you do not draw an else statement here, change the name according to the naming rule
#Already abnormal at the stage'name'Is omitted.
This may not be very familiar? Implement the ʻextend method of
List`.
example.py
x = [1, 2, 3]
y = [4, 5, 6]
x.append(y)
print(x)
Then
[1, 2, 3, [4, 5, 6]]
I think it will be like this. ʻExtend` method
example.py
x = [1, 2, 3]
y = [4, 5, 6]
x.extend(y)
print(x)
The result of
[1, 2, 3, 4, 5, 6]
It will be.
layermanager.py
def extend(self, lm):
"""
Another layer manager already in the extend method'lm'Elements of
Add all.
"""
if not isinstance(lm, LayerManager):
# 'lm'Error if the instance of is not LayerManager.
raise TypeError(type(lm), ": Unexpected type.")
if len(self) != 0:
if self.__layer_list[-1].n != lm[0].prev:
#With the number of units in your last layer
# 'lm'Error if the number of inputs in the first layer of is not the same.
raise UnmatchUnitError(self.__layer_list[-1].n,
lm[0].prev)
#Each'extend'Add by method
self.__layer_list.extend(lm.layer_list)
self.__name_list.extend(lm.name_list)
#Rename
self._rename()
This may not be very familiar either. A method that adds an element at a specified position.
layermanager.py
def insert(self, prev_name, name="Middle", **kwds):
"""
In the insert method, specify the name of the previous layer and combine it with that layer.
Add an element.
"""
# 'prev_name'Error if does not exist.
if not prev_name in self.__name_list:
raise KeyError(prev_name, ": No such key.")
# 'prev'Is included in the keyword
# 'prev_name'Error if it does not match the number of units in the layer specified in.
if "prev" in kwds:
if kwds["prev"] \
!= self.__layer_list[self.index(prev_name)].n:
raise UnmatchUnitError(
kwds["prev"],
self.__layer_list[self.index(prev_name)].n)
# 'n'Is included in the keyword
if "n" in kwds:
# 'prev_name'If is not the last
if prev_name != self.__name_list[-1]:
#Error if it does not match the number of units in the next layer.
if kwds["n"] != self.__layer_list[
self.index(prev_name)+1].prev:
raise UnmatchUnitError(
kwds["n"],
self.__layer_list[self.index(prev_name)].prev)
#If there are no elements yet'append'Give an error to use the method.
if len(self) == 0:
raise RuntimeError(
"You have to use 'append' method instead.")
#Get index of insertion location
index = self.index(prev_name) + 1
#Read the layer type and change the name according to the naming rule
if name == "mid" or "m":
name = "Middle"
elif name == "out" or "o":
name = "Output"
else:
raise UndefinedLayerError(name)
#Insert element
#At this time,'name'Does not yet follow the naming rules,
#I'll rename it later so don't worry about it.
if "Middle" in name:
self.__layer_list.insert(index,
MiddleLayer(name=name, **kwds))
self.__name_list.insert(index, name)
elif "Output" in name:
self.__layer_list.insert(index,
OutputLayer(name=name, **kwds))
self.__name_list.insert(index, name)
#Rename
self._rename()
This is the original. It behaves like a combination of the ʻextend and ʻinsert
methods.
layermanager.py
def extend_insert(self, prev_name, lm):
"""
This is the original function.
It behaves like a combination of extend and insert methods.
Simply put, it's like inserting another layer manager.
"""
if not isinstance(lm, LayerManager):
# 'lm'Error if the instance of is not LayerManager.
raise TypeError(type(lm), ": Unexpected type.")
# 'prev_name'Error if does not exist.
if not prev_name in self.__name_list:
raise KeyError(prev_name, ": No such key.")
#The number of units of the layers before and after the specified location and the first and last layers of lm
#If they do not match, an error occurs.
if len(self) != 0:
if self.__layer_list[self.index(prev_name)].n \
!= lm.layer_list[0].prev:
#With the number of units in your designated location'lm'The first number of units in
#If they do not match, an error occurs.
raise UnmatchUnitError(
self.__layer_list[self.index(prev_name)].n,
lm.layer_list[0].prev)
if prev_name != self.__name_list[-1]:
# 'prev_name'Is not my last layer
if lm.layer_list[-1].n \
!= self.__layer_list[self.index(prev_name)+1].prev:
# 'lm'The number of units at the end of and the layer next to your designated location
# 'prev'Error if it does not match the number of units.
raise UnmatchUnitError(
lm.layer_list[-1].n,
self.__layer_list[self.index(prev_name)+1].prev)
else:
#If you don't have any elements'extend'I get an error to use the method.
raise RuntimeError(
"You have to use 'extend' method instead.")
#Get index of insertion location
index = self.index(prev_name) + 1
#Elements after the insertion location'buf'After evacuating to, remove it once and
#Add an element using the extend method
layer_buf = self.__layer_list[index:]
name_buf = self.__name_list[index:]
del self.__layer_list[index:]
del self.__name_list[index:]
self.extend(lm)
#Add the element that was evacuated
self.__layer_list.extend(layer_buf)
self.__name_list.extend(name_buf)
#Rename
self._rename()
remove
methodThe remove
method removes the specified element, isn't it?
This is already done with [Implementation of del statement](implementation of #delitem method), so just call it.
layermanager.py
def remove(self, key):
"""
The remove method removes the element with the specified name.
It is also allowed to be specified by index.
"""
#Already implemented'del'The sentence is OK.
del self[key]
Other than the above, it is a method list that is useful.
layermanager.py
def index(self, target):
return self.__name_list.index(target)
def name(self, indices):
return self.__name_list[indices]
We will implement the property here. Basically, only getter
is needed, so decorate it with @ property
and implement it.
If you want to add a setter
, you can use the @ propertyname.setter
decorator.
layermanager.py
@property
def layer_list(self):
return self.__layer_list
@property
def name_list(self):
return self.__name_list
@property
def ntype(self):
return self.__ntype
The code of the layer manager so far is integrated and posted as the full text. Of course, we will add methods etc. in future articles.
layermanager.py
import numpy as np
#from .layer import *
class _TypeManager():
"""
Manager class for layer types
"""
N_TYPE = 2 #Number of layer types
MIDDLE = 0 #Middle layer numbering
OUTPUT = 1 #Output layer numbering
class LayerManager(_TypeManager):
"""
Manager class for managing layers
"""
def __init__(self):
self.__layer_list = [] #List of layers
self.__name_list = [] #Name list for each layer
self.__ntype = np.zeros(self.N_TYPE) #Number of layers by type
def __repr__(self):
layerRepr= "layer_list: " + repr(self.__layer_list)
nameRepr = "name_list: " + repr(self.__name_list)
ntypeRepr = "ntype: " + repr(self.__ntype)
return (layerRepr + "\n"
+ nameRepr + "\n"
+ ntypeRepr)
def __str__(self):
layerStr = "layer_list: " + str(self.__layer_list)
nameStr = "name_list: " + str(self.__name_list)
ntypeStr = "ntype: " + str(self.__ntype)
return (layerStr + "\n"
+ nameStr + "\n"
+ ntypeStr)
def __len__(self):
"""
Python built-in functions`len`Describes the operation when called from.
Returns the sum of the number of layers by type.
"""
return np.sum(self.__ntype)
def __getitem__(self, key):
"""
For example
lm = LayerManager()
+----------------+
| (Add element to lm) |
+----------------+
x = lm[3].~~
Because it is called when an element of a list or array is accessed, like
Describe the operation at that time.
slice and str,Only allow access via int.
"""
if isinstance(key, slice):
#If the key is a slice, refer to the list of layers with slice.
#Unusual value(Index out of range etc.)When is entered
#Python gives me an error.
return self.__layer_list[key]
elif isinstance(key, str):
#If key is a string, get the index from the name list of each layer and
#Returns the elements of the list of applicable layers.
if key in self.__name_list:
index = self.__name_list.index(key)
return self.__layer_list[index]
else:
#If the key does not exist, KeyError is issued.
raise KeyError("{}: No such item".format(key))
elif isinstance(key, int):
#If key is an integer, returns the corresponding element in the list of layers.
#Unusual value(Index out of range etc.)When is entered
#Python gives me an error.
return self.__layer_list[key]
else:
raise KeyError(key, ": Undefined such key type.")
def __setitem__(self, key, value):
"""
For example
lm = LayerManager()
+----------------+
| (Add element to lm) |
+----------------+
lm[1] = x
Because it is called when an element of a list or array is accessed, like
Describe the operation at that time.
Only overwriting elements is allowed, and adding new elements is prohibited.
"""
value_type = ""
if isinstance(value, list):
#Specified on the right side'value'But'list'If
#All elements'BaseLayer'Error if class or not inheriting it.
if not np.all(
np.where(isinstance(value, BaseLayer), True, False)):
self.AssignError()
value_type = "list"
elif isinstance(value, BaseLayer):
#Specified on the right side'value'But'BaseLayer'Is it a class?
#Error if it is not inherited.
self.AssignError(type(value))
if value_type == "":
value_type = "BaseLayer"
if isinstance(key, slice):
#If key is a slice, overwrite the element in the list of layers.
#However'value_type'But'list'Otherwise an error.
#Unusual value(Index out of range etc.)When is entered
#Python gives me an error.
if value_type != "list":
self.AssignError(value_type)
self.__layer_list[key] = value
elif isinstance(key, str):
#If key is a string, get the index from the name list of each layer and
#Overwrite the element in the list of applicable layers.
#However'value_type'But'BaseLayer'Otherwise an error.
if value_type != "BaseLayer":
raise AssignError(value_type)
if key in self.__name_list:
index = self.__name_list.index(key)
self.__layer_list[index] = value
else:
#If the key does not exist, KeyError is issued.
raise KeyError("{}: No such item".format(key))
elif isinstance(key, int):
#If key is an integer, overwrite the corresponding element in the layer list.
#However'value_type'But'BaseLayer'Otherwise an error.
#Also, an abnormal value(Index out of range etc.)When is entered
#Python gives me an error.
if value_type != "BaseLayer":
raise AssignError(value_type)
self.__layer_list[key] = value
else:
raise KeyError(key, ": Undefined such key type.")
def __delitem__(self, key):
"""
For example
lm = LayerManager()
+----------------+
| (Add element to lm) |
+----------------+
del lm[2]
Because it is called when the element of the list or array is accessed by the del statement like
Describe the operation at that time.
If the specified element exists, it will be deleted and renamed.
"""
if isinstance(key, slice):
#If the key is a slice, delete the specified element as it is
#Unusual value(Index out of range etc.)When is entered
#Python gives me an error.
del self.__layer_list[slice]
del self.__name_list[slice]
elif isinstance(key, str):
#If key is a string, get the index from the name list of each layer and
#Delete the relevant element.
if key in self.__name_list:
del self.__layer_list[index]
del self.__name_list[index]
else:
#If the key does not exist, KeyError is issued.
raise KeyError("{}: No such item".format(key))
elif isinstance(key, int):
#If key is an integer, delete the corresponding element in the layer list.
#Unusual value(Index out of range etc.)When is entered
#Python gives me an error.
del self.__layer_list[key]
else:
raise KeyError(key, ": Undefined such key type.")
#Rename
self._rename()
def _rename(self):
"""
When the name list naming violates the rules due to list operations
Rename the naming list and each layer to meet the rules again.
The naming rule is[Layer type][What number]will do.
If the layer type is Middle Layer, Middle
Output for Output Layer
It is abbreviated as.
The number is counted by type.
Also, here again__Counts ntypes.
"""
#Initialize the number of layers by type
self.__ntype = np.zeros(self.N_TYPE)
#Recount and rename each layer
for i in range(len(self)):
if "Middle" in self.__name_list[i]:
self.__ntype[self.MIDDLE] += 1
self.__name_list[i] = "Middle{}".format(
self.__ntype[self.MIDDLE])
self.__layer_list[i].name = "Middle{}".format(
self.__ntype[self.MIDDLE])
elif "Output" in self.__name_list[i]:
self.__ntype[self.OUTPUT] += 1
self.__name_list[i] = "Output{}".format(
self.__ntype[self.OUTPUT])
self.__layer_list[i].name = "Output{}".format(
self.__ntype[self.OUTPUT])
else:
raise UndefinedLayerType(self.__name_list[i])
def append(self, *, name="Middle", **kwds):
"""
Implementation of the familiar append method, which is a method for adding elements to a list.
"""
if "prev" in kwds:
# 'prev'Is included in the keyword
#This means that the number of elements in the previous layer is specified.
#Basically it is supposed to be the time to insert the first layer, so
#Other than that, it is basically determined automatically and is not specified.
if len(self) != 0:
if kwds["prev"] != self.__layer_list[-1].n:
#Error if it does not match the number of units at the end.
raise UnmatchUnitError(self.__layer_list[-1].n,
kwds["prev"])
else:
if len(self) == 0:
#The first layer must always specify the number of input units.
raise UnmatchUnitError("Input units", "Unspecified")
else:
#The number of units in the last layer'kwds'Add to
kwds["prev"] = self.__layer_list[-1].n
#Read the layer type and change the name according to the naming rule
if name == "mid" or "m":
name = "Middle"
elif name == "out" or "o":
name = "Output"
else:
raise UndefinedLayerError(name)
#Add a layer.
if name == "Middle":
#Increment the layer by type
self.__ntype[self.MIDDLE] += 1
#Add to name
name += str(self.__ntype[self.MIDDLE])
#Add to name list
self.__name_list.append(name)
#Finally, create a layer and add it to the list.
self.__layer_list.append(
MiddleLayer(name=name, **kwds))
elif name == "Output":
#This is also the same.
self.__ntype[self.OUTPUT] += 1
name += str(self.__ntype[self.OUTPUT])
self.__name_list.append(name)
self.__layer_list.append(
OutputLayer(name=name, **kwds))
#If you do not draw an else statement here, change the name according to the naming rule
#Already abnormal at the stage'name'Is omitted.
def extend(self, lm):
"""
Another layer manager already in the extend method'lm'Elements of
Add all.
"""
if not isinstance(lm, LayerManager):
# 'lm'Error if the instance of is not LayerManager.
raise TypeError(type(lm), ": Unexpected type.")
if len(self) != 0:
if self.__layer_list[-1].n != lm[0].prev:
#With the number of units in your last layer
# 'lm'Error if the number of inputs in the first layer of is not the same.
raise UnmatchUnitError(self.__layer_list[-1].n,
lm[0].prev)
#Each'extend'Add by method
self.__layer_list.extend(lm.layer_list)
self.__name_list.extend(lm.name_list)
#Rename
self._rename()
def insert(self, prev_name, name="Middle", **kwds):
"""
In the insert method, specify the name of the previous layer and combine it with that layer.
Add an element.
"""
# 'prev_name'Error if does not exist.
if not prev_name in self.__name_list:
raise KeyError(prev_name, ": No such key.")
# 'prev'Is included in the keyword
# 'prev_name'Error if it does not match the number of units in the layer specified in.
if "prev" in kwds:
if kwds["prev"] \
!= self.__layer_list[self.index(prev_name)].n:
raise UnmatchUnitError(
kwds["prev"],
self.__layer_list[self.index(prev_name)].n)
# 'n'Is included in the keyword
if "n" in kwds:
# 'prev_name'If is not the last
if prev_name != self.__name_list[-1]:
#Error if it does not match the number of units in the next layer.
if kwds["n"] != self.__layer_list[
self.index(prev_name)+1].prev:
raise UnmatchUnitError(
kwds["n"],
self.__layer_list[self.index(prev_name)].prev)
#If there are no elements yet'append'Give an error to use the method.
if len(self) == 0:
raise RuntimeError(
"You have to use 'append' method instead.")
#Get index of insertion location
index = self.index(prev_name) + 1
#Read the layer type and change the name according to the naming rule
if name == "mid" or "m":
name = "Middle"
elif name == "out" or "o":
name = "Output"
else:
raise UndefinedLayerError(name)
#Insert element
#At this time,'name'Does not yet follow the naming rules,
#I'll rename it later so don't worry about it.
if "Middle" in name:
self.__layer_list.insert(index,
MiddleLayer(name=name, **kwds))
self.__name_list.insert(index, name)
elif "Output" in name:
self.__layer_list.insert(index,
OutputLayer(name=name, **kwds))
self.__name_list.insert(index, name)
#Rename
self._rename()
def extend_insert(self, prev_name, lm):
"""
This is the original function.
It behaves like a combination of extend and insert methods.
Simply put, it's like inserting another layer manager.
"""
if not isinstance(lm, LayerManager):
# 'lm'Error if the instance of is not LayerManager.
raise TypeError(type(lm), ": Unexpected type.")
# 'prev_name'Error if does not exist.
if not prev_name in self.__name_list:
raise KeyError(prev_name, ": No such key.")
#The number of units of the layers before and after the specified location and the first and last layers of lm
#If they do not match, an error occurs.
if len(self) != 0:
if self.__layer_list[self.index(prev_name)].n \
!= lm.layer_list[0].prev:
#With the number of units in your designated location'lm'The first number of units in
#If they do not match, an error occurs.
raise UnmatchUnitError(
self.__layer_list[self.index(prev_name)].n,
lm.layer_list[0].prev)
if prev_name != self.__name_list[-1]:
# 'prev_name'Is not my last layer
if lm.layer_list[-1].n \
!= self.__layer_list[self.index(prev_name)+1].prev:
# 'lm'The number of units at the end of and the layer next to your designated location
# 'prev'Error if it does not match the number of units.
raise UnmatchUnitError(
lm.layer_list[-1].n,
self.__layer_list[self.index(prev_name)+1].prev)
else:
#If you don't have any elements'extend'I get an error to use the method.
raise RuntimeError(
"You have to use 'extend' method instead.")
#Get index of insertion location
index = self.index(prev_name) + 1
#Elements after the insertion location'buf'After evacuating to, remove it once and
#Add an element using the extend method
layer_buf = self.__layer_list[index:]
name_buf = self.__name_list[index:]
del self.__layer_list[index:]
del self.__name_list[index:]
self.extend(lm)
#Add the element that was evacuated
self.__layer_list.extend(layer_buf)
self.__name_list.extend(name_buf)
#Rename
self._rename()
def remove(self, key):
"""
The remove method removes the element with the specified name.
It is also allowed to be specified by index.
"""
#Already implemented'del'The sentence is OK.
del self[key]
def index(self, target):
return self.__name_list.index(target)
def name(self, indices):
return self.__name_list[indices]
@property
def layer_list(self):
return self.__layer_list
@property
def name_list(self):
return self.__name_list
@property
def ntype(self):
return self.__ntype
Introducing the _layererror
module. For the time being, only the errors that have appeared in the implementation so far are implemented.
This may be added as well.
_layererror.py
class LayerManagerError(Exception):
"""Base class for user-defined errors in layer modules"""
pass
class AssignError(LayerManagerError):
def __init__(self, value=None):
if not value is None:
self.value = value
self.message = (str(value)
+ ": Assigning that value is prohibited.")
else:
self.value = None
self.message = "Assigning that value is prohibited."
def __str__(self):
return self.message
class UnmatchUnitError(LayerManagerError):
def __init__(self, prev, n):
self.prev = prev
self.n = n
self.message = "Unmatch units: {} and {}.".format(prev, n)
def __str__(self):
return self.message
class UndefinedLayerError(LayerManagerError):
def __init__(self, type_name):
self.type = type_name
self.message = str(type_name) + ": Undefined layer type."
def __str__(self):
return self.message
the first
_layererror.py
class LayerManagerError(Exception):
"""Base class for user-defined errors in layer modules"""
pass
Catch exceptions for layers
example.py
try:
raise AssignError()
except LayerManagerError:
print("catch LayerManagerError")
This is to make it possible to catch all the errors that inherit from this. Of course, you can also catch errors individually.
Also, the error statement can be output only after defining the __str__
method.
So, here I created a layer object template, a manager to handle it, and a module for error control. We will continue to add the contents of the implementation in the next and subsequent articles.
baselayer.py
import numpy as np
class BaseLayer():
"""
All underlying layer classes
Describe the processing common to the intermediate layer and the output layer.
"""
def __init__(self):
pass
def forward(self):
"""
Implementation of forward propagation
"""
pass
def backward(self):
"""
Implementation of backpropagation
"""
pass
def update(self):
"""
Implementation of parameter learning
"""
pass
middlelayer.py
import numpy as np
#import .baselayer import BaseLayer
class MiddleLayer(BaseLayer):
"""
Middle class
The input layer is also treated as one of the intermediate layers in mounting.
"""
pass
outputlayer.py
import numpy as np
#from .baselayer import BaseLayer
class OutputLayer(BaseLayer):
"""
Output layer class
"""
pass
layermanager.py
import numpy as np
#from .layer import *
class _TypeManager():
"""
Manager class for layer types
"""
N_TYPE = 2 #Number of layer types
MIDDLE = 0 #Middle layer numbering
OUTPUT = 1 #Output layer numbering
class LayerManager(_TypeManager):
"""
Manager class for managing layers
"""
def __init__(self):
self.__layer_list = [] #List of layers
self.__name_list = [] #Name list for each layer
self.__ntype = np.zeros(self.N_TYPE) #Number of layers by type
def __repr__(self):
layerRepr= "layer_list: " + repr(self.__layer_list)
nameRepr = "name_list: " + repr(self.__name_list)
ntypeRepr = "ntype: " + repr(self.__ntype)
return (layerRepr + "\n"
+ nameRepr + "\n"
+ ntypeRepr)
def __str__(self):
layerStr = "layer_list: " + str(self.__layer_list)
nameStr = "name_list: " + str(self.__name_list)
ntypeStr = "ntype: " + str(self.__ntype)
return (layerStr + "\n"
+ nameStr + "\n"
+ ntypeStr)
def __len__(self):
"""
Python built-in functions`len`Describes the operation when called from.
Returns the sum of the number of layers by type.
"""
return np.sum(self.__ntype)
def __getitem__(self, key):
"""
For example
lm = LayerManager()
+----------------+
| (Add element to lm) |
+----------------+
x = lm[3].~~
Because it is called when an element of a list or array is accessed, like
Describe the operation at that time.
slice and str,Only allow access via int.
"""
if isinstance(key, slice):
#If the key is a slice, refer to the list of layers with slice.
#Unusual value(Index out of range etc.)When is entered
#Python gives me an error.
return self.__layer_list[key]
elif isinstance(key, str):
#If key is a string, get the index from the name list of each layer and
#Returns the elements of the list of applicable layers.
if key in self.__name_list:
index = self.__name_list.index(key)
return self.__layer_list[index]
else:
#If the key does not exist, KeyError is issued.
raise KeyError("{}: No such item".format(key))
elif isinstance(key, int):
#If key is an integer, returns the corresponding element in the list of layers.
#Unusual value(Index out of range etc.)When is entered
#Python gives me an error.
return self.__layer_list[key]
else:
raise KeyError(key, ": Undefined such key type.")
def __setitem__(self, key, value):
"""
For example
lm = LayerManager()
+----------------+
| (Add element to lm) |
+----------------+
lm[1] = x
Because it is called when an element of a list or array is accessed, like
Describe the operation at that time.
Only overwriting elements is allowed, and adding new elements is prohibited.
"""
value_type = ""
if isinstance(value, list):
#Specified on the right side'value'But'list'If
#All elements'BaseLayer'Error if class or not inheriting it.
if not np.all(
np.where(isinstance(value, BaseLayer), True, False)):
self.AssignError()
value_type = "list"
elif isinstance(value, BaseLayer):
#Specified on the right side'value'But'BaseLayer'Is it a class?
#Error if it is not inherited.
self.AssignError(type(value))
if value_type == "":
value_type = "BaseLayer"
if isinstance(key, slice):
#If key is a slice, overwrite the element in the list of layers.
#However'value_type'But'list'Otherwise an error.
#Unusual value(Index out of range etc.)When is entered
#Python gives me an error.
if value_type != "list":
self.AssignError(value_type)
self.__layer_list[key] = value
elif isinstance(key, str):
#If key is a string, get the index from the name list of each layer and
#Overwrite the element in the list of applicable layers.
#However'value_type'But'BaseLayer'Otherwise an error.
if value_type != "BaseLayer":
raise AssignError(value_type)
if key in self.__name_list:
index = self.__name_list.index(key)
self.__layer_list[index] = value
else:
#If the key does not exist, KeyError is issued.
raise KeyError("{}: No such item".format(key))
elif isinstance(key, int):
#If key is an integer, overwrite the corresponding element in the layer list.
#However'value_type'But'BaseLayer'Otherwise an error.
#Also, an abnormal value(Index out of range etc.)When is entered
#Python gives me an error.
if value_type != "BaseLayer":
raise AssignError(value_type)
self.__layer_list[key] = value
else:
raise KeyError(key, ": Undefined such key type.")
def __delitem__(self, key):
"""
For example
lm = LayerManager()
+----------------+
| (Add element to lm) |
+----------------+
del lm[2]
Because it is called when the element of the list or array is accessed by the del statement like
Describe the operation at that time.
If the specified element exists, it will be deleted and renamed.
"""
if isinstance(key, slice):
#If the key is a slice, delete the specified element as it is
#Unusual value(Index out of range etc.)When is entered
#Python gives me an error.
del self.__layer_list[slice]
del self.__name_list[slice]
elif isinstance(key, str):
#If key is a string, get the index from the name list of each layer and
#Delete the relevant element.
if key in self.__name_list:
del self.__layer_list[index]
del self.__name_list[index]
else:
#If the key does not exist, KeyError is issued.
raise KeyError("{}: No such item".format(key))
elif isinstance(key, int):
#If key is an integer, delete the corresponding element in the layer list.
#Unusual value(Index out of range etc.)When is entered
#Python gives me an error.
del self.__layer_list[key]
else:
raise KeyError(key, ": Undefined such key type.")
#Rename
self._rename()
def _rename(self):
"""
When the name list naming violates the rules due to list operations
Rename the naming list and each layer to meet the rules again.
The naming rule is[Layer type][What number]will do.
If the layer type is Middle Layer, Middle
Output for Output Layer
It is abbreviated as.
The number is counted by type.
Also, here again__Counts ntypes.
"""
#Initialize the number of layers by type
self.__ntype = np.zeros(self.N_TYPE)
#Recount and rename each layer
for i in range(len(self)):
if "Middle" in self.__name_list[i]:
self.__ntype[self.MIDDLE] += 1
self.__name_list[i] = "Middle{}".format(
self.__ntype[self.MIDDLE])
self.__layer_list[i].name = "Middle{}".format(
self.__ntype[self.MIDDLE])
elif "Output" in self.__name_list[i]:
self.__ntype[self.OUTPUT] += 1
self.__name_list[i] = "Output{}".format(
self.__ntype[self.OUTPUT])
self.__layer_list[i].name = "Output{}".format(
self.__ntype[self.OUTPUT])
else:
raise UndefinedLayerType(self.__name_list[i])
def append(self, *, name="Middle", **kwds):
"""
Implementation of the familiar append method, which is a method for adding elements to a list.
"""
if "prev" in kwds:
# 'prev'Is included in the keyword
#This means that the number of elements in the previous layer is specified.
#Basically it is supposed to be the time to insert the first layer, so
#Other than that, it is basically determined automatically and is not specified.
if len(self) != 0:
if kwds["prev"] != self.__layer_list[-1].n:
#Error if it does not match the number of units at the end.
raise UnmatchUnitError(self.__layer_list[-1].n,
kwds["prev"])
else:
if len(self) == 0:
#The first layer must always specify the number of input units.
raise UnmatchUnitError("Input units", "Unspecified")
else:
#The number of units in the last layer'kwds'Add to
kwds["prev"] = self.__layer_list[-1].n
#Read the layer type and change the name according to the naming rule
if name == "mid" or "m":
name = "Middle"
elif name == "out" or "o":
name = "Output"
else:
raise UndefinedLayerError(name)
#Add a layer.
if name == "Middle":
#Increment the layer by type
self.__ntype[self.MIDDLE] += 1
#Add to name
name += str(self.__ntype[self.MIDDLE])
#Add to name list
self.__name_list.append(name)
#Finally, create a layer and add it to the list.
self.__layer_list.append(
MiddleLayer(name=name, **kwds))
elif name == "Output":
#This is also the same.
self.__ntype[self.OUTPUT] += 1
name += str(self.__ntype[self.OUTPUT])
self.__name_list.append(name)
self.__layer_list.append(
OutputLayer(name=name, **kwds))
#If you do not draw an else statement here, change the name according to the naming rule
#Already abnormal at the stage'name'Is omitted.
def extend(self, lm):
"""
Another layer manager already in the extend method'lm'Elements of
Add all.
"""
if not isinstance(lm, LayerManager):
# 'lm'Error if the instance of is not LayerManager.
raise TypeError(type(lm), ": Unexpected type.")
if len(self) != 0:
if self.__layer_list[-1].n != lm[0].prev:
#With the number of units in your last layer
# 'lm'Error if the number of inputs in the first layer of is not the same.
raise UnmatchUnitError(self.__layer_list[-1].n,
lm[0].prev)
#Each'extend'Add by method
self.__layer_list.extend(lm.layer_list)
self.__name_list.extend(lm.name_list)
#Rename
self._rename()
def insert(self, prev_name, name="Middle", **kwds):
"""
In the insert method, specify the name of the previous layer and combine it with that layer.
Add an element.
"""
# 'prev_name'Error if does not exist.
if not prev_name in self.__name_list:
raise KeyError(prev_name, ": No such key.")
# 'prev'Is included in the keyword
# 'prev_name'Error if it does not match the number of units in the layer specified in.
if "prev" in kwds:
if kwds["prev"] \
!= self.__layer_list[self.index(prev_name)].n:
raise UnmatchUnitError(
kwds["prev"],
self.__layer_list[self.index(prev_name)].n)
# 'n'Is included in the keyword
if "n" in kwds:
# 'prev_name'If is not the last
if prev_name != self.__name_list[-1]:
#Error if it does not match the number of units in the next layer.
if kwds["n"] != self.__layer_list[
self.index(prev_name)+1].prev:
raise UnmatchUnitError(
kwds["n"],
self.__layer_list[self.index(prev_name)].prev)
#If there are no elements yet'append'Give an error to use the method.
if len(self) == 0:
raise RuntimeError(
"You have to use 'append' method instead.")
#Get index of insertion location
index = self.index(prev_name) + 1
#Read the layer type and change the name according to the naming rule
if name == "mid" or "m":
name = "Middle"
elif name == "out" or "o":
name = "Output"
else:
raise UndefinedLayerError(name)
#Insert element
#At this time,'name'Does not yet follow the naming rules,
#I'll rename it later so don't worry about it.
if "Middle" in name:
self.__layer_list.insert(index,
MiddleLayer(name=name, **kwds))
self.__name_list.insert(index, name)
elif "Output" in name:
self.__layer_list.insert(index,
OutputLayer(name=name, **kwds))
self.__name_list.insert(index, name)
#Rename
self._rename()
def extend_insert(self, prev_name, lm):
"""
This is the original function.
It behaves like a combination of extend and insert methods.
Simply put, it's like inserting another layer manager.
"""
if not isinstance(lm, LayerManager):
# 'lm'Error if the instance of is not LayerManager.
raise TypeError(type(lm), ": Unexpected type.")
# 'prev_name'Error if does not exist.
if not prev_name in self.__name_list:
raise KeyError(prev_name, ": No such key.")
#The number of units of the layers before and after the specified location and the first and last layers of lm
#If they do not match, an error occurs.
if len(self) != 0:
if self.__layer_list[self.index(prev_name)].n \
!= lm.layer_list[0].prev:
#With the number of units in your designated location'lm'The first number of units in
#If they do not match, an error occurs.
raise UnmatchUnitError(
self.__layer_list[self.index(prev_name)].n,
lm.layer_list[0].prev)
if prev_name != self.__name_list[-1]:
# 'prev_name'Is not my last layer
if lm.layer_list[-1].n \
!= self.__layer_list[self.index(prev_name)+1].prev:
# 'lm'The number of units at the end of and the layer next to your designated location
# 'prev'Error if it does not match the number of units.
raise UnmatchUnitError(
lm.layer_list[-1].n,
self.__layer_list[self.index(prev_name)+1].prev)
else:
#If you don't have any elements'extend'I get an error to use the method.
raise RuntimeError(
"You have to use 'extend' method instead.")
#Get index of insertion location
index = self.index(prev_name) + 1
#Elements after the insertion location'buf'After evacuating to, remove it once and
#Add an element using the extend method
layer_buf = self.__layer_list[index:]
name_buf = self.__name_list[index:]
del self.__layer_list[index:]
del self.__name_list[index:]
self.extend(lm)
#Add the element that was evacuated
self.__layer_list.extend(layer_buf)
self.__name_list.extend(name_buf)
#Rename
self._rename()
def remove(self, key):
"""
The remove method removes the element with the specified name.
It is also allowed to be specified by index.
"""
#Already implemented'del'The sentence is OK.
del self[key]
def index(self, target):
return self.__name_list.index(target)
def name(self, indices):
return self.__name_list[indices]
@property
def layer_list(self):
return self.__layer_list
@property
def name_list(self):
return self.__name_list
@property
def ntype(self):
return self.__ntype
_layererror.py
class LayerManagerError(Exception):
"""Base class for user-defined errors in layer modules"""
pass
class AssignError(LayerManagerError):
def __init__(self, value=None):
if not value is None:
self.value = value
self.message = (str(value)
+ ": Assigning that value is prohibited.")
else:
self.value = None
self.message = "Assigning that value is prohibited."
def __str__(self):
return self.message
class UnmatchUnitError(LayerManagerError):
def __init__(self, prev, n):
self.prev = prev
self.n = n
self.message = "Unmatch units: {} and {}.".format(prev, n)
def __str__(self):
return self.message
class UndefinedLayerError(LayerManagerError):
def __init__(self, type_name):
self.type = type_name
self.message = str(type_name) + ": Undefined layer type."
def __str__(self):
return self.message
-[Let's master Python's underscore (_)! ](Https://medium.com/lsc-psd/pythonic Various-python underscore-Let's master it-3c132842eeef)
-Introduction to Deep Learning ~ Basics ~ -Introduction to Deep Learning ~ Forward Propagation ~ -Thorough understanding of im2col -List of activation functions (2020)
Recommended Posts