Estimation de la profondeur par CNN (CNN SLAM # 3)
Je voudrais mentionner ici l'estimation des images de profondeur à l'aide de CNN dans CNN SLAM.
Dans l'article, la tâche de diviser la zone est également résolue par CNN, mais ici je voudrais me concentrer uniquement sur l'estimation de l'image de profondeur. Quoi qu'il en soit, la configuration de base du réseau est à peu près la même, vous n'avez donc pas à vous soucier trop de la compréhension de CNN SLAM.
La réponse est sur github
Dieu.
https://github.com/iro-cp/FCRN-DepthPrediction
Vous pouvez estimer la profondeur en clonant ceci!
recette
$ git clone https://github.com/iro-cp/FCRN-DepthPrediction.git
$ cd FCRN-DepthPrediction/tensorflow
$ python predict.py <Fichier de modèle formé> <Image RVB que vous souhaitez estimer>
J'ai besoin de tensorflow, alors installez-le. J'ai Python 3.5.3 et tensorflow a 1.2.1.
Le fichier de modèle entraîné est lié au README sur la page github ci-dessus. Vous n'avez pas besoin de vous entraîner ici car vous pouvez obtenir le modèle en cliquant à l'endroit où il est indiqué Modèle Tensorflow.
L'image dont vous souhaitez estimer la profondeur est préparée ici. Par exemple, si vous donnez l'image suivante et laissez-la estimer ...
Vous pouvez obtenir ce type de sortie.
Je ne peux pas l'évaluer car je ne connais pas la vraie valeur, mais il ne semble pas être en mauvais état.
Et si tu veux apprendre?
Si vous souhaitez également apprendre, vous devez mettre True dans l'argument entraînable de la classe Network dans Network.py, calculer la différence à partir de l'image correcte et ajouter un nouveau code pour la propagation arrière.
J'ai personnellement vu le style d'écriture dans fcn.py pour la première fois, donc je ne savais pas comment jouer avec. Alors, voici ce que j'ai changé de manière plus simple (pour moi):
network.py
'''
La fonction de configuration est une fonction membre de la classe Network
'''
def setup(self, trainable):
xs = self.inputs['data']
conv1 = self.conv(xs, 7, 7, 64, 2, 2, relu=False, name='conv1')
bn_conv1 = self.batch_normalization(conv1, relu=True, name='bn_conv1')
pool1 = self.max_pool(bn_conv1, 3, 3, 2, 2, name='pool1')
res2a_branch1 = self.conv(pool1, 1, 1, 256, 1, 1, biased=False, relu=False, name='res2a_branch1')
bn2a_branch1 = self.batch_normalization(res2a_branch1, name='bn2a_branch1')
res2a_branch2a = self.conv(pool1, 1, 1, 64, 1, 1, biased=False, relu=False, name='res2a_branch2a')
bn2a_branch2a = self.batch_normalization(res2a_branch2a, relu=True, name='bn2a_branch2a')
res2a_branch2b = self.conv(bn2a_branch2a, 3, 3, 64, 1, 1, biased=False, relu=False, name='res2a_branch2b')
bn2a_branch2b = self.batch_normalization(res2a_branch2b, relu=True, name='bn2a_branch2b')
res2a_branch2c = self.conv(bn2a_branch2b, 1, 1, 256, 1, 1, biased=False, relu=False, name='res2a_branch2c')
bn2a_branch2c = self.batch_normalization(res2a_branch2c, name='bn2a_branch2c')
res2a = self.add((bn2a_branch1, bn2a_branch2c), name='res2a')
res2a_relu = self.relu(res2a, name='res2a_relu')
res2b_branch2a = self.conv(res2a_relu, 1, 1, 64, 1, 1, biased=False, relu=False, name='res2b_branch2a')
bn2b_branch2a = self.batch_normalization(res2b_branch2a, relu=True, name='bn2b_branch2a')
res2b_branch2b = self.conv(bn2b_branch2a, 3, 3, 64, 1, 1, biased=False, relu=False, name='res2b_branch2b')
bn2b_branch2b = self.batch_normalization(res2b_branch2b, relu=True, name='bn2b_branch2b')
res2b_branch2c = self.conv(bn2b_branch2b, 1, 1, 256, 1, 1, biased=False, relu=False, name='res2b_branch2c')
bn2b_branch2c = self.batch_normalization(res2b_branch2c, name='bn2b_branch2c')
res2b = self.add((res2a_relu, bn2b_branch2c), name='res2b')
res2b_relu = self.relu(res2b, name='res2b_relu')
res2c_branch2a = self.conv(res2b_relu, 1, 1, 64, 1, 1, biased=False, relu=False, name='res2c_branch2a')
bn2c_branch2a = self.batch_normalization(res2c_branch2a, relu=True, name='bn2c_branch2a')
res2c_branch2b = self.conv(bn2c_branch2a, 3, 3, 64, 1, 1, biased=False, relu=False, name='res2c_branch2b')
bn2c_branch2b = self.batch_normalization(res2c_branch2b, relu=True, name='bn2c_branch2b')
res2c_branch2c = self.conv(bn2c_branch2b, 1, 1, 256, 1, 1, biased=False, relu=False, name='res2c_branch2c')
bn2c_branch2c = self.batch_normalization(res2c_branch2c, name='bn2c_branch2c')
res2c = self.add((res2b_relu, bn2c_branch2c), name='res2c')
res2c_relu = self.relu(res2c, name='res2c_relu')
res3a_branch1 = self.conv(res2c_relu, 1, 1, 512, 2, 2, biased=False, relu=False, name='res3a_branch1')
bn3a_branch1 = self.batch_normalization(res3a_branch1, name='bn3a_branch1')
res3a_branch2a = self.conv(res2c_relu, 1, 1, 128, 2, 2, biased=False, relu=False, name='res3a_branch2a')
bn3a_branch2a = self.batch_normalization(res3a_branch2a, relu=True, name='bn3a_branch2a')
res3a_branch2b = self.conv(bn3a_branch2a, 3, 3, 128, 1, 1, biased=False, relu=False, name='res3a_branch2b')
bn3a_branch2b = self.batch_normalization(res3a_branch2b, relu=True, name='bn3a_branch2b')
res3a_branch2c = self.conv(bn3a_branch2b, 1, 1, 512, 1, 1, biased=False, relu=False, name='res3a_branch2c')
bn3a_branch2c = self.batch_normalization(res3a_branch2c, name='bn3a_branch2c')
res3a = self.add((bn3a_branch1, bn3a_branch2c), name='res3a')
res3a_relu = self.relu(res3a, name='res3a_relu')
res3b_branch2a = self.conv(res3a_relu, 1, 1, 128, 1, 1, biased=False, relu=False, name='res3b_branch2a')
bn3b_branch2a = self.batch_normalization(res3b_branch2a, relu=True, name='bn3b_branch2a')
res3b_branch2b = self.conv(bn3b_branch2a, 3, 3, 128, 1, 1, biased=False, relu=False, name='res3b_branch2b')
bn3b_branch2b = self.batch_normalization(res3b_branch2b, relu=True, name='bn3b_branch2b')
res3b_branch2c = self.conv(bn3b_branch2b, 1, 1, 512, 1, 1, biased=False, relu=False, name='res3b_branch2c')
bn3b_branch2c = self.batch_normalization(res3b_branch2c, name='bn3b_branch2c')
res3b = self.add((res3a_relu, bn3b_branch2c), name='res3b')
res3b_relu = self.relu(res3b, name='res3b_relu')
res3c_branch2a = self.conv(res3b_relu, 1, 1, 128, 1, 1, biased=False, relu=False, name='res3c_branch2a')
bn3c_branch2a = self.batch_normalization(res3c_branch2a, relu=True, name='bn3c_branch2a')
res3c_branch2b = self.conv(bn3c_branch2a, 3, 3, 128, 1, 1, biased=False, relu=False, name='res3c_branch2b')
bn3c_branch2b = self.batch_normalization(res3c_branch2b, relu=True, name='bn3c_branch2b')
res3c_branch2c = self.conv(bn3c_branch2b, 1, 1, 512, 1, 1, biased=False, relu=False, name='res3c_branch2c')
bn3c_branch2c = self.batch_normalization(res3c_branch2c, name='bn3c_branch2c')
res3c = self.add((res3b_relu, bn3c_branch2c), name='res3c')
res3c_relu = self.relu(res3c, name='res3c_relu')
res3d_branch2a = self.conv(res3c_relu, 1, 1, 128, 1, 1, biased=False, relu=False, name='res3d_branch2a')
bn3d_branch2a = self.batch_normalization(res3d_branch2a, relu=True, name='bn3d_branch2a')
res3d_branch2b = self.conv(bn3d_branch2a, 3, 3, 128, 1, 1, biased=False, relu=False, name='res3d_branch2b')
bn3d_branch2b = self.batch_normalization(res3d_branch2b, relu=True, name='bn3d_branch2b')
res3d_branch2c = self.conv(bn3d_branch2b, 1, 1, 512, 1, 1, biased=False, relu=False, name='res3d_branch2c')
bn3d_branch2c = self.batch_normalization(res3d_branch2c, name='bn3d_branch2c')
res3d = self.add((res3c_relu, bn3d_branch2c), name='res3d')
res3d_relu = self.relu(res3d, name='res3d_relu')
res4a_branch1 = self.conv(res3d_relu, 1, 1, 1024, 2, 2, biased=False, relu=False, name='res4a_branch1')
bn4a_branch1 = self.batch_normalization(res4a_branch1, name='bn4a_branch1')
res4a_branch2a = self.conv(res3d_relu, 1, 1, 256, 2, 2, biased=False, relu=False, name='res4a_branch2a')
bn4a_branch2a = self.batch_normalization(res4a_branch2a, relu=True, name='bn4a_branch2a')
res4a_branch2b = self.conv(bn4a_branch2a, 3, 3, 256, 1, 1, biased=False, relu=False, name='res4a_branch2b')
bn4a_branch2b = self.batch_normalization(res4a_branch2b, relu=True, name='bn4a_branch2b')
res4a_branch2c = self.conv(bn4a_branch2b, 1, 1, 1024, 1, 1, biased=False, relu=False, name='res4a_branch2c')
bn4a_branch2c = self.batch_normalization(res4a_branch2c, name='bn4a_branch2c')
res4a = self.add((bn4a_branch1, bn4a_branch2c), name='res4a')
res4a_relu = self.relu(res4a, name='res4a_relu')
res4b_branch2a = self.conv(res4a_relu, 1, 1, 256, 1, 1, biased=False, relu=False, name='res4b_branch2a')
bn4b_branch2a = self.batch_normalization(res4b_branch2a, relu=True, name='bn4b_branch2a')
res4b_branch2b = self.conv(bn4b_branch2a, 3, 3, 256, 1, 1, biased=False, relu=False, name='res4b_branch2b')
bn4b_branch2b = self.batch_normalization(res4b_branch2b, relu=True, name='bn4b_branch2b')
res4b_branch2c = self.conv(bn4b_branch2b, 1, 1, 1024, 1, 1, biased=False, relu=False, name='res4b_branch2c')
bn4b_branch2c = self.batch_normalization(res4b_branch2c, name='bn4b_branch2c')
res4b = self.add((res4a_relu, bn4b_branch2c), name='res4b')
res4b_relu = self.relu(res4b, name='res4b_relu')
res4c_branch2a = self.conv(res4b_relu, 1, 1, 256, 1, 1, biased=False, relu=False, name='res4c_branch2a')
bn4c_branch2a = self.batch_normalization(res4c_branch2a, relu=True, name='bn4c_branch2a')
res4c_branch2b = self.conv(bn4c_branch2a, 3, 3, 256, 1, 1, biased=False, relu=False, name='res4c_branch2b')
bn4c_branch2b = self.batch_normalization(res4c_branch2b, relu=True, name='bn4c_branch2b')
res4c_branch2c = self.conv(bn4c_branch2b, 1, 1, 1024, 1, 1, biased=False, relu=False, name='res4c_branch2c')
bn4c_branch2c = self.batch_normalization(res4c_branch2c, name='bn4c_branch2c')
res4c = self.add((res4b_relu, bn4c_branch2c), name='res4c')
res4c_relu = self.relu(res4c, name='res4c_relu')
res4d_branch2a = self.conv(res4c_relu, 1, 1, 256, 1, 1, biased=False, relu=False, name='res4d_branch2a')
bn4d_branch2a = self.batch_normalization(res4d_branch2a, relu=True, name='bn4d_branch2a')
res4d_branch2b = self.conv(bn4d_branch2a, 3, 3, 256, 1, 1, biased=False, relu=False, name='res4d_branch2b')
bn4d_branch2b = self.batch_normalization(res4d_branch2b, relu=True, name='bn4d_branch2b')
res4d_branch2c = self.conv(bn4d_branch2b, 1, 1, 1024, 1, 1, biased=False, relu=False, name='res4d_branch2c')
bn4d_branch2c = self.batch_normalization(res4d_branch2c, name='bn4d_branch2c')
res4d = self.add((res4c_relu, bn4d_branch2c), name='res4d')
res4d_relu = self.relu(res4d, name='res4d_relu')
res4e_branch2a = self.conv(res4d_relu, 1, 1, 256, 1, 1, biased=False, relu=False, name='res4e_branch2a')
bn4e_branch2a = self.batch_normalization(res4e_branch2a, relu=True, name='bn4e_branch2a')
res4e_branch2b = self.conv(bn4e_branch2a, 3, 3, 256, 1, 1, biased=False, relu=False, name='res4e_branch2b')
bn4e_branch2b = self.batch_normalization(res4e_branch2b, relu=True, name='bn4e_branch2b')
res4e_branch2c = self.conv(bn4e_branch2b, 1, 1, 1024, 1, 1, biased=False, relu=False, name='res4e_branch2c')
bn4e_branch2c = self.batch_normalization(res4e_branch2c, name='bn4e_branch2c')
res4e = self.add((res4d_relu, bn4e_branch2c), name='res4e')
res4e_relu = self.relu(res4e, name='res4e_relu')
res4f_branch2a = self.conv(res4e_relu, 1, 1, 256, 1, 1, biased=False, relu=False, name='res4f_branch2a')
bn4f_branch2a = self.batch_normalization(res4f_branch2a, relu=True, name='bn4f_branch2a')
res4f_branch2b = self.conv(bn4f_branch2a, 3, 3, 256, 1, 1, biased=False, relu=False, name='res4f_branch2b')
bn4f_branch2b = self.batch_normalization(res4f_branch2b, relu=True, name='bn4f_branch2b')
res4f_branch2c = self.conv(bn4f_branch2b, 1, 1, 1024, 1, 1, biased=False, relu=False, name='res4f_branch2c')
bn4f_branch2c = self.batch_normalization(res4f_branch2c, name='bn4f_branch2c')
res4f = self.add((res4e_relu, bn4f_branch2c), name='res4f')
res4f_relu = self.relu(res4f, name='res4f_relu')
res5a_branch1 = self.conv(res4f_relu, 1, 1, 2048, 2, 2, biased=False, relu=False, name='res5a_branch1')
bn5a_branch1 = self.batch_normalization(res5a_branch1, name='bn5a_branch1')
res5a_branch2a = self.conv(res4f_relu, 1, 1, 512, 2, 2, biased=False, relu=False, name='res5a_branch2a')
bn5a_branch2a = self.batch_normalization(res5a_branch2a, relu=True, name='bn5a_branch2a')
res5a_branch2b = self.conv(bn5a_branch2a, 3, 3, 512, 1, 1, biased=False, relu=False, name='res5a_branch2b')
bn5a_branch2b = self.batch_normalization(res5a_branch2b, relu=True, name='bn5a_branch2b')
res5a_branch2c = self.conv(bn5a_branch2b, 1, 1, 2048, 1, 1, biased=False, relu=False, name='res5a_branch2c')
bn5a_branch2c = self.batch_normalization(res5a_branch2c, name='bn5a_branch2c')
res5a = self.add((bn5a_branch1, bn5a_branch2c), name='res5a')
res5a_relu = self.relu(res5a, name='res5a_relu')
res5b_branch2a = self.conv(res5a_relu, 1, 1, 512, 1, 1, biased=False, relu=False, name='res5b_branch2a')
bn5b_branch2a = self.batch_normalization(res5b_branch2a, relu=True, name='bn5b_branch2a')
res5b_branch2b = self.conv(bn5b_branch2a, 3, 3, 512, 1, 1, biased=False, relu=False, name='res5b_branch2b')
bn5b_branch2b = self.batch_normalization(res5b_branch2b, relu=True, name='bn5b_branch2b')
res5b_branch2c = self.conv(bn5b_branch2b, 1, 1, 2048, 1, 1, biased=False, relu=False, name='res5b_branch2c')
bn5b_branch2c = self.batch_normalization(res5b_branch2c, name='bn5b_branch2c')
res5b = self.add((res5a_relu, bn5b_branch2c), name='res5b')
res5b_relu = self.relu(res5b, name='res5b_relu')
res5c_branch2a = self.conv(res5b_relu, 1, 1, 512, 1, 1, biased=False, relu=False, name='res5c_branch2a')
bn5c_branch2a = self.batch_normalization(res5c_branch2a, relu=True, name='bn5c_branch2a')
res5c_branch2b = self.conv(bn5c_branch2a, 3, 3, 512, 1, 1, biased=False, relu=False, name='res5c_branch2b')
bn5c_branch2b = self.batch_normalization(res5c_branch2b, relu=True, name='bn5c_branch2b')
res5c_branch2c = self.conv(bn5c_branch2b, 1, 1, 2048, 1, 1, biased=False, relu=False, name='res5c_branch2c')
bn5c_branch2c = self.batch_normalization(res5c_branch2c, name='bn5c_branch2c')
res5c = self.add((res5b_relu, bn5c_branch2c), name='res5c')
res5c_relu = self.relu(res5c, name='res5c_relu')
layer1 = self.conv(res5c_relu, 1, 1, 1024, 1, 1, biased=True, relu=False, name='layer1')
layer1_BN = self.batch_normalization(layer1, relu=False, name='layer1_BN')
layer2 = self.up_project(layer1_BN, [3, 3, 1024, 512], id='2x', stride=1, BN=True)
layer3 = self.up_project(layer2, [3, 3, 512, 256], id='4x', stride=1, BN=True)
layer4 = self.up_project(layer3, [3, 3, 256, 128], id='8x', stride=1, BN=True)
layer5 = self.up_project(layer4, [3, 3, 128, 64], id='16x', stride=1, BN=True)
layer5_drop = self.dropout(layer5, name='drop', keep_prob=1.)
self.predict = self.conv(layer5_drop, 3, 3, 1, 1, 1, name='ConvPred')
if trainable:
ts = self.inputs['true']
differ = tf.subtract(x=self.predict, y=ts)
abs_differ = tf.abs(differ)
self.loss = tf.reduce_mean(abs_differ, name='loss')
self.train_step = tf.train.GradientDescentOptimizer(0.001).minimize(self.loss)
self.inputs est un type de dictionnaire, et lorsque trainable == True, il existe un espace réservé pour stocker l'image d'entrée et un espace réservé pour stocker l'image de réponse correcte, et lorsque False, il n'y a qu'un espace réservé pour stocker l'image d'entrée afin que le programme fonctionne correctement. Je le fais.
Nous étudions toujours, mais il semble que la précision de l'estimation diffère selon la fonction de perte. Je voudrais utiliser la fonction de perte qui donne le plus de précision lors de l'exécution d'un apprentissage supplémentaire.
Recommended Posts