[TF] How to save and load Tensorflow learning parameters

Use ** tf.train.Saver ** to save and load parameters learned in Tensorflow.

Save

When saving, use the ** save ** method of the created saver class.

python


saver = tf.train.Saver()

Some processing
 
#Save
saver.save(sess, "model.ckpt")

Saving can be at the end of learning or at the middle of learning.

Read

When reading, use the ** restore ** method of the created saver class. Since session is required, load it after creating session. When running on ipython, create a session with tf.InteractiveSession (), usually tf.Session ().

python


sess=tf.InteractiveSession()

saver.restore(sess, "model.ckpt")

The state of actually saving and loading is shown below.

The flow is as follows.

    1. Modeling
  1. Learning
    1. Save parameters to another variable for later comparison
  2. Save parameters to file 5.Session Close
  3. Session creation
  4. Initialization (This is not necessary in the first place. It was intentionally initialized for comparison.)
  5. Compare with the saved parameters (This is different because it was initialized one time ago.)
  6. Read parameters from file
  7. Compare with saved parameters (this matches)
  8. Learning

TF_SaveAndRestoreModel-20-1-html.png

code

python


# # import

# In[1]:

import tensorflow as tf
import numpy as np
from tensorflow.examples.tutorials.mnist import input_data


# # load dataset

# In[2]:

mnist = input_data.read_data_sets("./data/mnist/", one_hot=True) 


# # build model

# In[3]:

def mlp(x, output_dim, reuse=False):
        
    w1 = tf.get_variable("w1", [x.get_shape()[1], 1024], initializer=tf.random_normal_initializer())
    b1 = tf.get_variable("b1", [1024], initializer=tf.constant_initializer(0.0))
    w2 = tf.get_variable("w2", [1024, output_dim], initializer=tf.random_normal_initializer())
    b2 = tf.get_variable("b2", [output_dim], initializer=tf.constant_initializer(0.0))
    
    fc1 = tf.nn.relu(tf.matmul(x, w1) + b1)
    fc2 = tf.matmul(fc1, w2) + b2

    return fc2, [w1, b1, w2, b2]

def slp(x, output_dim):
    w1 = tf.get_variable("w1", [x.get_shape()[1], output_dim], initializer=tf.random_normal_initializer())
    b1 = tf.get_variable("b1", [output_dim], initializer=tf.constant_initializer(0.0))
    
    fc1 = tf.nn.relu(tf.matmul(x, w1) + b1)
    return fc1, [w1, b1]

n_batch = 32
n_label = 10
n_train = 10000
imagesize = 28
learning_rate = 0.5

x_node = tf.placeholder(tf.float32, shape=(n_batch, imagesize*imagesize))
y_node = tf.placeholder(tf.float32, shape=(n_batch, n_label))

with tf.variable_scope("MLP") as scope:
    out_m, theta_m = mlp(x_node, n_label)
           
loss = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(out_m, y_node))
opt  = tf.train.GradientDescentOptimizer(learning_rate).minimize(loss)
tr_pred = tf.nn.softmax(out_m)

test_data = mnist.test.images
test_labels = mnist.test.labels
tx = tf.constant(test_data)
ty_ = tf.constant(test_labels)

with tf.variable_scope("MLP") as scope:
    scope.reuse_variables()
    ty, _ = mlp(tx, n_label)
    
te_pred = tf.nn.softmax(ty) 


# In[4]:

def accuracy(y, y_):
    return 100.0 * np.sum(np.argmax(y, 1) == np.argmax(y_, 1)) / y.shape[0]


# In[5]:

saver = tf.train.Saver()

sess=tf.InteractiveSession()

init = tf.initialize_all_variables()
sess.run(init)


# In[6]:

for step in xrange(n_train):
    bx, by = mnist.train.next_batch(n_batch)
    feed_dict = {x_node: bx, y_node: by}
    _, _loss, _tr_pred = sess.run([opt, loss, tr_pred], feed_dict=feed_dict)
    if step % 500 == 0:
        _accuracy = accuracy(_tr_pred, by)
        print 'step = %d, loss=%.2f, accuracy=%.2f' % (step, _loss, _accuracy)

print 'test accuracy=%.2f' % accuracy(te_pred.eval(), test_labels)


# In[8]:

old_theta_m = [ p.eval() for p in theta_m] # for comparing


# In[9]:

saver.save(sess, "model.ckpt")


# In[10]:

sess.close()


# In[11]:

sess=tf.InteractiveSession()

# for clear
init = tf.initialize_all_variables()
sess.run(init)


# In[12]:

for prm, prm_o in zip(theta_m, old_theta_m):
    p1 = prm.eval()
    p2 = prm_o
    print np.sum(p1 != p2) 


# In[13]:

saver.restore(sess, "model.ckpt")


# In[14]:

for prm, prm_o in zip(theta_m, old_theta_m):
    p1 = prm.eval()
    p2 = prm_o
    print np.sum(p1 != p2) 


# In[15]:

print 'test accuracy=%.2f' % accuracy(te_pred.eval(), test_labels)


# In[16]:

for step in xrange(n_train):
    bx, by = mnist.train.next_batch(n_batch)
    feed_dict = {x_node: bx, y_node: by}
    _, _loss, _tr_pred = sess.run([opt, loss, tr_pred], feed_dict=feed_dict)
    if step % 500 == 0:
        _accuracy = accuracy(_tr_pred, by)
        print 'step = %d, loss=%.2f, accuracy=%.2f' % (step, _loss, _accuracy)

print 'test accuracy=%.2f' % accuracy(te_pred.eval(), test_labels)


# In[17]:

sess.close()


# In[ ]:

tf.reset_default_graph()

Recommended Posts

[TF] How to save and load Tensorflow learning parameters
[TF] How to load / save Model and Parameter in Keras
How to split and save a DataFrame
[TF] How to build Tensorflow in Proxy environment
How to share folders with Docker and Windows with tensorflow
[How to!] Learn and play Super Mario with Tensorflow !!
[TensorFlow 2 / Keras] How to run learning with CTC Loss in Keras
[Google Colab] How to interrupt learning and then resume it
How to learn TensorFlow for liberal arts and Python beginners
How to interactively draw a machine learning pipeline with scikit-learn and save it in HTML
How to install and use Tesseract-OCR
How to install and configure blackbird
How to use .bash_profile and .bashrc
How to install CUDA and nvidia-driver
How to install and use Graphviz
How to convert Tensorflow model to Lite
I summarized how to change the boot parameters of GRUB and GRUB2
How to run TensorFlow 1.0 code in 2.0
How to collect machine learning data
How to solve slide puzzles and 15 puzzles
Coursera Machine Learning Challenges in Python: ex6 (How to Adjust SVM Parameters)
It's really useful to add save () and load () methods to Target in Luigi
How to query BigQuery with Kubeflow Pipelines and save the result and notes
[Linux] How to subdivide files and folders
How to package and distribute Python scripts
Introduction to Machine Learning: How Models Work
scikit-learn How to use summary (machine learning)
How to install and use pandas_datareader [Python]
Introduction to Deep Learning ~ Convolution and Pooling ~
[TF] How to use Tensorboard from Keras
How to study deep learning G test
python: How to use locals () and globals ()
How to use tensorflow under docker environment
How to use Python zip and enumerate
How to use is and == in Python
How to install fabric and basic usage
How to write pydoc and multi-line comments
How to install the deep learning framework Tensorflow 1.0 in the Anaconda environment of Windows
How to use lists, tuples, dictionaries, and sets
[Tensorflowjs_converter] How to convert Tensorflow model to Tensorflow.js format
Conformity and recall-Understanding how to evaluate classification performance ①-
How to generate permutations in Python and C ++
How to create explanatory variables and objective functions
[Python] How to read data from CIFAR-10 and CIFAR-100
How to run CNN in 1 system notation in Tensorflow 2
How to convert SVG to PDF and PNG [Python]
How to switch between Linux and Mac shells
Introduction to Deep Learning ~ Localization and Loss Function ~
[Python] How to use hash function and tuple.
[AWS / Lambda] How to load Python external library
Tensorflow, Tensorflow After all, which one (How to read Tensorflow)
Data cleaning How to handle missing and outliers
How to write async and await in Vue.js
[TF] How to specify variables to update with Optimizer
Aggregate Spotify views ranking and save to Excel
How to install Cascade detector and how to use it
Learn how to inflate images from TensorFlow code
How to force build TensorFlow 2.3.0 for CUDA11 + cuDNN8
How to split machine learning training data into objective variables and others in Pandas
Steps to quickly create a deep learning environment on Mac with TensorFlow and OpenCV