Hello. In my work, I'm away from engineering, and there aren't many things I can write about what I've done. Now that I have a little time, I would like to summarize what I learned a little. Advent calendar I'm not scared! You can write an article that you just did a tutorial! That's one ...

Assumed reader

I've heard a lot about machine learning, but I haven't touched on implementation so much, so I'll touch on it. So the intended readers are those who have never touched machine learning but are interested in it and want to take a look at the implementation of machine learning. Experienced people are warm-eyed and would appreciate it if you could point out the mistakes: smile_cat:

theme

If you don't have an easy-to-follow theme, you won't know the goal you should aim for and you'll get lost. So this time, when I give an image, I classify what is in the picture, Let's make an image classifier.

Anything is fine, but let's first decide what classifier to use. I've been addicted to curry lately, so I'll try to make a spice image classifier.

Our blog Thank you for your cooperation.

Prerequisite knowledge

Since image classifiers are a catchy theme, there are many tutorial articles, but rather than starting reading without any prerequisite knowledge, it is recommended that you grasp the whole thing with the following articles.

Introduction to Neural Networks from Zero without Mathematical Knowledge [Understanding the "Basics of Neural Networks" -Introduction to Deep Learning | Part 1](https://www.imagazine.co.jp/%E3%83%8B%E3%83%A5%E3%83 % BC% E3% 83% A9% E3% 83% AB% E3% 83% 8D% E3% 83% 83% E3% 83% 88% E3% 83% AF% E3% 83% BC% E3% 82% AF % E3% 82% 92% E7% 90% 86% E8% A7% A3% E3% 81% 99% E3% 82% 8B% E3% 80% 80% EF% BD% 9E% E3% 83% 87% E3 % 82% A3% E3% 83% BC% E3% 83% 97 /)

Also, please refer to the reference link for words that look like technical terms.

Do a tutorial

tensorflow (Tensorflow * 1) We will do a tutorial on transfer learning (* 2). Transfer learning with TensorFlow Hub.

I wrote it according to the item of the version of 2019/12, but if there is a change, I hope that you can read it in a nice way. The code for each section has been reprinted, but the results are not included except for some, so please read it while executing it yourself.

(* 1) tensorflow is one of the machine learning libraries, made by google. wiki (* 2) This is a method of diverting a model learned in a task different from transfer learning to another task. In this case, the accuracy will not improve unless a fairly large amount of data is prepared. It seems that it can be solved with a fairly small number of sheets. If you master one programming language, the second mastery will be quicker. Reference

Setup As a premise, this tutorial uses Google Colaboratory (hereinafter referred to as Colab * 3), so let's click "Run in Google Colab" at the top to enable it. When the Colab screen opens, let's run it in order from the top. Place the cursor around [] in the upper left of the code and you should see the execute button.

from __future__ import absolute_import, division, print_function, unicode_literals

import matplotlib.pylab as plt

import tensorflow as tf

You are importing various libraries around here. The description of __future__ .. is for maintaining compatibility of python2-3. ʻImport matplotlib.pylab as plt` is a library for plots for later use.

!pip install -q -U tf-hub-nightly
import tensorflow_hub as hub

from tensorflow.keras import layers

The description of ! pip install -q -U tf-hub-nightly is the installation of the python library. tensorflow_hub is a library for using a platform (tensorflow hub) that can easily use pre-trained models. tensorflow.keras is a machine learning library that seems to have been created by a google engineer but can be used by people other than tensorflow.

It's amazing how easy it is to run on a virtual machine just by writing it in text.

(* 3) Colaboratory is a Jupyter notebook environment that runs completely in the cloud. No settings are required and you can use it for free. With Colaboratory, you can write and execute code, store and share analyzes, access powerful computing resources, and more, all for free from your browser. From Welcome to Colaboratory

By the way, if you get such an error, you should follow the support and press the [RESTART RUNTIME] button and then the big play button.

An ImageNet classifier It is an item to try image classification using the trained model.

Download the classifier

classifier_url ="https://tfhub.dev/google/tf2-preview/mobilenet_v2/classification/2" #@param {type:"string"}

The URL of the trained classifier. As I wrote in the tutorial, it doesn't matter if it is the one placed here.

IMAGE_SHAPE = (224, 224)

classifier = tf.keras.Sequential([
    hub.KerasLayer(classifier_url, input_shape=IMAGE_SHAPE+(3,))
])

Poke the trained model as a layer of neural network. It seems that it is necessary to tell the format of the given image, so give it by (width, height, channel). Channel (number of colors) 3 will be added later. Does it have a deep meaning?

Run it on a single image When you're ready, let's make a distinction with a suitable image.

import numpy as np
import PIL.Image as Image

grace_hopper = tf.keras.utils.get_file('image.jpg','https://storage.googleapis.com/download.tensorflow.org/example_images/grace_hopper.jpg')
grace_hopper = Image.open(grace_hopper).resize(IMAGE_SHAPE)
grace_hopper

[grace_hopper](https://ja.wikipedia.org/wiki/%E3%82%B0%E3%83%AC%E3%83%BC%E3%82%B9%E3%83%BB%E3% 83% 9B% E3% 83% 83% E3% 83% 91% E3% 83% BC) images are being acquired.

grace_hopper = np.array(grace_hopper)/255.0
grace_hopper.shape

It seems that you are doing type conversion.

result = classifier.predict(grace_hopper[np.newaxis, ...])
result.shape

If you add one dimension to the image data and run the prediction, you will get an array of vectors of 1001 elements (to be exact, ndarray).

predicted_class = np.argmax(result[0], axis=-1)
predicted_class

This 1001 element represents the probability of each class (in this case, the inferred element), so let's take out the one with the highest probability. The value 653 is returned. This represents the thing that this classifier inferred.

Decodethe predictions

labels_path = tf.keras.utils.get_file('ImageNetLabels.txt','https://storage.googleapis.com/download.tensorflow.org/data/ImageNetLabels.txt')
imagenet_labels = np.array(open(labels_path).read().splitlines())

Download the labels and pack them into an array to find out what the previous results show. As you can see immediately by accessing the URL directly, it corresponds to the item 1001 mentioned earlier.

plt.imshow(grace_hopper)
plt.axis('off')
predicted_class_name = imagenet_labels[predicted_class]
_ = plt.title("Prediction: " + predicted_class_name.title())

Use the image display library to output what the 653rd image was. You got the result of Military Uniform. Congratulations.

Simple transfer learning From here is the production. The current one was "You can use a trained classifier like this", but Next is "I will make a classifier with the images I have collected".

Dataset In this tutorial, we will use a flower image. Finally, let's swap this dataset appropriately to make it a different classifier.

data_root = tf.keras.utils.get_file(
  'flower_photos','https://storage.googleapis.com/download.tensorflow.org/example_images/flower_photos.tgz',
   untar=True)

Dataset download

image_generator = tf.keras.preprocessing.image.ImageDataGenerator(rescale=1/255)
image_data = image_generator.flow_from_directory(str(data_root), target_size=IMAGE_SHAPE)

Dive into the ImageDataGenarator to handle the images well. By the way, you can see the classification of images by downloading the image group from the URL mentioned above, It feels like collecting specific images for each directory.

for image_batch, label_batch in image_data:
  print("Image batch shape: ", image_batch.shape)
  print("Label batch shape: ", label_batch.shape)
  break

Iterate image_data to create an array of image_batch and label_batch.

Run the classifier on a batch of images In order to compare with the state after learning, let's classify in the state without learning.

result_batch = classifier.predict(image_batch)
result_batch.shape

Let's make the same prediction as before.

predicted_class_names = imagenet_labels[np.argmax(result_batch, axis=-1)]
predicted_class_names

There are 32 classification results. Check against the image.

plt.figure(figsize=(10,9))
plt.subplots_adjust(hspace=0.5)
for n in range(30):
  plt.subplot(6,5,n+1)
  plt.imshow(image_batch[n])
  plt.title(predicted_class_names[n])
  plt.axis('off')
_ = plt.suptitle("ImageNet predictions")

Looking at the results, it's quite different. It may be because they are selected from 1001 objects, but they are not correct at all.

Download the headless model So let's make a classifier yourself.


feature_extractor_url = "https://tfhub.dev/google/tf2-preview/mobilenet_v2/feature_vector/2" #@param {type:"string"}

As usual, we will download the trained model from the tensorflow hub. The extractor looks like an extractor.


feature_extractor_layer = hub.KerasLayer(feature_extractor_url,
                                         input_shape=(224,224,3))

Create a layer to plunge into the model.


feature_batch = feature_extractor_layer(image_batch)
print(feature_batch.shape)

It returns 1280 vectors for each image. You try to classify images from various elements.


feature_extractor_layer.trainable = False

This time, the extractor part will not be tuned, so specify it so that it will not be learned.

Attach a classification head


model = tf.keras.Sequential([
  feature_extractor_layer,
  layers.Dense(image_data.num_classes, activation='softmax')
])

model.summary()

I'm pushing the instance of the extractor I made earlier into the classifier. You also see a summary of the model you made.


predictions = model(image_batch)

predictions.shape

I am checking the shape of the tensor.

Train the model


model.compile(
  optimizer=tf.keras.optimizers.Adam(),
  loss='categorical_crossentropy',
  metrics=['acc'])

Compile the model. optimizer: It seems that there are various types. Reference loss (loss function): A function used for training to adjust the loss to be as small as possible. This seems to be various. Reference metrics (evaluation function): A value used to evaluate the performance of a model, but not used for training, such as a loss function. This also seems to be various. Reference

class CollectBatchStats(tf.keras.callbacks.Callback):
  def __init__(self):
    self.batch_losses = []
    self.batch_acc = []

  def on_train_batch_end(self, batch, logs=None):
    self.batch_losses.append(logs['loss'])
    self.batch_acc.append(logs['acc'])
    self.model.reset_metrics()

It is a function definition for observing what the values of the loss function and the evaluation function are.

steps_per_epoch = np.ceil(image_data.samples/image_data.batch_size)

batch_stats_callback = CollectBatchStats()

history = model.fit_generator(image_data, epochs=2,
                              steps_per_epoch=steps_per_epoch,
                              callbacks = [batch_stats_callback])

Start learning with fit_generator. You specify the number of learnings and so on. Reference Learning will take some time.

plt.figure()
plt.ylabel("Loss")
plt.xlabel("Training Steps")
plt.ylim([0,2])
plt.plot(batch_stats_callback.batch_losses)

This is the transition of the loss function. It's good that it is gradually decreasing.

plt.figure()
plt.ylabel("Accuracy")
plt.xlabel("Training Steps")
plt.ylim([0,1])
plt.plot(batch_stats_callback.batch_acc)

It is the transition of the evaluation function. This is up so it feels good.

Check the predictions Let's finally classify.

class_names = sorted(image_data.class_indices.items(), key=lambda pair:pair[1])
class_names = np.array([key.title() for key, value in class_names])
class_names

Let's take a class that actually exists. There are only 5 types.

predicted_batch = model.predict(image_batch)
predicted_id = np.argmax(predicted_batch, axis=-1)
predicted_label_batch = class_names[predicted_id]

Predict as you did earlier, grab the maximum value and label it.

label_id = np.argmax(label_batch, axis=-1)

Reserve the correct label

plt.figure(figsize=(10,9))
plt.subplots_adjust(hspace=0.5)
for n in range(30):
  plt.subplot(6,5,n+1)
  plt.imshow(image_batch[n])
  color = "green" if predicted_id[n] == label_id[n] else "red"
  plt.title(predicted_label_batch[n].title(), color=color)
  plt.axis('off')
_ = plt.suptitle("Model predictions (green: correct, red: incorrect)")

If the answer is correct, it will be output in green, and if it is unsuccessful, it will be output in red. I think you're generally correct. Yosage.

After this, the tutorial continues for a while, and the export of the learned model is written, but since this is the end of use, I will omit it from now on.

Build your own classifier

Now, let's create your own classifier using the code that came out in the tutorial. This time, I will set a goal until it moves on Colab.

Image collection

Image collection is essential for image classification.

Image download

google-images-download is useful. (Be careful about the license of the downloaded image)

Since it can be installed via pip, I think that it can be used as follows on mac.

pip install google_images_download
googleimagesdownload --keywords "The image you want to find"

Image sorting

If you try to use the downloaded image as it is, there is a lot of garbage and you cannot use it. So, let's remove strange images while looking at the images one by one. This work was the hardest ..

In the end, let's create a state where specific clean images are collected for each directory like this.

Package and upload

For the time being, let's upload the tar-compressed file somewhere so that it can be downloaded from Colab. I uploaded it to Google Drive. In the case of Google Drive, upload tar.gz, get a shareable link, and replace the ID part with the link below. https://drive.google.com/uc?export=download&id=XXXXXXXXXXXXXXXXXXXXX

Reflected in Colab code

I have prepared a Colab file that extracts only the part that makes the classifier here, so please replace the xxxx part.

Did it work?

It looks like this at my hand. image (85).png

in conclusion

I was able to experience the upper side of machine learning somehow, but I feel that I want the photos I took to be classified if it is an image of a classifier. I want to create something that feels a little more casual by building an environment at hand. I will hang on.

First image classifier