[Pokemon Sword Shield] I tried to visualize the judgment basis of deep learning using the three family classification as an example.
Introduction
Do you guys play Pokemon? I bought it for the first time in 10 years ~~ I got it from Santa. Aiming to be a strong force, we plan to stay home and carefully select during the year-end and New Year holidays. I was wondering if Advent Calendar could do something with Pokemon material, so I tried ** a method to show the judgment basis of the deep learning model ** that I am interested in recently ** using Pokemon Three Family Classification as an example ** Saw.
- Since the explanation of the method and the preparation of the data set will be described, if you want to know only the result, please skip to Result.
Method to show the judgment basis of deep learning model: What is TCAV?
Deep learning is beginning to be implemented in society in various fields, but it tends to be a black box on what the model makes decisions. In recent years, research on the "explanatory" and "interpretability" of models has been underway.
Therefore, this time, I would like to try the method Quantitative Testing with Concept Activation Vectors (TCAV) adopted in ICML 2018.
Paper summary
-** Method to show the judgment basis of the neural network model ** --Indicates the importance of the ** concept ** (color, gender, race, etc.) of the prediction class, rather than the traditional method of calculating importance for each ** pixel. --Since it generates ** explanations for each class ** (≒ global) instead of explanations for each image (≒ local), it has easy-to-understand explanations for humans. -** You can understand the explanation without specialized knowledge of ML model ** --There is no need to retrain or change the existing model you want to interpret **
Concept Activation Vectors (CAV) concept
Derivation of CAV by training a linear classifier between a conceptual image and a random counterexample and obtaining a vector orthogonal to the decision boundary. (It is faster to see the figure below).
- More detailed paper memos can be found at here, so please have a look if you are interested.
What do you know
--The ** "concept" ** that the model is learning can be quantified in a form that can be interpreted by humans.
Example: Learning "striped pattern" from "dot pattern" in "zebra" classification
You can also see learning in any layer, so you can see how coarse / detailed features are captured in the shallow / deep layers.
--Understand the ** bias ** of the dataset
Example: In the "Apron" class, the concept of "female" is related, in the "Rugby ball" class, the concept of "white" is related
--Can be used as an image sorter (can be sorted based on similarity to conceptual images)
First make a suitable classifier
This time, my goal is to move TCAV, so I made it a simple task. Make a Pokemon Three Family Classifier.
Data set preparation
① Crawling
I collected the following images using icrawler. I will put the code.
import os
from icrawler.builtin import GoogleImageCrawler
save_dir = '../datasets/hibany'
os.makedirs(save_dir, exist_ok=True)
query = 'Scorbunny'
max_num = 200
google_crawler = GoogleImageCrawler(storage={'root_dir': save_dir})
google_crawler.crawl(keyword=query, max_num=max_num)
② Pretreatment
Only minimal processing.
- ① Manually crop the image acquired by crawling into a square
- Resize to 256 x 256
- Divide into train / val / test
Misanke image sample
The images were collected like this. (By the way, I was a quick decision for Hibani. I love flame type)
| Scorbunny | Messon | Sarnori |
|---|---|---|
 |
 |
 |
| 156 sheets | 147 sheets | 182 sheets |
The following Pokemon other than the three families, character images, and over-deformed illustrations were also confused, so they are excluded by visual inspection.
~~ Kibana San Cool ~~
Create classifier
It's a simple CNN.
Since there are few images of test data (about 15 images), the accuracy of test data is fluttering, but we have created a classification model with accuracy that will be sufficient for TCAV verification.
You will need a **. Pb file ** to calculate the CAV, so save the model in .pb. Next, get ready to see what the model is learning.
Preparing to execute TCAV
Follow the steps below to prepare. (The code used this time is in here. I will write the README properly later ...)
Step1: Preparation of conceptual images (positive and negative examples)
The following image is prepared for the regular image. We prepared several colors based on the hypothesis that we would classify the three families by looking at the colors. (Although it works with 10 to 20 sheets, it is better to have about 50 to 200 sheets)
** Regular image sample **
| White | Red | Blue | yellow | Green | black |
|---|---|---|---|---|---|
 |
 |
 |
 |
 |
 |
| 22 sheets | 20 sheets | 15 sheets | 18 sheets | 21 sheets | 17 sheets |
I'm excluding those that have too many colors mixed together.
** Negative example image sample ** Anything that does not fit into any of the above examples is desirable. (In this case, it is difficult to say that it does not correspond to any color.) This time, I randomly took images from Caltech256.
The directory structure of the images collected so far is as follows. All sets of conceptual images should be subdirectories.
├── datasets
│ ├── for_tcav #Data set for TCAV
│ │ ├── black
│ │ ├── blue
│ │ ├── green
│ │ ├── hibany
│ │ ├── messon
│ │ ├── random500_0
│ │ ├── random500_1
│ │ ├── random500_2
│ │ ├── random500_3
│ │ ├── random500_4
│ │ ├── random500_5
│ │ ├── red
│ │ ├── sarunori
│ │ ├── white
│ │ └── yellow
│ └── splited #Data set for image classification model creation
│ ├── test
│ │ ├── hibany
│ │ ├── messon
│ │ └── sarunori
│ ├── train
│ │ ├── hibany
│ │ ├── messon
│ │ └── sarunori
│ └── validation
│ ├── hibany
│ ├── messon
│ └── sarunori
Step2: Implement the model wrapper
I will clone it first.
git clone [email protected]:tensorflow/tcav.git
Here, we will create a wrapper to convey model information to TCAV. Add this class to tcav / model.py.
class SimepleCNNWrapper_public(PublicImageModelWrapper):
def __init__(self, sess, model_saved_path, labels_path):
self.image_value_range = (0, 1)
image_shape_v3 = [256, 256, 3]
endpoints_v3 = dict(
input='conv2d_1_input:0',
logit='activation_6/Softmax:0',
prediction='activation_6/Softmax:0',
pre_avgpool='max_pooling2d_3/MaxPool:0',
logit_weight='activation_6/Softmax:0',
logit_bias='dense_1/bias:0',
)
self.sess = sess
super(SimepleCNNWrapper_public, self).__init__(sess,
model_saved_path,
labels_path,
image_shape_v3,
endpoints_v3,
scope='import')
self.model_name = 'SimepleCNNWrapper_public'
Now you are ready to go. Let's see the result immediately.
result
Let's take a look at the concepts (colors this time) that are important in each class. Concepts not marked with * are important.
| Hibani class | Messon class | Sarnori class |
|---|---|---|
 Red / yellow / white |
 Red(!?) |
 Green |
|
|
|
I think Hibani and Sarnori are like that. The messon is a mystery, so it is important to consider it. If you change the number of trials or the number of conceptual images / target images during the experiment, the results will change considerably, so I think it is necessary to consider a little more. It seems to be worth trying various things because it seems to change depending on how you choose the conceptual image.
Summary
I tried a method to show the judgment basis of the neural network model. It was easy for humans to interpret, and ** "intuitively like that" ** results were obtained. This time, I chose the color as the conceptual image because it is classified as a three-family family, but it is difficult to prepare the conceptual image. .. Various preparations are required, but there is no need to relearn the model, and if you try the series of steps once and get used to it, you can use it easily. Please try by all means try!
Recommended Posts