GradCAM with 22 lines of code. tf_explain may be easy to use, I recommend it!

Purpose

There seems to be ** tf_explain **. It seems that ** GradCAM ** is included as a module there. Looking at the github that uses it, ** GradCAM ** could be executed with only 22 lines (with blank lines). I think it's wonderful I tried to move it.

22 lines of code

You can find this code on github below. https://github.com/sicara/tf-explain/blob/master/examples/core/grad_cam.py

The code below is all, so

python grad_cam_py.py

It just works.

grad_cam.py


import tensorflow as tf

from tf_explain.core.grad_cam import GradCAM

IMAGE_PATH = "./cat.jpg "

if __name__ == "__main__":
    model = tf.keras.applications.vgg16.VGG16(weights="imagenet", include_top=True)

    img = tf.keras.preprocessing.image.load_img(IMAGE_PATH, target_size=(224, 224))
    img = tf.keras.preprocessing.image.img_to_array(img)

    model.summary()
    data = ([img], None)

    tabby_cat_class_index = 281
    explainer = GradCAM()
    # Compute GradCAM on VGG16
    grid = explainer.explain(
        data, model, class_index=tabby_cat_class_index, layer_name="block5_conv3"
    )
    explainer.save(grid, ".", "grad_cam.png ")

Execution result

The image of the cat ("tabby cat" in it) is taken from wikipedia. https://en.wikipedia.org/wiki/Tabby_cat This kind of cat seems to be characterized by having the letter M around the eyes in the middle of the face ~~ ** the letter M on the "forehead" *. ( "There is a letter M so as to surround the eyes in the middle of the face" (wrong), but it is not invisible to M ...)

sample1

cat_sample1.jpg grad_cam_sample1.png

sample2

cat_sample2.jpg grad_cam_sample2.png

Comments on the results

The result of GradCAM seems to be right. (However, The purpose of this article is that tf_explain is easy to use, so No comment on the good or bad of GradCAM itself. Just in case, ** If you put too much feeling on the GradCAM display, you will feel that the fortune-teller's fortune-telling will be a hit, so I think you should look at it as cold as possible, which is a complete digression. **)

environment

It's Windows 10. There is no GPU. tensorflow 2.0.0 tf-explain 0.2.1

The above tf-explain is probably not included, so install it using pip or other normal method. ** I think that 2.1.x is the latest version of tensorflow now (March 11, 2020), but I got a terrible error. ** ** ** I don't know how to cure it, I have a complete idea! So, when I set it to 2.0.0, it worked. ** **

This section is a section to actually see the nature of CNN rather than tf_explain (a digression from this article?)

cat2_2nd_224.jpggrad_cam_cat2_2nd_224.png

cat2_3rd_224.jpggrad_cam_cat2_3rd_224.png

cat2_4th_224.jpggrad_cam_cat2_4th_224.png

cat2_5th_224.jpggrad_cam_cat2_5th_224.png

cat2_6th_224.jpggrad_cam_cat2_6th_224.png

In the case of this example Is it the range of about 60x60 pixels that captures the features all at once? ⇒ In the last image, it feels like you are reacting individually to individual areas. Also, it may be more accurate to display the image and the point of interest even in the non-superimposed mode (is there?). (Because it is difficult to understand if it is affected by the brightness of the original image.) ⇒⇒ CNN will be vulnerable to "more images (up images)". ⇒⇒⇒ I'm impressed when a model that pursues the overall configuration and shows the feeling of grasping with GradCAM appears. .. .. (Whether it can be expressed by GradCAM is another story ... Apart from the fact that it can not be expressed by GradCAM, I think it would be great if such a model came out. Capsule system etc. If there is a model that pops in. )

Related to the above section (hence, aside). CNN is "weak to leaning (weak to zooming up)".

We've also added diagrams to help you better understand the focus of GradCAM. I also added the probability.

cat2_2nd_224.jpggrad_cam_cat2_2nd_224.pnggrad_cam_cat2_2nd_224ex.png

[[('n02123045', 'tabby', 0.40362296),
  ('n02124075', 'Egyptian_cat', 0.34350035),
  ('n02123159', 'tiger_cat', 0.1646882),
  ('n02747177', 'ashcan', 0.022324266),
  ('n02127052', 'lynx', 0.009675921),
  ('n03223299', 'doormat', 0.008641529),
  ('n02123394', 'Persian_cat', 0.00528028),
  ('n02909870', 'bucket', 0.0034843169),
  ('n04040759', 'radiator', 0.0028082374),
  ('n03958227', 'plastic_bag', 0.002630277),
  ('n04265275', 'space_heater', 0.002209679),
  ('n04493381', 'tub', 0.0015652123),
  ('n04049303', 'rain_barrel', 0.001464855),
  ('n04553703', 'washbasin', 0.0014180988),
  ('n04589890', 'window_screen', 0.0012623073),
  ('n03887697', 'paper_towel', 0.0012330494),
  ('n04522168', 'vase', 0.0012083148),
  ('n02123597', 'Siamese_cat', 0.0010707852),
  ('n03950228', 'pitcher', 0.0010204213),
  ('n03255030', 'dumbbell', 0.00096622825)]]

cat2_6th_224.jpggrad_cam_cat2_6th_224.pnggrad_cam_cat2_6th_224ex.png

[[('n03958227', 'plastic_bag', 0.23590706),
  ('n04209133', 'shower_cap', 0.117050014),
  ('n02124075', 'Egyptian_cat', 0.068308175),
  ('n01968897', 'chambered_nautilus', 0.052455623),
  ('n03825788', 'nipple', 0.042889122),
  ('n02123597', 'Siamese_cat', 0.040462725),
  ('n02120079', 'Arctic_fox', 0.02897999),
  ('n03868863', 'oxygen_mask', 0.018255476),
  ('n04370456', 'sweatshirt', 0.018049669),
  ('n02123045', 'tabby', 0.017420992),
  ('n04525038', 'velvet', 0.01728542),
  ('n02123394', 'Persian_cat', 0.0140852835),
  ('n03534580', 'hoopskirt', 0.012244948),
  ('n03724870', 'mask', 0.0106809465),
  ('n03045698', 'cloak', 0.007704126),
  ('n02120505', 'grey_fox', 0.0072637224),
  ('n02326432', 'hare', 0.006367313),
  ('n04127249', 'safety_pin', 0.006034479),
  ('n03887697', 'paper_towel', 0.0056772656),
  ('n04033995', 'quilt', 0.0056173983)]]

** tabby is about 10th. .. .. ** ** ⇒ It may be said that you should do it in various sizes, but I think it's better to be a little more conscious of the composition, CNN. ** I made a sample by misunderstanding the location of the letter M, which is a feature of tabby **, so the letter M, which is important, is outside the area of the image. (Actually, it's better to start over ... I think it's probably not good even if I try again, so I'll postpone it for a moment.)

I referred to the following to get the rate. https://medium.com/@gkadusumilli/image-recognition-using-pre-trained-xception-model-in-5-steps-96ac858f4206

Indicates the modified source.

import tensorflow as tf

import numpy as np ##
import pprint ##

from tf_explain.core.grad_cam import GradCAM

from tensorflow.keras.applications.vgg16 import decode_predictions ##

IMAGE_PATH = "./cat2_2nd_224.jpg "

if __name__ == "__main__":
    model = tf.keras.applications.vgg16.VGG16(weights="imagenet", include_top=True)

    img = tf.keras.preprocessing.image.load_img(IMAGE_PATH, target_size=(224, 224))
    img = tf.keras.preprocessing.image.img_to_array(img)
    
    predictions=model.predict(np.array([img])) ##
    pprint.pprint(decode_predictions(predictions,top=20))##

    model.summary()
    data = ([img], None)

    tabby_cat_class_index = 281
    explainer = GradCAM()
    # Compute GradCAM on VGG16
    grid = explainer.explain(
        data, model, class_index=tabby_cat_class_index, layer_name="block5_conv3"
    )
    explainer.save(grid, ".", "grad_cam_cat2_2nd_224.png ")
    
    grid = explainer.explain( ##
        data, model, class_index=tabby_cat_class_index, layer_name="block5_conv3", image_weight=0.01 ##
    ) ##
    explainer.save(grid, ".", "grad_cam_cat2_2nd_224ex.png ") ##

Summary

The environment is advancing every day (** tf_explain **). If you have any comments, please let us know.

Recommended Posts

GradCAM with 22 lines of code. tf_explain may be easy to use, I recommend it!
The sound of tic disorder at work is ... I managed to do it with the code
Comparison of GCP computing services [I want to use it serverless]
[AWS] [GCP] I tried to make cloud services easy to use with Python
Use Python from Java with Jython. I was also addicted to it.
I tried to get the authentication code of Qiita API with Python.
[Zaif] I tried to make it easy to trade virtual currencies with Python
How to use VS Code (code server) with Google Colab in just 3 lines
The tree.plot_tree of scikit-learn was very easy and convenient, so I tried to summarize how to use it easily.
I wanted to know the number of lines in multiple files, so I tried to get it with a command
I want to display only different lines of a text file with diff
I tried to use Java with Termux using Termux Arch but it didn't work
Make it easy to specify the time of AWS CloudWatch Events with CDK.
I tried to use lightGBM, xgboost with Boruta
I want to use MATLAB feval with python
I tried Flask with Remote-Containers of VS Code
I want to be an OREMO with setParam!
Story of trying to use tensorboard with pytorch
I want to use Temporary Directory with Python2
I don't want to use -inf with np.log
I want to use ip vrf with SONiC
I tried to make it easy to change the setting of authenticated Proxy on Jupyter
I don't like to be frustrated with the release of Pokemon Go, so I made a script to detect the release and tweet it
I tried to extract features with SIFT of OpenCV
I tried to summarize how to use matplotlib of python
Preparing to use Tensorflow (Anaconda) with Visual Studio Code
[Introduction to WordCloud] It's easy to use even with Jetson-nano ♬
Easy build of C ++ code with CMake on Docker
I tried to implement Grad-CAM with keras and tensorflow
I made a function to crop the image of python openCV, so please use it.
[Python] I wrote a test of "Streamlit" that makes it easy to create visualization applications.
It may be a problem to use Japanese for folder names and notebook names in Databricks