For beginners, how to deal with common errors in keras

Keras (& tensorflow) often used for machine learning with python.

I didn't have an article summarizing how to deal with errors when using keras, so I'll summarize it based on the solved questions of teratail.

ImportError,AttributeError -Resolved by lowering or raising the version (ImportError, AttributeError) -Sometimes it can be solved by reinstalling (Import Error) -The file to be imported is incorrect (Attribute Error) ・ Tensorflow.keras and keras are different ・ Typo (case sensitive)

In keras, the influence of various library versions is large. If it is too old or too new, it may cause an Inport Error.

For AttributeError, refer to the following article.

[python] 5 things to do when "Attribute Error: module (object) ‘xxx’ has no attribute ‘yyy’ ”

ValueError

Common terms

-Check the model summary model.summary () ・ Check the shape of the data with print (data.shape)

Most of the ValueErrors are that the dimensions of the data are different from the model's expectations. Check the shape of the data, and if it is different, use reshape () to shape the data.

Numpy.reshape documentation

Shapes A and B are incompatible ・ Whether the dimensions of the model output and the output data match

For example, when the output of the model is 3 (Dense (3), etc.) even though it should be a binary classification. You need to analyze the model with model.summary ().

expected ndim=A, found ndim=B -Since the input of Dense is a basic one-dimensional array, reshape or Flatten to format it into one dimension. ・ Or you need to change the input_shape -Input_shape does not include batch size -Reshape the image data so that it becomes (number of samples, height, width, channel) -For LSTM, it is necessary to set [Batch number, time axis, number of channels]

expected layer_name to have shape A dimensions but got array with shape B ・ Isn't RGB and black and white mistaken (in the case of images)? ・ Whether the dimensions of the input data and the model input match -Reshape to a tensor with 1 x number of elements (occurs when there is only one test data)

Input arrays should have the same number of samples as target arrays. Found A input samples and B target samples

・ Is the number of input data and output data correct?

ResourceExhaustedError ・ OOM (Out Of Memory, Out of Memory, Resource Exhausted Error) ・ Reduce batch size ・ Restart your PC

An error that occurs when the memory of the GPU or PC is insufficient. A common method is to increase memory or reduce processing.

Is the program stored in memory for some things? Sometimes it happens, so resetting it with a reboot may work (for example, if it worked).

When learning accuracy is not good

・ Pre-process data ・ There is a problem with the input data itself ・ Is the forecast data correct? ・ Change hyperparameters ・ There are some things that cannot be learned even with machine learning

If it is image data, is it normalized ([0,255] → [-1,1]), is the input data or prediction data damaged, etc.

Anyway, you need to make sure that the data is accurate. If the data is not accurate, neither learning nor prediction is of course accurate.

Also, if the types of data are insufficient, the learning accuracy will not increase. If the weather forecast can be made only by the temperature, the forecaster will not have any trouble.

And there are cases where machine learning cannot be used for learning. For example, the dice rolls are random and cannot be predicted using machine learning. There are some directions.

Summary

This is how to deal with the error.

I'm not that professional myself, so there may be mistakes in the content. In that case, please point out in the comments.

Recommended Posts

For beginners, how to deal with common errors in keras
How to deal with run-time errors in subprocess.call
How to deal with DistributionNotFound errors
How to deal with enum compatibility errors
[Python] How to deal with module errors
How to deal with memory leaks in matplotlib.pyplot
How to deal with errors when hitting pip ②
[For beginners] How to use say command in python!
How to deal with pyenv initialization failure in fish 3.1.0
How to deal with Executing transaction: failed in Anaconda
[TensorFlow 2 / Keras] How to run learning with CTC Loss in Keras
How to deal with imbalanced data
[AWS] How to deal with "Invalid codepoint" error in CloudSearch
[For beginners] How to implement O'reilly sample code in Google Colab
How to deal with python installation error in pyenv (BUILD FAILED)
How to deal with errors when installing whitenoise and deploying to Heroku
How to deal with errors when installing Python and pip with choco
How to work with BigQuery in Python
10 Python errors that are common to beginners
How to deal with garbled characters in json of Django REST Framework
How to deal with old Python versions in Cloud9 made by others
How to reduce GPU memory usage with Keras
[REAPER] How to play with Reascript in Python
How to make Spigot plugin (for Java beginners)
I tried to integrate with Keras in TFv1.1
can't pickle annoy. How to deal with Annoy objects
How to deal with module'tensorflow' has no attribute'〇〇'
How to deal with SessionNotCreatedException when using Selenium
Try to calculate RPN in Python (for beginners)
How to use tkinter with python in pyenv
How to make Python faster for beginners [numpy]
[For beginners] How to study programming Private memo
[Introduction for beginners] Working with MySQL in Python
[For beginners] Introduction to vectorization in machine learning
[EC2] How to deal with errors that selenium cannot execute (No module named selenium)
How to deal with "No module named'〇〇'" error in Jupyter Notebook | Install with! Pip!
How to convert / restore a string with [] in python
How to convert Python # type for Python super beginners: str
[For beginners] How to study Python3 data analysis exam
How to do hash calculation with salt in Python
Explain in detail how to make sounds with python
How to run python in virtual space (for MacOS)
How to deal with Django's Template Does Not Exist
How to do zero-padding in one line with OpenCV
How to run tests in bulk with Python unittest
[Python] How to deal with pandas read_html read error
How to resolve SSL module errors in Anaconda environment
Python # How to check type and type for super beginners
How to support x-nullable for swagger in python's jsonschema library
How to output a document in pdf format with Sphinx
[For beginners] How to read Numerai's HP + Submit + Convenient links
How to extract any appointment in Google Calendar with Python
[For beginners] Summary of standard input in Python (with explanation)
[For beginners] How to use for statements on Linux (variables, etc.)
[Explanation for beginners] Introduction to convolution processing (explained in TensorFlow)
How to implement 100 data science knocks for data science beginners (for windows10 Home)
How to define multiple variables in a python for statement
[Explanation for beginners] Introduction to pooling processing (explained in TensorFlow)
How to update with SQLAlchemy?
[Django] How to give input values in advance with ModelForm
How to learn TensorFlow for liberal arts and Python beginners