Solution for ValueError in Keras imdb.load_data

Origin

I tried to read with the following code.

from keras.datasets import imdb

(X_train,y_train),(X_test,y_test) = imdb.load_data(num_words=10000)

Then, the following error occurs

ValueError                                Traceback (most recent call last)
<ipython-input-203-2fc6b409cd07> in <module>
      5 #np_load_old = np.load
      6 #np.load = lambda *a,**k: np_load_old(*a, allow_pickle=True, **k)
----> 7 (X_train,y_train),(X_test,y_test) = imdb.load_data(num_words=10000)
      8 #np.load = np_load_old
      9 
/opt/anaconda3/lib/python3.7/site-packages/keras/datasets/imdb.py in load_data(path, num_words, skip_top, maxlen, seed, start_char, oov_char, index_from, **kwargs)
     57                     file_hash='599dadb1135973df5b59232a0e9a887c')
     58     with np.load(path) as f:
---> 59         x_train, labels_train = f['x_train'], f['y_train']
     60         x_test, labels_test = f['x_test'], f['y_test']
     61 
/opt/anaconda3/lib/python3.7/site-packages/numpy/lib/npyio.py in __getitem__(self, key)
    260                 return format.read_array(bytes,
    261                                          allow_pickle=self.allow_pickle,
--> 262                                          pickle_kwargs=self.pickle_kwargs)
    263             else:
    264                 return self.zip.read(key)
/opt/anaconda3/lib/python3.7/site-packages/numpy/lib/format.py in read_array(fp, allow_pickle, pickle_kwargs)
    720         # The array contained Python objects. We need to unpickle the data.
    721         if not allow_pickle:
--> 722             raise ValueError("Object arrays cannot be loaded when "
    723                              "allow_pickle=False")
    724         if pickle_kwargs is None:
ValueError: Object arrays cannot be loaded when allow_pickle=False

Cause

It seems that the cause is that ʻallow_pickle = False` has become the default when upgrading the numpy version.

Solution

It can be handled by lowering the version of numpy, but it is troublesome, so try to solve it in the code.

Specifically, it is solved by replacing it with the following code.

#Specify up to 10000 words in frequency order
from keras.datasets import imdb
​
np_load_old = np.load
np.load = lambda *a,**k: np_load_old(*a, allow_pickle=True, **k)
(X_train,y_train),(X_test,y_test) = imdb.load_data(num_words=10000)
np.load = np_load_old

In np.load, it is temporarily overwritten so that ʻallow_pickle = True is specified. When ʻimdb.load_data is complete, restore the definition of np.load.

reference

Recommended Posts

Solution for ValueError in Keras imdb.load_data
Solution for run-time error in OpenVINO IEPlugin.load ()
Write DCGAN in Keras
Implemented hard-swish in Keras
For beginners, how to deal with common errors in keras
Implement LSTM AutoEncoder in Keras
Visualize Keras model in Python 3.5
Search for strings in Python
Search for strings in files
Techniques for sorting in Python
· Address already in use solution
About "for _ in range ():" in python
Tips for implementing a slightly difficult Model or Training in Keras