Introduction

Moving MNIST is a test set for evaluating sequence prediction and reconstruction. A collection of images created so that handwritten numbers move around the screen.

The dataset is on the left and the forecast is on the right. Quoted from this site.

There are 10,000 types of data, each with 20 frames. The size of the image is 64 x 64 and it shows two numbers.

Data reading

You can download it from this site.

!curl -o mnist_test_seq.npy http://www.cs.toronto.edu/~nitish/unsupervised_video/mnist_test_seq.npy

Read the data.

import numpy as np

path="./mnist_test_seq.npy"
data = np.load(path)

print(data.shape)  # (20, 10000, 64, 64)

Use ipywidgets to make sure it's a video.

%matplotlib inline
import matplotlib.pyplot as plt
from ipywidgets import interact

def f(k):
    plt.imshow(data[k][0], 'gray')
    plt.show()

interact(f, k=(0,19,1) )

タイトルなし.gif

There are 10,000 such sets.

How to make a Dataset when predicting 10 frames using 10 frames.

import numpy as np
from torch.utils.data import Dataset


class MovingMnistDataset(Dataset):
    def __init__(self, path="./mnist_test_seq.npy"):
        self.data = np.load(path)
        # (t, N, H, W) -> (N, t, C, H, W)
        self.data = self.data.transpose(1, 0, 2, 3)[:, :, None, ...]

    def __len__(self):
        return len(self.data)

    def __getitem__(self, i):
        return self.data[i, :10, ...].astype(np.int32), self.data[i, 10:, ...].astype(np.int32)

dataset = MovingMnistDataset()

Data set "Moving MNIST"

Introduction

Data reading