A story about machine learning with Kyasuket

At PeerCast's creative event "Kyasuket", participants were asked to draw a large number of pictures, and based on that, an identification AI was created.

Kyasuket is like an online Comiket, a relative event where participants can publish their work and enjoy the work of others. The main contents are pictures, games and tools. It is held once every six months.

At this time (Autumn 2016), we asked them to draw pictures of multiple types of Sutatsu-san (the figure below, an example), and based on that data, we identified which type of Sutatsu-san. Created an AI to do.

UI We have prepared four pages: a page for writing pictures that will be data, a page for using AI, a page for information on how many posts have been posted, and a page for writing brief explanations.

Post page AI page I did it like this.

Randomly draw 9 kinds of pictures and ask them to imitate them. On the contrary, AI decided to answer with probability which picture it was close to.

I don't think I'd see it in the old IE of some company, so I wrote it in ES2015 and placed it without compiling with babel, huh.

server

I pushed it into heroku's free tier. I used Flask in Python3. If the original size of the image data is the same, the data size will be large, so I made the image 32x32 and put it in heroku's PostgreSQL along with information about which picture it is.

I chose Python to use TensorFlow for machine learning, and I didn't intend to complicate it as a web app, so I used Flask. I used Flask-SQLAlchemy to work with the database.

It would be very painful to have to draw a picture and refresh the page, so I took the method of straightening the canvas after sending the data with Ajax. So, the page itself is almost static, and the server is mainly operated by json API.

AI The AI challenges are at the same level as mnist, so I made it a level that can be understood with a little work on the TensorFlow tutorial. The reason I made it safe without pushing my limits is that I will make adjustments within the period.

The model used softmax regression, multi-layer perceptron, and CNN, as shown in the TensorFlow tutorial, so that the user could select the model and see the results of each.

result

Currently, more than 4,500 data are collected. (Of which, I wrote about 1000) Many people wrote along with me as I was distributing the state of writing data with PeerCast, and most of the posts were at that time. .. I think that the event was lively with a reaction like PeerCast, which has a culture of enjoying repetitive work like this kind of training. (It would have been painful to continue writing the entire schedule for three days, but it was helpful because I collected enough data only on the first day to get enough learning results.) It was easy to write with a pen tab, but it is compatible with smartphones. It wasn't done, so maybe some people wrote it on their smartphones.

On the contrary, as expected, there must have been a story post that hindered learning. I'd like to try to detect material posts, but that's a future story. For the time being, I exposed the material-like things on Twitter when I saw it with my own eyes. In the udon picture, there are people who write fried rice, girls who seem to be girls, and other characters.

The accuracy of AI is about 70% for softmax regression and multi-layer perceptron, and 86% for CNN. Compared to mnist, the total amount of data is small for the large amount of data, so I think it's about this. Unlike the tutorial, the accuracy did not come out unless I tampered with the dimensions and parameters by myself, and I had to make various trials and errors. I didn't do it on the delivery because I was wondering if the results would come out after learning at midnight, but I regret that some people would have expected it. However, it takes time to learn CNN ...

The handling of data and source code will be considered in the future. I'm tired of drawing.

reaction

I didn't have a function to display the posted image, but after all, there were some people who wanted to see it. I was worried about how to show it, but I got a plan to display one when I posted it.

Some people wanted to try it from the outside with a smartphone, but what should I do because I don't have a smartphone?

There is also a comment that it was a service where humans learned how to write Sutatsu rather than AI. In fact, I wrote one in a few seconds.

from now on

What to do with the data. What to do with the public. At the next time, I will tackle another problem from the same data. Organize to publish the source code.

Take a break and think if you can reset your mind.