I tried to make AI for Smash Bros.

background

I like Smash Bros. I'd like to make something AI-like, but character detection seems to be troublesome for annotation, and conversely, it seems that it is not interesting because it can be done if you do your best with annotation. So, I would like to make a stage classifier for Smash Bros. (Because it is a part of play, details are appropriate)

Condition setting

Very hard sober. Number of classes: 103

One-shot learning to learn from one image because it is troublesome In other words, there are 103 learning images

conditions ・ End point ・ Characters are pudding and pudding ・ Overall view of training mode ・ If the background moves, a suitable moment

Was used for learning. There is no battlefield when it reaches the end point, and there is no DLC. Surprisingly, there were many stages, so it became a problem of multi-class one-shot classification.

The goal is simply to be able to classify the stage as it is.

Class list

\ ['3D Land', '75m','WiiFit Studio','Squid and Waterfall','Ancient Kingdom','Ancient Kingdom','Ancient Kingdom USA','Sumamura','Passing Legend' ',' A certain star','Tower of the beginning',' Umbra's clock tower',' Ish Pokemon League',' Windy Hill',' Woohoo Island',' Eight Cross Circuit',' Electroplankton',' Enger Land',' Onet','Oldin Ohashi',' Karos Pokemon League','Gaul Plains','Green Greens',' Green Hill Zone',' Great Bay','Geldo Valley',' Gamer',' Kotobuki Land','Congo Jungle','Shadow Moses Island','Jungle Garden','Skyloft','Super Happy Tree','Super Mario Maker','Tachiuo Parking',' Duck Hunt',' Tenganzan "Yoshi's Island", "Tomodachi Collection", "Training", "Dracula Castle", "Dolpic Town", "New Donk City Hall", "New Poke City", "Norfair", "Hyral Castle", "Balloon Fight" ,'Pilot Wings',' Pokeland',' Big Blue',' Picto Chat 2','Peach Castle','Peach Castle Sky',' Feria Arena',' Four Sides',' Flat Zone X',' Frigate Orpheon',' Brinsta',' Deep Brinsta','Puppland','Puppland GB','Prism Tower','Paper Mario',' Pokemon Stadium',' Pokemon Stadium 2',' Port Town Aerodive' ',' Magicant','Mario U World','Mario Galaxy','Mario Circuit','Mario Brothers','Midgar',' Mute City SFC','Made in Wario',' Yamabukishiti','Yoshi's Island' ,'Yoshi's Story','Yoshi's Island','Lyrat Cruise',' Luigi Mansion','Rainbow Cruise',' Wrecking Crew',' Wiley Base','Reopening Garden',' Initialization Bomb Forest', "Dream Fountain", "Sky Sea", "Living with Puppies", " Planet Cornelia','Planet Venom',' Battlefield',' Battleship Halvard','Siege Battle','Suzaku Miss',' Village and Town',' Train','Cave Daisakusen','Pirate Ship', 'Special Ring','Temple','End Station',' Field','Battlefield','Crest']

Stage name There are many mysterious guys ...

Strategy

The overall view of the stage looks like this.

いにしえっぽい王国.PNG

It's not that big in an actual battle, so I'll cut out this huge photo to make it that big. You can increase the number of photos by changing the cutting method.

Image cut out for learning

スクリーンショット 2020-04-20 17.20.26.png

When cut out ・ Cut out on several scales (three for the time being) ・ There is duplication ・ Basically suitable, but I was wondering if I could somehow learn the background texture. ・ Maybe 162 sheets are generated for each stage ・ When I cut it out, I thought I would make it for the time being and if it didn't work, I would fix it as appropriate.

The verification data looks like this on a real scale

いにしえっぽい王国.PNG It is verified with this photo. This image is reshaped to (224,224) and preprocessed before use. Not partial compared to train. Also, it is in a large range However, since it is troublesome, I have prepared only one image for verification.

Network structure

I extracted mobile-net-v2 from pytorch.torchvision and used it. Since the number of images has increased considerably with 103 class x 162 images, I made it a lightweight network. Only the output layer is changed to 103 class.

result

First, let's check how correct the answer is. For the time being, let's check the result after turning 10 epoch,

accuracy

train acc test acc test
96.9% 91.3% 94 / 103

It was surprisingly successful lol

Qualitative evaluation

For the time being, I tried to save one scene of my battle video. I just resized this image to (224,224) and networked it.

test.jpg

result スクリーンショット 2020-04-21 23.27.06.png

The 65th class magic cant ... correct answer!

Great ... lol

Let's put some more.

Since it's a big deal, I'll try to make an image without pudding. The second piece is Green Greens.

test2.jpg

スクリーンショット 2020-04-23 14.38.55.png

The third piece is the end point battlefield.

test3.jpg

スクリーンショット 2020-04-23 14.40.36.png

All were correct.

What went wrong

・ What I noticed at the time of annotation in the first place Mario Brothers マリオブラザーズ.PNG

Balloon fight バルーンファイト.PNG

75m (What is that stage name w) 75m.PNG

The table is different, but the texture is quite close, or if you make it low resolution, the background is almost the same lol I don't think they can make a mistake for a few minutes.

・ The rest is a series with a moving background. The background of the Umbra Clock Tower stage moves a lot, so it is often said that it feels uncomfortable during a match. The end point is also the basic stage, but the background moves quite violently. This time it's a one-shot study, so if the background changes, it may be completely different, so this is also unavoidable.

In that sense, I expected 70% to 90%, so the result was pretty good.

Correct answer and incorrect answer for each class
Stage name Correct answer?
3D Land
75m
Wii Fit Studio
Raft and waterfall
Ancient kingdom
Ancient Kingdom
Ancient Kingdom USA
Sumamura
Passing Legend
A certain star
Tower of the beginning
Umbra Clock Tower ×
Ish Pokemon League
Windy Hill
Uhu Island
Eight Cross Circuit
Electroplankton
Angelland
Onet
Oldin Ohashi
Kalos Pokemon League
Gaur Plain
Green Greens
Green Hill Zone
Great Bay
Valley of Gerd
Gamer
Kotobuki Land
Congo Jungle
Shadow Moses Island
Jungle Garden
Sky Loft
Super Happy Tree
Super Mario Maker ×
Hairtail parking
Duck Hunt
Tengan Zari no Hashira
Tomodachi Collection
Training
Dracula Castle
Dolpic Town ×
New Donk City Hall
New Polk City
Norfair
Hyrule Castle
Balloon Fight
Pilotwings ×
Pac-Land
Big blue
Picto Chat 2
Peach Castle
Over Peach Castle
Feria Arena
Forside
Flat zone X
Frigate Orpheon
Brinsta
Deep Brinsta
Dream Land
Dream Land GB
Prism Tower
Paper Mario
Pokemon Stadium
Pokemon Stadium 2
Port Town Aero Dive
Magicant
Mario U World
Mario Galaxy
Mario Circuit ×
Mario Bros.
Midgar
Mute City SFC
Made in Wario
Yamabukishiti
Yoshi's Island
Yoshi's Story ×
Yostar Island
Lyrat Cruise
Luigi's Mansion
Rainbow Cruise
Wrecking Crew
Wiley Base
Reopening Garden
Initialization Bomb Forest
Dream Fountain
Sky Sea
Living room with puppies
Planet Cornelia
Planet Venom ×
Battlefield
Battleship Halbird
Siege Battle
Miss Suzaku
Village and town
Train
Cave Operation
Pirate ship
Special ring
Temple
End point ×
Field ×
Arena
Summit

Impressions

The difficulty level is rather high at the points of classification of 100 classes or more. It was better than I expected, so I thought I made a mistake.

Well, the image that has nothing to do with test works well, so I wonder if there is any problem in operation. Considering that 103 classes are not suitable, it is good.

Data and code

code https://github.com/daikiclimate/sumaAI

data https://drive.google.com/drive/folders/1BUvnfLVzl34CTk3eE5lUc69PXvSx_4LT?usp=sharing

Well, there isn't much application, and basically there is no demand, so it's appropriate lol I did it if I could move.

Recommended Posts

I tried to make AI for Smash Bros.
I tried to make a face diagnosis AI for a female professional golfer ①
I tried to make a face diagnosis AI for a female professional golfer ②
I tried to make a Web API
I tried to make a strange quote for Jojo with LSTM
I tried to predict next year with AI
[First scraping] I tried to make a VIP character of Smash Bros. [Beautiful Soup] [Data analysis]
I tried to debug.
I tried to paste
I refactored "I tried to make Othello AI when programming beginners studied python"
I tried to make a ○ ✕ game using TensorFlow
[Python] I tried to make a Shiritori AI that enhances vocabulary through battles
Continuation ・ I tried to make Slackbot after studying Python3
[TensorFlow] I tried to introduce AI to rolling stock iron
I tried to make an OCR application with PySimpleGUI
AI Gaming I tried it for the first time
I tried to learn PredNet
I tried to organize SVM.
I tried to implement PCANet
I tried to reintroduce Linux
I tried to introduce Pylint
I tried to summarize SparseMatrix
I tried to touch jupyter
I tried to implement StarGAN (1)
I tried to make an air lip detection & automatic response BOT for remote work
I tried to create a reinforcement learning environment for Othello with Open AI gym
I tried to make creative art with AI! I programmed a novelty! (Paper: Creative Adversarial Network)
I tried porting the code written for TensorFlow to Theano
I tried to make various "dummy data" with Python faker
I tried to create a bot for PES event notification
I tried to make a stopwatch using tkinter in python
I tried to make GUI tic-tac-toe with Python and Tkinter
I tried to make a simple text editor using PyQt
I tried to make Othello AI that I learned 7.2 million hands by deep learning with Chainer
I tried to implement Deep VQE
I tried to create Quip API
I tried to touch Python (installation)
I tried to explain Pytorch dataset
I tried Watson Speech to Text
I tried to touch Tesla's API
I tried to implement hierarchical clustering
I tried to organize about MCMC.
I tried to implement Realness GAN
I tried to move the ball
I tried to estimate the interval.
[5th] I tried to make a certain authenticator-like tool with python
I tried to make an activity that collectively sets location information
I tried to make a system that fetches only deleted tweets
[2nd] I tried to make a certain authenticator-like tool with python
I tried to make deep learning scalable with Spark × Keras × Docker
I tried to make a regular expression of "amount" using Python
[Python] I tried to implement stable sorting, so make a note
I tried to make Kana's handwriting recognition Part 1/3 First from MNIST
I tried to make a regular expression of "date" using Python
[Pandas] I tried to analyze sales data with Python [For beginners]
I tried to make a periodical process with Selenium and Python
I tried to make a 2channel post notification application with Python
I tried to make an analysis base of 5 patterns in 3 years
I tried to make a todo application using bottle with python
[4th] I tried to make a certain authenticator-like tool with python
[Python] Simple Japanese ⇒ I tried to make an English translation tool