[Part 2] Use Deep Learning to forecast the weather from weather images

Last time

Last time, we used meteorological images to determine the image using a CNN model and estimated the weather for the next day. Click here for the previous article Weather forecast from weather images using Deep Learning 1.

this time

This time, I would like to use more detailed weather for the results of the previous survey.

Detailed weather during the test period

In the analysis, the test period was daily from January to July 2016. This time, we estimated the binary classification of "fine" and "rain", but the weather is actually more detailed. Therefore, I would like to use the estimated "rain" and "fine" probabilities to compare with detailed weather.

Estimated probability and detailed weather

The average probabilities for each detailed weather during the estimated period are as follows.

weather Rain probability Clear probability The number of data
Cloudy 42% 58% 26
Fine 21% 79% 22
Sunny 22% 78% 19
Cloudy temporary rain 67% 33% 11
Sunny and cloudy 32% 68% 9
Partially cloudy 36% 64% 9
Cloudy temporary clear 34% 66% 8
Lightly cloudy after fine weather 24% 76% 7
Light cloud 33% 67% 7
Rain sometimes cloudy 53% 47% 6
Cloudy and sometimes rain 59% 41% 6
Sunny Temporary cloudy 33% 67% 5
Temporary clear after cloudy 43% 57% 5
Sometimes cloudy after rain 59% 41% 4
Cloudy after rain 67% 33% 4
Sometimes cloudy after fine weather 21% 79% 4
Rain after cloudy 65% 35% 4
After cloudy weather 21% 79% 4
Light cloudy temporary clear 47% 53% 4
Temporary rain after cloudy weather 48% 52% 3
Cloudy and sometimes sunny 44% 56% 3
Temporary cloudy rain 53% 47% 2
Temporarily cloudy after rain 61% 39% 2
Sunny after rain 67% 33% 2
Temporarily cloudy after fine weather 31% 69% 2
Cloudy after sunny 57% 43% 2
Temporary cloudy after heavy rain 74% 26% 2
Sometimes rain after cloudy 24% 76% 2
Cloudy and sometimes rainy 57% 43% 2
Cloudy and sometimes sunny and then temporary rain 20% 80% 2
Sometimes sunny after light cloudy 16% 84% 2
Light cloudy and sometimes sunny 30% 70% 2
rain 89% 11% 1
Sometimes cloudy after a temporary sleet of rain 24% 76% 1
Cloudy after rain, accompanied by hail 92% 8% 1
Rainy but sunny temporary drizzle 84% 16% 1
Rainy and sometimes cloudy 74% 26% 1
Sunny temporary rain 72% 28% 1
Temporary rain after fine weather, accompanied by lightning 76% 24% 1
After clear rain Sometimes cloudy 3% 97% 1
Sunny and sometimes cloudy 14% 86% 1
heavy rain 92% 8% 1
Sometimes sunny after heavy rain, accompanied by sleet 93% 7% 1
Cloudy and sunny after rain 20% 80% 1
Cloudy and sometimes sunny after a temporary rain 81% 19% 1
Cloudy and sunny after a temporary rain 20% 80% 1
Sometimes sunny after cloudy 89% 11% 1
Cloudy and sometimes sunny with temporary rain and thunder 79% 21% 1

This is difficult to understand, so let's look at the top of the "rain" probability and the top of the "fine" probability. First, the top 10 items with a "rain" probability are as follows.

weather Rain probability Clear probability The number of data
Sometimes sunny after heavy rain, accompanied by sleet 93% 7% 1
heavy rain 92% 8% 1
Cloudy after rain, accompanied by hail 92% 8% 1
rain 89% 11% 1
Sometimes sunny after cloudy 89% 11% 1
Rainy but sunny temporary drizzle 84% 16% 1
Cloudy and sometimes sunny after a temporary rain 81% 19% 1
Cloudy and sometimes sunny with temporary rain and thunder 79% 21% 1
Temporary rain after fine weather, accompanied by lightning 76% 24% 1
Temporary cloudy after heavy rain 74% 26% 2
Rainy and sometimes cloudy 74% 26% 1

Next, the top 10 items with a "fine" probability are as follows.

weather Rain probability Clear probability The number of data
After clear rain Sometimes cloudy 3% 97% 1
Sunny and sometimes cloudy 14% 86% 1
Sometimes sunny after light cloudy 16% 84% 2
Cloudy and sunny after a temporary rain 20% 80% 1
Cloudy and sunny after rain 20% 80% 1
Cloudy and sometimes sunny and then temporary rain 20% 80% 2
Fine 21% 79% 22
Sometimes cloudy after fine weather 21% 79% 4
After cloudy weather 21% 79% 4
Sunny 22% 78% 19
Sometimes cloudy after a temporary sleet of rain 24% 76% 1

Isn't it estimated to be pretty good? Especially for rain, there is a relatively high probability of "heavy rain". On the other hand, when it comes to fine weather, there are many things like "temporary rain." If "fine", "cloudy", and "rain" are all included, the estimation seems to be blurred. This is an issue.

next time

Next time, I would like to improve the teacher data and update the model.

Postscript: Regarding the weather forecast

By law, weather forecasts must be accompanied by a weather forecaster. It is described in detail in Konohen. Of course, I'm not a weather forecaster, so I can't generally deliver something like "tomorrow's weather." Also, forecasting according to software or model is basically not allowed.

To quote

** We have created software that automatically calculates the weather and temperature from the numerical weather prediction materials of the Japan Meteorological Agency. Do I need a forecasting license to make forecasts using this software? ** **

Forecasting business permission is required to perform forecasting business regardless of the forecasting method. However, since the forecast of the phenomenon must be done by the weather forecaster, if the forecast is made only by software without the intervention of the weather forecaster, the forecasting work cannot be permitted.

There is.

The "phenomenon" is defined as follows.

Forecast is defined by the Meteorological Service Act as "announcement of forecasts of phenomena based on observation results". Specifically, by specifying the "time" and "place", the situation of natural phenomena that will occur in the future is predicted by a natural science method based on the results of observation, and the result is predicted by the user (third party). It means to provide to. Business means "acts that are repeated and continuously performed".

Offering to third parties also includes publishing on the Web.

Therefore, I would like you to consider this analysis as a personal study.

Recommended Posts

[Part 1] Use Deep Learning to forecast the weather from weather images
[Part 3] Use Deep Learning to forecast the weather from weather images
[Part 2] Use Deep Learning to forecast the weather from weather images
Deep Learning from the mathematical basics Part 2 (during attendance)
I tried to implement Perceptron Part 1 [Deep Learning from scratch]
Reinforcement learning to learn from zero to deep
Image alignment: from SIFT to deep learning
"Deep Learning from scratch" Self-study memo (Part 12) Deep learning
Get data from Poloniex, a cryptocurrency exchange, via API and use deep learning to forecast prices for the next day.
About the order of learning programming languages (from beginner to intermediate) Part 2
Deep Learning beginners tried weather forecasting from meteorological satellite images using Keras
Tweet the weather forecast with a bot Part 2
Deep Learning from scratch ① Chapter 6 "Techniques related to learning"
POST images from ESP32-CAM (MicroPython) to the server
Stock Price Forecast Using Deep Learning (TensorFlow) -Part 2-
Create a dataset of images to use for learning
[Deep Learning from scratch] I implemented the Affine layer
I wanted to use the Python library from MATLAB
Othello ~ From the tic-tac-toe of "Implementation Deep Learning" (4) [End]
[Deep Learning from scratch] I tried to explain Dropout
Deep Learning from scratch 1-3 chapters
How to use the generator
[Deep learning] Investigating how to use each function of the convolutional neural network [DW day 3]
[Deep Learning from scratch] I tried to explain the gradient confirmation in an easy-to-understand manner.
How to increase the number of machine learning dataset images
(Deep learning) Images were collected from the Flickr API and discriminated by transfer learning with VGG16.
Introduction to Deep Learning ~ Learning Rules ~
I captured the Touhou Project with Deep Learning ... I wanted to.
Deep Reinforcement Learning 1 Introduction to Reinforcement Learning
"Deep Learning from scratch" Self-study memo (Part 8) I drew the graph in Chapter 6 with matplotlib
How to use the decorator
Introduction to Deep Learning ~ Backpropagation ~
[Deep Learning from scratch] About the layers required to implement backpropagation processing in a neural network
Chapter 1 Introduction to Python Cut out only the good points of deep learning made from scratch
[Machine learning] Understand from mathematics why the correlation coefficient ranges from -1 to 1.
Lua version Deep Learning from scratch Part 6 [Neural network inference processing]
How to use machine learning for work? 01_ Understand the purpose of machine learning
[Python + heroku] From the state without Python to displaying something on heroku (Part 1)
[Python + heroku] From the state without Python to displaying something on heroku (Part 2)
Deep Learning / Deep Learning from Zero 2 Chapter 4 Memo
How to use the zip function
Deep Learning / Deep Learning from Zero Chapter 3 Memo
How to use SWIG from waf
Deep Learning / Deep Learning from Zero 2 Chapter 5 Memo
Introduction to Deep Learning ~ Function Approximation ~
Try deep learning with TensorFlow Part 2
Deep learning from scratch (cost calculation)
Deep learning to start without GPU
Introduction to Deep Learning ~ Coding Preparation ~
Post images from Python to Tumblr
Use the Flickr API from Python
Deep Learning / Deep Learning from Zero 2 Chapter 7 Memo
Deep Learning / Deep Learning from Zero 2 Chapter 8 Memo
Deep Learning / Deep Learning from Zero Chapter 5 Memo
Deep Learning / Deep Learning from Zero Chapter 4 Memo
Deep Learning / Deep Learning from Zero 2 Chapter 3 Memo
Deep Learning memos made from scratch
Introduction to Deep Learning ~ Dropout Edition ~
Introduction to Deep Learning ~ Forward Propagation ~
Introduction to Deep Learning ~ CNN Experiment ~
How to use the ConfigParser module