[Part 3] Use Deep Learning to forecast the weather from weather images
Until last time
Until the last time
Weather forecast from weather images using Deep Learning 1 Weather forecast from weather images using Deep Learning 2
In summary,
- Image processing is performed from the weather image (satellite image), and the weather of the next day is predicted using the convolutional neural network.
- Binary values of "fine" and "rain" were used for the teacher data.
- The image was an RGB image near Japan, and evaluation was performed using 2015 as learning data and 2016 as test data.
- As a result, the average AUC is 0.70 and the F Score is 0.72. Looking at the actual probability and detailed weather, although clear things such as heavy rain and fine weather are correct, the accuracy in "cloudy" and "temporary rain" is It was not good.
- The details of the statistics were as follows.
| weather | Precision | Recall | F-Score |
|---|---|---|---|
| rain | 0.54 | 0.67 | 0.6 |
| Fine | 0.83 | 0.74 | 0.78 |
| average | 0.74 | 0.72 | 0.72 |
This story
This time, I would like to improve the accuracy by improving the model, teacher data, and explanatory variables.
In particular
- Trivalent (fine, cloudy, rainy) classification
- Increase in Epoch
- Addition of explanatory variable (image) processing
- Try changing the image
It will be around.
analysis
Trivalent classification
Until now, it was a binary classification of "fine" and "rain". However, this contains a large amount of "cloudiness", which was one of the factors that reduced accuracy. Therefore, let's simply classify it as a ternary including "cloudy".
The conditions for teacher data generation are as follows.
flag = []
for tk in tenki_list:
if "rain" in tk:
flag.append(0)
elif "Fine" in tk:
flag.append(2)
else:
flag.append(1)
Rain is the highest priority, followed by fine weather and the last remaining cloudy weather.
The test result is as follows. Each probability is the average probability in the weather. Sorted in descending order of rainfall probability.
| weather | Rain probability | Cloudy probability | Clear probability |
|---|---|---|---|
| Sometimes sunny after heavy rain, accompanied by sleet | 98% | 0% | 2% |
| Cloudy after rain, accompanied by hail | 97% | 0% | 3% |
| rain | 96% | 1% | 3% |
| heavy rain | 95% | 1% | 4% |
| Sunny temporary rain | 95% | 0% | 5% |
| Cloudy and sometimes sunny after a temporary rain | 91% | 1% | 8% |
| Sometimes sunny after cloudy | 90% | 3% | 6% |
| Rainy but sunny temporary drizzle | 89% | 2% | 10% |
| Temporary cloudy after heavy rain | 80% | 3% | 17% |
| Cloudy and sometimes rainy | 77% | 4% | 19% |
| Cloudy and sometimes sunny with temporary rain and thunder | 74% | 5% | 21% |
| Temporarily cloudy after rain | 72% | 6% | 22% |
| Sunny after rain | 72% | 4% | 24% |
| Cloudy temporary rain | 71% | 5% | 24% |
| Cloudy after rain | 71% | 3% | 26% |
| Cloudy and sometimes rain | 70% | 5% | 25% |
| Sometimes cloudy after rain | 67% | 3% | 29% |
| Temporary cloudy rain | 63% | 7% | 30% |
| Cloudy after sunny | 63% | 4% | 33% |
| Rainy and sometimes cloudy | 62% | 8% | 30% |
| Rain sometimes cloudy | 61% | 4% | 35% |
| Sunny and sometimes cloudy | 60% | 3% | 37% |
| Cloudy and sometimes sunny | 59% | 5% | 36% |
| Rain after cloudy | 57% | 7% | 36% |
| Temporary rain after fine weather, accompanied by lightning | 52% | 11% | 36% |
| Temporary clear after cloudy | 51% | 6% | 44% |
| Sometimes cloudy after a temporary sleet of rain | 50% | 7% | 44% |
| Temporary rain after cloudy weather | 49% | 10% | 41% |
| Cloudy | 45% | 8% | 47% |
| Light cloud | 42% | 7% | 52% |
| Light cloudy temporary clear | 38% | 10% | 52% |
| Sunny Temporary cloudy | 38% | 8% | 55% |
| Sometimes sunny after light cloudy | 38% | 11% | 52% |
| Temporarily cloudy after fine weather | 34% | 9% | 57% |
| Light cloudy and sometimes sunny | 34% | 8% | 58% |
| Cloudy temporary clear | 33% | 7% | 60% |
| Partially cloudy | 33% | 9% | 58% |
| Cloudy and sunny after rain | 31% | 13% | 56% |
| Fine | 30% | 9% | 61% |
| Sunny and cloudy | 30% | 8% | 62% |
| Lightly cloudy after fine weather | 28% | 11% | 61% |
| Sometimes cloudy after fine weather | 27% | 8% | 65% |
| Sunny | 25% | 7% | 68% |
| After cloudy weather | 22% | 10% | 68% |
| Sometimes rain after cloudy | 20% | 8% | 72% |
| Cloudy and sunny after a temporary rain | 20% | 11% | 70% |
| Cloudy and sometimes sunny and then temporary rain | 18% | 10% | 73% |
| After clear rain Sometimes cloudy | 4% | 5% | 91% |
As a result, cloudiness is completely unpredictable. (seriously)
Since the teacher data in the above conditional expression is as follows, I think that it makes it very difficult to estimate cloudiness.
| weather | Percentage |
|---|---|
| rain | 33% |
| Cloudy | 7% |
| Fine | 59% |
With this, it feels like playing only the obvious "cloudy". However, for example, if you bring in the condition that "cloudiness is included" first, the number of rain and fine weather will decrease too much, and it will be useless. Hmmm difficult.
However, the average AUC calculated only for "rain" and "fine" is 0.720, and the accuracy of these two seems to be improving. The rain and clear statistics are as follows.
| weather | precision | recall | f1-score |
|---|---|---|---|
| rain | 0.60 | 0.76 | 0.67 |
| Fine | 0.69 | 0.71 | 0.70 |
| average | 0.65 | 0.74 | 0.69 |
Apparently, the estimation accuracy of "rain" has improved.
Increase in Epoch
Last time I did it with Epoch = 50, but this time I will set Epoch to 100. The teacher data is the same as the last time, and is a binary classification of "rain" and "fine".
The average AUC was 0.724, which is more accurate than the previous and ternary classification. (Epoch important)
The statistics are as follows.
| weather | precision | recall | f1-score |
|---|---|---|---|
| rain | 0.53 | 0.77 | 0.63 |
| Fine | 0.86 | 0.68 | 0.76 |
| average | 0.76 | 0.71 | 0.72 |
It feels like Precision is up.
Addition of explanatory variable (image) processing
Up to this point, the explanatory variables have been used as the image data of the previous day for 3 channels. However, the original map image data is information that is common at any time and has no meaning in classification. Therefore, we will consider incorporating "changed" information by taking the difference from the previous day.
Written in code, it looks like this:
"""Convert to difference series"""
img_mat_new = np.zeros((img_mat.shape[0],3*2,img_mat.shape[2],img_mat.shape[3]),dtype=np.float32)
img_mat_new = img_mat_new[0:-1] #Reduce by one
for l in range(1,len(img_mat)):
"""Difference"""
img_mat_new[l-1,0,:,:] = img_mat[l-1,0,:,:] - img_mat[l,0,:,:]
img_mat_new[l-1,1,:,:] = img_mat[l-1,1,:,:] - img_mat[l,1,:,:]
img_mat_new[l-1,2,:,:] = img_mat[l-1,2,:,:] - img_mat[l,2,:,:]
"""that day"""
img_mat_new[l-1,3,:,:] = img_mat[l,0,:,:]
img_mat_new[l-1,4,:,:] = img_mat[l,1,:,:]
img_mat_new[l-1,5,:,:] = img_mat[l,2,:,:]
Don't forget to reduce teacher data by one day.
In this state, set the input channel to 6 and introduce it into the model.
Then the average AUC drops to 0.658. (That's it)
The statistics are as follows.
| weather | precision | recall | f1-score |
|---|---|---|---|
| rain | 0.57 | 0.48 | 0.52 |
| Fine | 0.78 | 0.83 | 0.8 |
| average | 0.71 | 0.72 | 0.71 |
The accuracy of the fine weather has improved, but the rain is no good. Well difficult.
Try changing the image data
What if I change the image data to another type? Until now, the resolution was 640x480, but let's change it to a high resolution image of 3000x3000. However, the problem is the ** visible ** image. (That is, the sunlight is affected by the seasons)
The image will look like the one below.
Source: Provided by Kochi University, University of Tokyo, Japan Meteorological Agency
Wow there is a black spot. .. ..
Well, it's a trial.
The average AUC is 0.675, which is lower than the conventional one.
The statistics are as follows.
| weather | precision | recall | f1-score |
|---|---|---|---|
| rain | 0.69 | 0.45 | 0.54 |
| Fine | 0.77 | 0.9 | 0.83 |
| average | 0.75 | 0.76 | 0.74 |
The accuracy is improved when viewed with F Score. Since the AUC is 0.79, the accuracy of the fine weather has improved, but the accuracy of the rain seems to have decreased slightly.
result
I tried various things,
- In the ternary classification, how to handle "cloudy" is still a problem, but the accuracy of rain has improved.
- Increasing Epoch seems to be effective.
- Introducing a difference series will rather reduce the accuracy. There is a possibility that the method of introduction is bad. Since the scale is different between pure image data and difference series, it may not be good for 6 channels.
- If you make a visible image, the accuracy of fine weather will increase unexpectedly. (Since it is converted to RGB, does it matter much in dark places?)
It became like.
Is it a good pattern to increase Epoch, classify it into three values (I want to improve rain accuracy), and make it a visible image (I want to improve fine weather accuracy)?
I want to aim for AUC 0.8 for the time being.
Since there are time-series elements, is it more accurate to extend it to RNN? (I have to study)
Recommended Posts