Try to draw a "weather map-like front" by machine learning based on weather data (1)

Let's draw a "weather map-like front" by machine learning based on weather data Part 1

It is the whole flow

Introduction

I am an SE who has been in charge of technical calculation systems for a long time after graduating from earth science when I was a student. In response to the machine learning boom that started several years ago, I was playing with weather data using my Mac mini at home because I wanted to try something after studying.

This time, I tried to draw a "front" by machine learning, so I summarized it (Qiita's first post).

Since there are multiple processes, I am thinking of posting in several times. This article is a general flow and a preliminary announcement.

What I did this time

"Weather map" is created by the Japan Meteorological Agency based on meteorological observation data. Fronts, high pressures, and low pressures are drawn on isobars, which are often seen on TV and newspapers. The figure below is the "Breaking Weather Map (SPAS)" at 18:00 UTC on November 25, 2019 (3:00 JST 26th).

A "stationary front" extends from the southeastern sea of the Kanto region toward the northeast, and a "cold front" extends further east from 150 ° east longitude.

SPAS_COLOR_201911251800.v8.256.png

A "front" is the boundary between air masses of different nature, usually the boundary between cold and warm air. However, when we visualize the meteorological data (wind, atmospheric pressure, temperature) on the surface of the earth at this time,

gsm_wnd_srf_2019112518.v8.256.png

It's like that (I used the GSM GPV downloaded from the Research Institute for Sustainable Humanosphere, Kyoto University). There seems to be a boundary between cold and warm air all over the place, and I feel like "Why is only one drawn on the weather map the front?" In this figure, the wind is arrow feathers, the thick contour line is atmospheric pressure, and the thin contour line and color (change from warm to cold from red to blue) represent temperature.

If the front is drawn automatically, it may be that the place where the difference is large is calculated based on the temperature data, but in that case the one drawn in the weather map is I don't feel like being identified. Therefore, the theme was to draw a "weather map-like front" by machine learning.

In conclusion, it is the front as shown in the figure below that finally generated machine learning. tst_chart_ovlyd2019_112518.v8.256.png

This figure is a pure visualization of barometric pressure overlaid with machine-learned fronts. Well, a front is generated at a reasonable position.

In reality, this is a fairly successful case, and in other cases it's not.

Breaking weather map (September 19, 2019, 6:00 UTC) SPAS_COLOR_201909190600.v8.256.png

Machine learning drawing tst_chart_ovlyd2019_091906.v8.256.png

In this case, a stationary front in the south is generated, but various fronts (occluded, cold, warm) extending from the blocked cyclone near Sakhalin in the north of Hokkaido are not generated.

Machine learning flow

Specifically, using numerical meteorological information (GPV), visualization of some meteorological elements is input, and the frontal map of the "flash weather map" released by the Japan Meteorological Agency is used as teacher data, U-Net. I learned it by a like CNN.

Data to enter

An image that visualizes ground surface data (atmospheric pressure, temperature, atmospheric pressure) and sky data (atmospheric pressure, temperature, wind, equivalent potential temperature, dew point) from the initial time data of the Global Forecast Model (GSM) of the Meteorological Agency.

Taking UTC at 18:00 on November 25, 2019 as an example, it is as follows.

Ground surface data gsm_wnd_srf_2019112518.v8.256.png

Data in the sky Atmospheric pressure, wind, temperature 850hPa (near 1500m above the sky) gsm_wnd_850_2019112518.v8.256.png

500hPa (near 5500m above the sky) gsm_wnd_500_2019112518.v8.256.png

Equivalent potential temperature (850hPa) Equivalent potential temperature is a concept that combines the amount of water vapor and the temperature. A large value indicates high temperature and humidity. gsm_ept_850_2019112518.v8.256.png

Dew point depression (700hPa) Dew point depression is the difference between air temperature and dew point temperature, and the higher the humidity, the smaller the value. This is colored within 3 degrees. gsm_situ_700_2019112518.v8.256.png

Plumb bob (700hPa) It represents the movement of air in the vertical direction and is a measure of convective activity (precipitation activity). gsm_vvl_700_2019112518.v8.256.png

What is output

From the image above, CNN will generate the following figure.

tst_maskout2019_112518.v8.256.png

This and the barometric pressure map visualized from the meteorological data gsm_prs_srf_2019112518.v8.256.png

The final output is the result of superimposing the above and then entering the time.

tst_chart_ovlyd2019_112518.v8.256.png

In other words, where should CNN itself look at multiple images and draw a "front"? You are learning.

Flow of future posts (planned)

I will post about the process I did this time in the following order.

[Part 2: Try to draw a "weather map-like front" by machine learning based on weather data (2)](https://qiita.com/m-taque/items/988b08185097dca5f5b5 "Based on weather data Let's draw a "weather map-like front" by machine learning (2) ") It is a story of weather data visualization as input. The story of visualizing numerical data in GPV format (handling of matplotlib, GRIB format)

[3rd: Draw a "weather map-like front" by machine learning based on weather data (3)](https://qiita.com/m-taque/items/4d5bb45e6b5dc42dc833 "Based on weather data Let's draw a "weather map-like front" by machine learning (3) ") A story about extracting fronts from color weather maps for training data, that is, a story about extracting only fronts from SPAS of the Japan Meteorological Agency to create teacher data.

[4th: Try to draw a "weather map-like front" by machine learning based on weather data (4)](https://qiita.com/m-taque/items/80ba51b74167b2aa669e "Based on weather data Let's draw a "weather map-like front" by machine learning (4) ") The story of making a CNN that colorizes black and white weather maps to increase color weather maps In the second spin-out story, there were few color weather maps, so I used CNN again to colorize the black and white weather maps.

[5th: Let's draw a "weather map-like front" by machine learning based on weather data (5) Machine learning Automatic Front Detection in Weather Data](https://qiita.com/m-taque/ items / 2788f623365418db4078)

This is the story of the front line drawing CNN, that is, the story of the CNN that is the learning body.

Well then.

Recommended Posts

Try to draw a "weather map-like front" by machine learning based on weather data (5)
Try to draw a "weather map-like front" by machine learning based on weather data (3)
Try to draw a "weather map-like front" by machine learning based on weather data (1)
Try to draw a "weather map-like front" by machine learning based on weather data (4)
Try to draw a "weather map-like front" by machine learning based on weather data (2)
A story about data analysis by machine learning
Machine learning beginners try to make a decision tree
Try to draw a Bezier curve
How to collect machine learning data
Try to make a blackjack strategy by reinforcement learning ((1) Implementation of blackjack)
Search for technical blogs by machine learning focusing on "easiness to understand"
Try to predict the value of the water level gauge by machine learning using the open data of Data City Sabae
[Machine learning] Create a machine learning model by performing transfer learning with your own data set
Try to draw a life curve with python
Notes on machine learning (updated from time to time)
Try to create a new command on linux
[Keras] I tried to solve a donut-type region classification problem by machine learning [Study]
Try to make a blackjack strategy by reinforcement learning (② Register the environment in gym)
Build a machine learning Python environment on Mac OS
Try to predict forex (FX) with non-deep machine learning
Time series data prediction by AutoML (automatic machine learning)
[Machine learning] Try to detect objects using Selective Search
xgboost: A valid machine learning model for table data
An introduction to machine learning from a simple perceptron
Build a machine learning environment natively on Windows 10 (x64)
How to interactively draw a machine learning pipeline with scikit-learn and save it in HTML
Data visualization with Python-It's too convenient to draw a graph by attribute with "Facet" at once
Try to make a blackjack strategy by reinforcement learning (③ Reinforcement learning in your own OpenAI Gym environment)
Introduction to machine learning
Build a python machine learning study environment on macOS sierra
Build a machine learning environment on mac (pyenv, deeplearning, opencv)
Try to build a deep learning / neural network with scratch
Try to evaluate the performance of machine learning / regression model
Introduction to Machine Learning with scikit-learn-From data acquisition to parameter optimization
Made icrawler easier to use for machine learning data collection
Try to evaluate the performance of machine learning / classification model
Dockerfile for creating a data science environment based on pip3
Machine learning beginners try to reach out to Naive Bayes (2) --Implementation
Create AI to identify Zuckerberg's face by deep learning ③ (Data learning)
Try to predict if tweets will burn with machine learning
Machine learning environment settings based on Python 3 on Mac (coexistence with Python 2)
Machine learning beginners try to reach out to Naive Bayes (1) --Theory
Using open data from Data City Sabae to predict water level gauge values by machine learning Part 2
I was frustrated by Kaggle, so I tried to find a good rental property by scraping & machine learning
How to quickly create a machine learning environment using Jupyter Notebook on macOS Sierra with anaconda