I continue to skip writing articles on microeconomics and environmental economics. I have a feeling that the peer-reviewed results of the papers submitted to domestic journals (a rag-like criticism named, I don't think it will be rejected, but I feel that it hurts ...) will be returned, so for the correction work. I don't think I can write that article for the time being ...

This time, I will introduce the java application that I made in class when I was a general manager of Kyoto University (2016, about 3 years ago) (because it is a jar file, you need to install java to use it). Since that time, I've been in the lab all the time, and I've been developing small-scale apps that could be used for research in environmental economics, but the java class I was taking at that time was "anything is fine." "Make an app from the beginning and announce it", so I thought that it could be used for research anyway. Past Meteorological Data Search I made a program to scrape (/etrn/index.php). Below is the link.

I still use this program often because I often use meteorological data in my research in environmental economics. Below is an image of the app.

First, select the prefecture where you want to acquire weather data from the pull-down menu of the prefecture. Then, the name of the city / ward / town / village office of the prefecture will appear in the city / ward / town / village (office name). Simply select the name of the municipal office for which you want to acquire weather data, specify the period of the data you want to acquire, and download it. It is output in csv file format (with UTF8 BOM).

The only information to be acquired is precipitation, minimum / maximum / average temperature, sunshine duration and date, which are frequently used data in terms of environmental economics.

Not all cities, wards, towns and villages have meteorological stations. Therefore, this program automatically identifies the observatory in the same prefecture that is the shortest distance from the specified municipality (latitude / longitude of the government office), and records the records held by that observatory. Download. You can find out which observatory is the closest observatory to the designated municipal office by looking in the text area of this program.

For the latitude and longitude of the municipal office, see the National Land Numerical Information Download Service of the Geographical Survey Institute, which provides GIS-related data. , Obtained from an XML file that contains facility information (address, latitude, longitude, etc.) of the municipal office. This XML file is included with the program. (If the latitude / longitude changes due to the relocation of the government office, you can download and update the XML file each time, but I have never done it, so the default is 2016)

~~ By the way, there may be no download data depending on the municipality in the countryside (!) ~~ I'm sorry, this was apparently because the uploaded jar file was left as it was the one that exported the old source project that didn't fix the bug. Exporting from the latest version fixed it. The files in DropBox above have been updated to the new version.

As mentioned above, this program was originally created to collect data on precipitation, temperature, and sunshine duration, but meteorological stations nationwide do not record all of these. For example, there are small automated stations that do not record daylight hours. Except for observatories that do not record such necessary data, the program selects the one closest to the designated municipal office among the observatories that observe at least the three points of "precipitation, temperature, and sunshine time."

Also, I think that it is the case in the university practice room and the computer room, but for those who work in an environment that requires proxy setting when connecting to the Internet from such a program, a proxy setting function is provided.

This time is over.

Source code

Since the source is long, I gave up the individual explanation. So below, the entire source code is released. Source (Dropbox)

Since I was working with basic eclipse, the directory structure is similar to that. The source is in the src directory. The GIS_XML directory contains XML files related to the location information of public facilities (government offices, etc.) for 47 prefectures downloaded from the Geographical Survey Institute. Since I am using jsoup as an HTML parser, it is included. It may be good to change the area to the latest version as appropriate.

[JAVA] A scraping of past weather that can be seen on the Japan Meteorological Agency website

Source code