[RUBY] Get weather forecast from OpenWeatherMap in Rails

at first

This is Qiita's first post! Since I am a beginner, I think there are many points that I cannot reach, but I would be grateful if you could comment warmly!

Overview

--OpenWeatherMap which provides free api ) To get the weather forecast for all parts of the country by hitting the api. --HTTP request uses httpclient and implements it so that it can be hit regularly by making it a rake task.

Development environment

ruby: 2.7.1 rails: 6.0.3.2

procedure

1. Get API KEY
2. Get the CITY ID of the city you want to get
3. Create and save City table
4. Implementation of HTTP request
5. Create and save WeatherForecast table

1. Get API KEY

Go to the OpenWeatherMap home page and go to Sign in and create an account with Create an account. If you enable it from the email sent, you will receive an API KEY.

Save this as an environment variable or credentials. This time we will use credentials. Regarding credentials, this person's article is very helpful.

EDITOR=vi bin/rails credentials:edit

credentials.yml.enc

open_weahter:
    appid: <API_KEY>
    uri: https://samples.openweathermap.org/data/2.5/forecast

In addition, since I am going to get the weather every 3 hours this time, I am getting the URI to send the request by referring to the API document.

It seems that there are many types of weather that can be obtained even with the free tier! It's interesting to look around the API documentation.

2. Get CITY ID

Download city.list.json from the API documentation (https://openweathermap.org/forecast5). 

From this file, get the CITY ID of the city you want to get. This is a partial excerpt of the contents. By the way, lon means longitude and lat means latitude.

city.list.json

  {
    "id": 1850147,
    "name": "Tokyo",
    "state": "",
    "country": "JP",
    "coord": {
      "lon": 139.691711,
      "lat": 35.689499
    }
  },

I was crying and crying to get this id ... Please note that some cities with the same name have different latitudes and longitudes!

If you are using Excel or mac, you should list them in numbers and convert them to CSV! By the way, the CSV I created looks like this. I'm scraping some columns.

db/csv/cities.csv

Sapporo,2128295
Aomori,2130658
Morioka,2111834
Sendai,2111149
Akita,2113126
Yamagata,2110556
Fukushima,2112923
Mito,2111901
Utsunomiya,1849053
Maebashi,1857843
Saitama,6940394
Chiba,2113015
Tokyo,1850147
Yokohama,1848354
Niigata,1855431
Toyama,1849876
Kanazawa,1860243
Fukui,1863983
Yamanashi,1848649
Nagano,1856215
Gifu,1863640
Shizuoka,1851715
Nagoya,1856057
Tsu,1849796
Otsu,1853574
Kyoto,1857910
Osaka,1853909
Kobe,1859171
Nara,1855612
Wakayama,1926004
Tottori,1849890
Matsue,1857550
Okayama,1854383
Hiroshima,1862415
Yamaguchi,1848689
Tokushima,1850158
Takamatsu,1851100
Matsuyama,1926099
Kochi,1859146
Fukuoka,1863967
Saga,1853303
Nagasaki,1856177
Kumamoto,1858421
Oita,1854487
Miyazaki,1856717
Kagoshima,1860827
Naha,1856035

3. Create and save City table

Write the code in seeds.rb or task and save it in the database. This time, I implemented it under lib / tasks. CITY ID stores the column name as location_id.

import_csv.rake

  desc 'Import cities'
  task cities: [:environment] do
    list = []
    CSV.foreach('db/csv/cities.csv') do |row|
      list << {
        name: row[0],
        location_id: row[1],
      }
    end

    puts 'start creating cities'
    begin
      City.create!(list)
      puts 'completed!'
    rescue ActiveModel::UnknownAttributeError
      puts 'raised error: unknown attributes'
    end
  end

Read the cities.csv created earlier with the CSV.foreach method line by line. You can get the city name in the first column with row [0] and the CITY ID in the second column with row [1], so create an array of hashes and save it in the database with City.create! I am.

4. Implementation of HTTP request

Response analysis

Before implementing the HTTP request, first parse the response JSON file.

API document has a detailed explanation of each item, so please refer to it to get the key of the data you want. A request for one City will be returned in JSON format as shown below. (If you are using the curl command or VScode, you should try it with REST Client etc.)

example_resopnse.json

{
  "cod": "200",
  "message": 0,
  "cnt": 40,
  "list": [
    {
      "dt": 1578409200,
      "main": {
        "temp": 284.92,
        "feels_like": 281.38,
        "temp_min": 283.58,
        "temp_max": 284.92,
        "pressure": 1020,
        "sea_level": 1020,
        "grnd_level": 1016,
        "humidity": 90,
        "temp_kf": 1.34
      },
      "weather": [
        {
          "id": 804,
          "main": "Clouds",
          "description": "overcast clouds",
          "icon": "04d"
        }
      ],
      "clouds": {
        "all": 100
      },
      "wind": {
        "speed": 5.19,
        "deg": 211
      },
      "sys": {
        "pod": "d"
      },
      "dt_txt": "2020-01-07 15:00:00"
    },
    

This time, I am creating a table using the following items.

• list
  • main --feels_like: Feeling temperature --temp_max: Maximum temperature --temp_min: Minimum temperature
  • weather --id: Weather ID
  • rain ―― 3h: Precipitation

Rain is added to the list only if there is precipitation.

Please refer to here for the weather ID. OpenWeatherMap classifies the weather by ID. It is also possible to get the weather from description. However, since there are many types, in this implementation, the weather ID is stored in the database and the weather is assigned by the method.

HTTP request implementation

First, add httpclient to your Gemfile and bundle install.

gem 'httpclient', '~> 2.8', '>= 2.8.3'

Then create lib / api / open_weather_map / request.rb. I thought that it was not necessary to make it so deep, but I decided to arrange the file like this in consideration of implementing other classes for this api and implementing other api classes.

By default, only task is loaded under lib, so the following settings are required in config / application.rb. Since it is ʻeager_load_paths`, the production environment is also okay.

config/application.rb

config.paths.add 'lib', eager_load: true

I would be very happy if you could point out any points such as "Hey, this is awesome." File placement was the biggest problem with this implementation ...

WeatherForecast table to save requests ↓

The following is the Request class implemented.

request.rb

module Api
  module OpenWeatherMap
    class Request
      attr_accessor :query

      def initialize(location_id)
        @query = {
          id: location_id,
          units: 'metric',
          appid: Rails.application.credentials.open_weather[:appid],
        }
      end

      def request
        client = HTTPClient.new
        request = client.get(Rails.application.credentials.open_weather[:uri], query) #The return value is data every 3 hours for 5 days
        JSON.parse(request.body)
      end

      def self.attributes_for(attrs)
        rainfall = attrs['rain']['3h'] if attrs['rain']
        date = attrs['dt_txt'].in_time_zone('UTC').in_time_zone

        {
          temp_max: attrs['main']['temp_max'],
          temp_min: attrs['main']['temp_min'],
          temp_feel: attrs['main']['feels_like'],
          weather_id: attrs['weather'][0]['id'],
          rainfall: rainfall,
          date: date,
          aquired_at: Time.current,
        }
      end
    end
  end
end

I'm setting the query string with ʻinitialize. The query string required for this request adds location_id and API KEY to indicate the CITY ID, and ʻunits:'metric' to change the temperature display to Celsius.

The ʻattributes_for` method is made into a class method in order to convert the returned request into a form that can be saved in the database.

It is the precipitation and forecast date that need attention.

--For precipitation, there is no item if there is no precipitation. Therefore, it is conditional to get it only when it is. —— Regarding the forecast date, you need to pay attention to the time zone. Since the time zone of OpenWeatherMap is UTC, it is converted to JTC and then saved.

Please refer to this article for the handling of time zones.

5. Create and save WeatherForecast table

Creating a rake task

I want to save / update to the database regularly, so I will write it in rake task. That said, I wrote most of the methods in the Request class earlier, so all I have to do now is use them.

open_weather_api.rake

namespace :open_weather_api do
  desc 'Requests and save in database'
  task weather_forecasts: :environment do
    City.all.each do |city|
      open_weather = Api::OpenWeatherMap::Request.new(city.location_id)

      #Request limit: 60 times/min
      response = open_weather.request

      #Save 2 days of data every 3 hours
      16.times do |i|
        params = Api::OpenWeatherMap::Request.attributes_for(response['list'][i])
        if weather_forecast = WeatherForecast.where(city: city, date: params[:date]).presence
          weather_forecast[0].update!(params)
        else
          city.weather_forecasts.create!(params)
        end
      end
    end
    puts 'completed!'
  end
end

This time, we made it a specification to save and update the data every 3 hours in the database for 2 days. The point is the request limit and whether it is data creation or update.

--The request limit is 60 calls / min in Free Plan. If you have more than 60 registered cities, you will need to send them separately. This time it is 47, so there is no problem. --The presence method is a method that calls thepresent?Method and returns the receiver itself if true. If the forecast for the same time in the same city already exists in the database, ʻupdate!Is called, otherwisecreate!` Is called.

Finally

If you prepare a weather icon corresponding to the saved weather_id, it will look like a weather forecast! I think it's a good idea to hit the api regularly with cron or heroku schedular for heroku!

It was displayed like this!

Thank you for reading the lengthy text!

reference

https://openweathermap.org/api https://qiita.com/yoshito410kam/items/26c3c6e519d4990ed739