I made an original program guide using the NHK program guide API.

Motivation

API key acquisition flow

  1. Register your account at NHK Program API.
  2. After email authentication, register the application name: (your favorite name) and URL: (unregistered).
  3. When registration is complete, you will be issued an ʻAPI key`. ** This is required for acquisition. ** **

API overview

No. name Description
1 Program List API Specify region, service, date
2 Program Genre API Specify region, genre, date
3 Program Info API Specify the program ID
4 Now On Air API Specify region and service
https://api.nhk.or.jp/v2/pg/list/{area}/{service}/{date}.json?key={apikey}
Parameters Description value
area Region ID(3byte) 130:Tokyo,46 other regions
service Service ID(2byte) g1: NHK General,e1:E-tele,s1:BS,s3:BSP
date date(YYYY-MM-DD format) Example:2020-04-07
apikey API key(32byte) Obtained by app registrationAPI key
{
  "list": {
    "g1": [
      {
        "id": "2020040704706",
        "event_id": "04706",
        "start_time": "2020-04-07T04:02:00+09:00",
        "end_time": "2020-04-07T04:30:00+09:00",
        "area": {
          "id": "130",
          "name": "Tokyo"
        },
        "service": {
          "id": "g1",
          "name": "NHK General 1",
          "logo_s": {
            "url": "//www.nhk.or.jp/common/img/media/gtv-100x50.png ",
            "width": "100",
            "height": "50"
          },
          "logo_m": {
            "url": "//www.nhk.or.jp/common/img/media/gtv-200x100.png ",
            "width": "200",
            "height": "100"
          },
          "logo_l": {
            "url": "//www.nhk.or.jp/common/img/media/gtv-200x200.png ",
            "width": "200",
            "height": "200"
          }
        },
        "title": "Darwin is here! "Infiltrate the turbulent lion school! Train the king of beasts !!"",
        "subtitle": "Growing lion children learn hunting techniques and parenting techniques from adults who play the role of teachers in a herd. Unscrupulous students will be expelled! ?? Adhere to the turbulent days of the school drama!",
        "content": "In the first new year, we will introduce Lion's "school". Growing lion children learn everything from adult teachers to hunting and parenting to repelling rivals in a herd. However, rival hyenas have no teeth at all, and when hunting, they can be seen in full view and easily escape to prey. Even more unscrupulous students will be expelled from school! ?? Infiltrate a turbulent class that is as good as a school drama! Song: MISIA",
        "act": "[Talk] Mayuko Wakuda, Naoki Tatsuta, Machiko Toyoshima, Takayuki Yamada, Inori Minase",
        "genres": [
          "0802",
          "1000"
        ]
      },
{Omitted below}

Data acquisition

import pandas as pd
import json
import requests
import datetime

#Set the obtained API key
apikey = 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'

# 130:Tokyo
area = '130'
# g1:NHK General 1,e1:E-tele,s1:BS,s3:BSP
service = ['g1','e1','s1','s3']
#Date and time
date = datetime.date.today()

all_results = pd.DataFrame(index=None, columns=[])
for i in range(len(service)):
    url = 'https://api.nhk.or.jp/v2/pg/list/{0}/{1}/{2}.json?key={3}'.format(area,service[i],date,apikey)
    request_get = requests.get(url)
    if request_get.status_code != 200:
        print('The data of NHK program guide API cannot be acquired.')
        break
    result = pd.json_normalize(request_get.json(), ['list',[service[i]]])
    all_results = pd.concat([all_results, result])

all_results = all_results[~all_results['title'].str.contains('Broadcast suspension')]

Summary of acquired data:

<class 'pandas.core.frame.DataFrame'>
Int64Index: 235 entries, 0 to 37
Data columns (total 22 columns):
 #   Column                 Non-Null Count  Dtype 
---  ------                 --------------  ----- 
 0   id                     235 non-null    object
 1   event_id               235 non-null    object
 2   start_time             235 non-null    object
 3   end_time               235 non-null    object
 4   title                  235 non-null    object
 5   subtitle               235 non-null    object
 6   content                235 non-null    object
 7   act                    235 non-null    object
 8   genres                 235 non-null    object
 9   area.id                235 non-null    object
 10  area.name              235 non-null    object
 11  service.id             235 non-null    object
 12  service.name           235 non-null    object
 13  service.logo_s.url     235 non-null    object
 14  service.logo_s.width   235 non-null    object
 15  service.logo_s.height  235 non-null    object
 16  service.logo_m.url     235 non-null    object
 17  service.logo_m.width   235 non-null    object
 18  service.logo_m.height  235 non-null    object
 19  service.logo_l.url     235 non-null    object
 20  service.logo_l.width   235 non-null    object
 21  service.logo_l.height  235 non-null    object
dtypes: object(22)
memory usage: 42.2+ KB

Data processing

#Convert to date format, calculate broadcast time
all_results['start_time'] = pd.to_datetime(all_results['start_time'], format='%Y/%m/%d %H:%M')
all_results['end_time'] = pd.to_datetime(all_results['end_time'], format='%Y/%m/%d %H:%M')
all_results['airtime'] = all_results['end_time'] - all_results['start_time']
all_results['link'] = all_results['id']
#Creating in-page link information
func = lambda x: x.replace(x, f'<div id="{x}">Program details</div>')
all_results['link'] = all_results['link'].apply(func)

#time. Get hours and minutes. In addition, add the time line name of the program guide
tmp = pd.concat([all_results['start_time'].dt.month ,all_results['start_time'].dt.day,
                 all_results['start_time'].dt.hour, all_results['start_time'].dt.minute,
                 (all_results['start_time'].dt.day.astype(str)+all_results['start_time'].dt.strftime('%H'))], axis=1)
tmp.columns = ['month', 'day', 'hour', 'minute','time_bins']

#Combine the retrieved data with the new column
all_results = pd.concat([all_results, tmp], axis=1)

#Aggregation for program listings
data = all_results.iloc[:,[23,0,1,2,3,4,5,6,7,8,9,10,11,12,22,24,25,26,27,28]]

Data shaping (1)

#Creating a data frame for the final program guide
tv_index = data.time_bins.sort_values().unique()
tv_table = pd.DataFrame(index=tv_index, columns=['H', 'g1', 'e1', 's1', 's3']).fillna('')

#Combine content and insert in time zone.
cell = ''
for s in range(len(service)):
    for i in range(len(tv_table.index)):
        tmp = data[(data['time_bins'] == tv_table.index[i]) & (data['service.id'] == service[s])]

        for c in range(len(tmp)):
            cell += '[' + str(tmp['minute'].iloc[c]) + '~]'
            cell += tmp['title'].iloc[c]
            cell += f'<a href="#{tmp["id"].iloc[c]}">▼</a><br>'
        tv_table[service[s]].iloc[i] = cell
        cell = ''

#Add time zone
for h in range(len(tv_index)):
    tv_table['H'].iloc[h] = int(tv_index[h][-2:])
tv_table.columns = ['H', 'NHK General', 'E-tele', 'BS', 'BSP']

Data shaping (2)

detail_table = data.stack().reset_index()
detail_table.drop(columns='level_0', inplace=True)
detail_table.columns = ['Type', 'value']
detail_table.set_index('Type', drop=True, inplace=True)

Data shaping (3)

html_template = '''
<!doctype html>
<html lang="ja">
  <head>
    <!-- Required meta tags -->
    <meta charset="utf-8">
    <meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">

    <!-- Bootstrap CSS -->
    <link rel="stylesheet" href="https://stackpath.bootstrapcdn.com/bootstrap/4.4.1/css/bootstrap.min.css" integrity="sha384-Vkoo8x4CGsO3+Hhxv8T/Q5PaXtkKtu6ug5TOeNV6gBiFeWPGFN9MuhOf23Q9Ifjh" crossorigin="anonymous">
    <title>Private program schedule</title> <!-- #1 -->
  </head>
  <body>
    <h2>&nbsp;I made an original program guide using the NHK program guide API.{date}Edition</h2> <!-- #2 -->

    <!-- Optional JavaScript -->
    <!-- jQuery first, then Popper.js, then Bootstrap JS -->
    <script src="https://code.jquery.com/jquery-3.4.1.slim.min.js" integrity="sha384-J6qa4849blE2+poT4WnyKhv5vZF5SrPo0iEjwBvKU7imGFAV0wwj1yYfoRSJoZ+n" crossorigin="anonymous"></script>
    <script src="https://cdn.jsdelivr.net/npm/[email protected]/dist/umd/popper.min.js" integrity="sha384-Q6E9RHvbIyZFJoft+2mJbHaEWldlvI9IOYy5n3zV9zzTtmI3UksdQRVvoxMfooAo" crossorigin="anonymous"></script>
    <script src="https://stackpath.bootstrapcdn.com/bootstrap/4.4.1/js/bootstrap.min.js" integrity="sha384-wfSDF2E50Y2D1uUdj0O3uMBJnjuUD4Ih7YwaYd1iqfktj0Uod8GCExl3Og8ifwB6" crossorigin="anonymous"></script>
    <style type="text/css"> th {{ text-align: center; background-color: #f5f5f5}}</style>
    <div class="container-fluid"> <!-- #3 -->
        {table}
    </div>
    <div class="container-fluid"> <!-- #4 -->
        {table1}
    </div>
    <div>Information provision:NHK</div> <!-- #5 -->
</body>
</html>

The following contents have been added to the definition information.

  1. title: Page title
  2. h2: Heading + {date}
  3. Program table of 4 stations table {table}-> table
  4. Program details table {table1}-> table1
  5. Credit display
table = tv_table.to_html(classes=['table', 'table-bordered', 'table-hover'],escape=False)
table1 = detail_table.to_html(classes=['table', 'table-bordered', 'table-hover'],escape=False)
html = html_template.format(table=table, table1=table1, date=date)

with open('g1_table_' + str(date) + '.html', 'w') as f:
    f.write(html)

Click ▼ to scroll to the detailed program data with the link on the page. スクリーンショット 2020-04-07 16.08.21.png スクリーンショット 2020-04-07 16.08.48.png

Reference site

Summary

Recommended Posts

I made an original program guide using the NHK program guide API.
How to use the NHK program guide API
I tried using the checkio API
Create an application using the Spotify API
I tried using the BigQuery Storage API
I made my own research tool using the legal API [Smart Roppo]
I checked the library for using the Gracenote API
I tried using the Google Cloud Vision API
I made a LINE BOT that returns a terrorist image using the Flickr API
I made an Ansible-installer
I tried using the API of the salmon data project
I built an application with Lambda that notifies LINE of "likes" using the Qiita API
Try using the Twitter API
I made an Xubuntu server.
I touched the Qiita API
Try using the Twitter API
I made an API with Docker that returns the predicted value of the machine learning model
I made a VGG16 model using TensorFlow (on the way)
Try using the PeeringDB 2.0 API
I made an automated program for pasting screenshots in Excel
[Python] I tried collecting data using the API of wikipedia
I made a Chatbot using LINE Messaging API and Python
[For beginners] I tried using the Tensorflow Object Detection API
I made a program to solve (hint) Saizeriya's spot the difference
I made a program that solves the spot the difference in seconds
I tried using the COTOHA API (there is code on GitHub)
I made an AI that crops an image nicely using Salience Map
I made a Chatbot using LINE Messaging API and Python (2) ~ Server ~
I made a program that automatically calculates the zodiac with tkinter
[Kaggle] I made a collection of questions using the Titanic tutorial
I tried the Naro novel API 2
I made a Line-bot using Python!
I made an Anpanman painter discriminator
I made an Angular starter kit
I tried the Naruro novel API
I tried to summarize various sentences using the automatic summarization API "summpy"
I made a program to check the size of a file in Python
I made an APL part with the Alexa skill "Industry Terminology Conversion"
I made an image discrimination (cifar10) model using a convolutional neural network.
Try using the Kraken API in Python
I made a payroll program in Python!
I tried using YOUTUBE Data API V3
I stumbled on the Hatena Keyword API
I made an online frequency analysis app
I made an alternative module for japandas.DataReader
Tweet using the Twitter API in Python
I created an Anaconda environment using Docker!
An introductory guide to program performance tuning
Beginner: I made a launcher using dictionary
I tried using UnityCloudBuild API from Python
I tried to touch the COTOHA API
I want to make an automation program!
Play with puns using the COTOHA API
Record custom events using the Shotgun API
When I tried using Microsoft's Computer Vision API, I recognized the Galapagos sign "Stop"
I made an appdo command to execute a command in the context of the app
With LINEBot, I made an app that informs me of the "bus time"
Create an app that works well with people's reports using the COTOHA API
I made an in-house peer bonus tool (aggregation only) using Python and BigQuery
I made a program to look up words on the window (previous development)
I made a script to record the active window using win32gui of Python