Python: Reading JSON data from web API

First of all, what is JSON?

It seems that the data structure is easy for both humans and computers to understand (roughly) [What is JSON?-Qiita](http://qiita.com/SotaSuzuki/items/c3b46c4e24c1ca9b4d37#%E6%96%87%E5%AD%97%E5%88%97%E3%81%AE%E8%A1 % A8% E7% 8F% BE)


It seems that you can read json data with python by importing the json library.

It seems that if you read it, you can handle it in the same way as the dictionary type. (I'm not sure about the dictinary type, so I'll study here.)

About dictionary type

--The dictionary type is a set (element) of key and value. --Something like hashMap in java? (I remember hashMap) --Since there is no order of elements, you have to specify the key to retrieve the elements.

Extract the value by specifying the key

test01.py


# -*- coding: utf-8 -*-

dict = {"name": "tamago", "color": "yellow"}

#Extract value by specifying key
print dict["name"]
print dict["color"]

test01.Output result of py


tamago
yellow

If you specify a key that does not exist and try to retrieve it, an Error will occur.

Get a list of keys and values

--Get a list of keys --Get a list of values --Get a list of keys and values

test02.py


# -*- coding: utf-8 -*-

dict = {"name": "tamago", "color", "yellow"}

#Get a list of keys
keyList = dict.keys()
print keyList

#Get a list of values
vakList = dict.values()
print valList

#Get a list of keys and values
list = dict.items()
print list

test02.Output result of py


['color', 'name']
['yellow', 'tamago']
[('color', 'yellow'), ('name', 'tamago')]

I don't know how to get it because the dictionary type doesn't have the order of the elements

I got a rough idea of the dictionary! !!

Get JSON data from Web API

The web API used this time is TwitCasting's comment acquisition API (Comment acquisition API --TwitCasting).

When getting JSON data from WebAPI, import ʻurlliband use the method to get it. To be able to handle the acquired data with python, importjson` and use the method

test02.py


# -*- coding: utf-8 -*-

import urllib
import json
import sys
import codecs

#Now you can eliminate the characters that cannot be converted to cp932
sys.stdout = codecs.getwriter(sys.stdout.encoding)(sys.stdout, errors='ignore')

#Get the result of JSON format string from webAPI
def dataGet():

    #URI scheme
    url = 'http://api.twitcasting.tv/api/commentlist?'

    #URI parameter data
    param = {
        'user': 'tamago324_pad',    #ID of the person you want to get
        'type': 'json'             #Specifying the data to be acquired
    }

    #Creating a string of URI parameters
    paramStr = urllib.urlencode(param)  # type=json&user=tamago324_Shaped as pad

    #Creating an object to read
    readObj = urllib.urlopen(url + paramStr)

    #Get JSON from web API
    response = readObj.read()

    # print type(response)  # >> <type 'str'>

    return response

#Convert data obtained from webAPI to JSON
def jsonConversion(jsonStr):

    #Convert JSON data obtained from webAPI into a form that can be used with python
    data = json.loads(jsonStr)
    return data

    #Japanese is u'\u767d'Because it becomes like, convert to Unicode
    # return json.dumps(data[0], ensure_ascii=False)

#Comment posting time h:mm:Convert to ss
def getElapsedTime(duration):

    secStr = ""
    minStr = ""

    hourInt = duration / 3600
    minInt = (duration -3600 * hourInt) / 60
    secInt = duration % 60

    if minInt <= 9:
        minStr = "0" + str(minInt)
    else:
        minStr = str(minInt)

    if secInt <= 9:
        secStr = "0" + str(secInt)
    else:
        secStr = str(secInt)

    if hourInt >= 1:
        return str(hourInt) + ":" + minStr + ":" + secStr
    else:
        return minStr + ":" + secStr

if __name__ == '__main__':

    resStr = dataGet()
    res = jsonConversion(resStr)

    #Display the acquired data
    for item in res:
        print getElapsedTime(item['duration']) + " " + item['userstatus']['name'] + " " + item['message']

10:14 It happens to be a test
09:49 It happens to be a comment

Get JSON data from Web API

def jsonGet (): is getting JSON data from webAPI </ strong> I am creating a string of URI parameters from dictionary with ʻurllib.urlencode () . If there are multiple strings returned by this method in the form of key = value, they will be combined with & (example: key1 = value1 & key2 = value2`). Since there is no order in the dictionary, you cannot specify the order in which they are combined.

ʻUrllib.urlopen () `creates an object to read JSON data

JSON data is obtained from webAPI by the method of the created object (ʻobject.read () ). The return value seems to be a str` type

Convert the acquired data to JSON that can be handled by Python

Convert the JSON obtained by json.loads () so that it can be handled by python. However, because it is returned in unicode, Japanese becomes like \ u3063`` json.dumps (data, ensure_ascii = False) was able to display Japanese firmly! I thought it was a string ... data was list ... Is this the only way to convert from unicode to str one by one ...

I was worried about what to do, but I tried to get only one comment, and when I displayed it with prinnt data [0] ['message'], it was displayed firmly in Japanese! !! !! However, the comment contains , and when I try to display it, I get the error ʻUnicodeEncodeError:'cp932' codec can't encode character u'\ ua4b3' in position 0: illegal multibyte sequence`.

What to do if characters cannot be displayed on the console due to UnicodeEncodeError </ b> When you want to print a Unicode character string that occasionally contains characters that cannot be expressed by cp932 on the Windows console --Yasukazu Nishio's Hatena Diary sys.stdout = codecs.getwriter (sys.stdout.encoding) (sys.stdout, errors ='ignore') Anyway, you can hide the characters that cannot be expressed by cp932 (the mechanism is unknown .. I'm thinking of clarifying it now)

Right now I'm doing it on the console, but I think that it will look better if it can be displayed with a windows application etc.

Dictionary-Introduction to Python Library: json --Life with Python 20.5. urllib — Accessing any resource by URL — Python 2.7.x documentation Try using Yahoo API with Python-BTY memorandum Get, post communication memo in Python --Qiita Get a Unicode JSON string including Japanese. --Qiita 18.2. json — JSON encoder and decoder — Python 2.7.x documentation

Recommended Posts