Crawling with Python and Twitter API 2-Implementation of user search function

What to do this time

Until the last time, we implemented the Butler class for simple search of tweets and the Tweet class for handling tweets. This time, I would like to implement the function to search for User and the User class in the Butler class.

Implementation up to the last time

I will put the code up to the last time first.

tweet_butler.py

from . import config
from requests_oauthlib import OAuth1Session
import json

class Butler:
    def __init__(self,setting = config):
        self.api = OAuth1Session(setting.Consumer_Key, setting.Consumer_Secret_Key, setting.Access_Token, setting.Access_Secret_Token)
        self.search_url = "https://api.twitter.com/1.1/search/tweets.json"

    def search(self,word,count=10):
        params = {"q":word,"count":count}
        res = self.api.get(self.search_url, params = params)
        if res.status_code == 200:
            search_results = json.loads(res.text)
        tweets = [Tweet(result) for result in search_results["statuses"]]
        return tweets

class Tweet:
    def __init__(self,tweet):
        self.dict = tweet
        self.tweet_id = tweet["id"]
        self.user_id = tweet["user"]["id"]
        self.user_name = tweet["user"]["screen_name"]
        self.text = tweet["text"]
        self.created_at = tweet["created_at"]
        self.favorite_count = tweet["favorite_count"]
        self.hashtags = tweet["entities"]["hashtags"]
        self.symbols = tweet["entities"]["symbols"]
        self.mention = tweet["entities"]["user_mentions"]

    def get_text(self):
        return self.text

User search method

You can search the API simply by sending a GET request to a URL. There seem to be the following three methods when searching for users. ・ Users / lookup ・ Users / search ・ Users / show search is to search for users who hit a certain word, show is to output only one user with the same id and screen_name, lookup is extended to multiple people in show, and it is listed in id and screen_name. Can be entered. None of them are difficult to implement, but this time we will use show first.

Implementation of user search function

The first thing you need is an API token. This was explained last time, so I will omit it. Crawling with Python and Twitter API 1-Simple search function

Next is the resource URL for users / show. 　https://api.twitter.com/1.1/users/show.json You also need to know about the json files you throw into this API to implement.

{
  "Name": "name",
  "user_id": "id",
  "screen_name": "screen_name",
  "include_entities": "include_entities",
}

The above four are the elements that can be used as the input of the json file. Of these, one of Name, user_id, screen_name is always required.

The rest is implementation. This is not much different from the last time. Implement as a method of the Butler class. Butler.get_user_by_id() First, it is a method to get the user from the ID using showAPI.

tweet_butler.py

class Butler:
    def __init__(self, setting = config):
        self.get_user_url = "https://api.twitter.com/1.1/users/show.json"
              :
              :

    def get_user_by_id(self, id):
        params = {"user_id":user_id}
        res = self.api.get(self.get_user_url, params = params)
        if res.status_code == 200:
            search_results = json.loads(res.text)
        else:
            assert False, "{} error happen in get_user_by_id method".format(res.status_code)
        return User(search_results)

Butler.get_user_by_name Next is the method to get the user from screen_name. The API used is the same

tweet_butler.py

        :
        :
def get_user_by_name(self,name):
        params = "Name":name}
        res = self.api.get(self.get_user_url, params =params)
        if res.status_code == 200:
            search_results = json.loads(res.text)
        else:
            assert False, "{} error happen in get_user_by_name method".format(res.status_code)
        return User(search_results)

Implementation of User class

This time, I won't put a big deal on the User class.

tweet_butler.py

class User:
    def __init__(self, user_dict):
        self.dict = user_dict
        self.id = user_dict["id"]
        self.name = user_dict["name"]
        self.screen_name = user_dict["screen_name"]

I'm sorry for the omission.

Tweet class improvements

It seems that the user object is included in the Tweet object returned in the response, so add a new get_user method as a method of the Tweet class. I've also changed some of the class variable names.

tweet_butler.py

class Tweet:
    def __init__(self,tweet):
        self.dict = tweet
        self.id = tweet["id"]
        self.user_id = tweet["user"]["id"]
        self.user_name = tweet["user"]["screen_name"]
        self.text = tweet["text"]
        self.created_at = tweet["created_at"]
        self.favorite_count = tweet["favorite_count"]
        self.hashtags = tweet["entities"]["hashtags"]
        self.symbols = tweet["entities"]["symbols"]
        self.mention = tweet["entities"]["user_mentions"]
    
    def get_text(self):
        return self.text
    
##########The part below is newly added###########
    def get_user(self):
        return User(self.dict["user"])

Continue

Next, I would like to enable user search to be searched using the users / search API and to be able to retrieve tweets of a specific user.

reference

Get tweets from specific users on Twitter twitter developer