I don't tweet, but I want to use tweepy: just display the search results on the console

1. What you can and cannot learn from this article

What you can learn

--Part of the basic usage of tweepy. --Example of a program that utilizes tweepy.

What you can't learn

--Beautiful Python design. --Creative usage of tweepy.

2. Actual code

twiiter.py


import tweepy
from collections import deque
from threading import Thread

#Minimum preparation before using tweepy
CONSUMER_KEY='XXXXXXX'
CONSUMER_SECRET='XXXXXXXX'
ACCESS_TOKEN='XXXXXXXX'
ACCESS_SECRET='XXXXXXXX'
auth = tweepy.OAuthHandler(CONSUMER_KEY, CONSUMER_SECRET)
auth.set_access_token(ACCESS_TOKEN, ACCESS_SECRET)
#So far

class Format(object):
    """Class to provide with a variety of formats."""

    def __init__(self, tweet):
        """Store some data of tweets in a hash-table. """
        self.status = {
                'text': tweet.text,
                'author': tweet.author.screen_name,
                'reply_name': tweet.in_reply_to_screen_name,
                'created_at': tweet.created_at,
                }

class StreamingFormat(Format):

    def format(self):
        """Format tweet text readable with some information."""
        author     = self.status['author']
        created_at = self.status['created_at']
        string = '@\033[33m{}\033[0m:{}\n'.format(author, created_at)
        string += self.status['text']
        string += '\n'
        return string

class Memory(deque):
    """Class to have hashed tweet contents."""

    def __init__(self, capacity):
        """Define max capacity of self."""
        super().__init__()
        self.capacity = capacity

    def append(self, item):
        """Append unless over capacity."""
        if len(self) > self.capacity:
            self.popleft
        super().append(item)

class Producer(Thread):
    """Class to get tweets and avoid duplication. """

    def __init__(self, queue, api, query):
        """ Initialize memory to avoid duplication. """
        super().__init__(target=self.main)
        self.api = api
        self.queue = queue
        self.query = query
        self.memory = Memory(1000)
    
    def main(self):
        """ Thread to add tweets to queue. """
        while True:
            for tweet in self.twigen():
                if tweet is not None:
                    self.queue.put(tweet)
            """
            https://syncer.jp/what-is-twitter-api-limit
According to the above site, Twitter takes Tweet
There is a profit limit, and if you access more than that,
You need to be careful because it will be limited.
15 minutes to search from all Tweets
There is a limit of 180 times, so once every 5 seconds
The pace is maximum. You don't have to attack the last minute, so
Set it about every 10 seconds.
            """
            time.sleep(10)

    def twigen(self):
        """Yield a tweet after hashing the tweet and memorizing it"""
        import hashlib

        tweets = self.api.search(q=self.query, count=100)
        for tweet in tweets:
            hashing = hashlib.md5(tweet.text.encode('utf-8'))
            hashed_tweet = hashing.hexdigest()
            if hashed_tweet in self.memory:
                yield None
            else:
                self.memory.append(hashed_tweet)
                yield tweet

class Consumer(Thread):

    def __init__(self, queue):
        super().__init__(target=self.main)
        self.queue = queue

    def main(self):
        """Take tweet from queue and print it."""
        while True:
            q = self.queue.get()
            f = StreamingFormat(q)
            print(f.format())
            time.sleep(1)

class TwitterStreamer(object):
    """Class to print formatted tweet data on console."""

    def __init__(self, api, query=None):
        from queue import Queue

        queue = Queue()
        Producer(queue, api, query).start()
        Consumer(queue).start()

if __name__ == '__main__':

    opts = {
            "api":tweepy.API(auth), 
            "query":'Honey badger' 
            }

    TwitterStreamer(**opts)

The acquisition method of CONSUMER_KEY, CONSUMER_SECRET, ACCESS_TOKEN, ACCESS_SECRET is omitted. Prepare it because it is absolutely necessary

@decimal1010:2016-04-02 10:36:36
Is Mitsuanaguma a honey badger?

@anaguma86:2016-04-02 09:53:06
A honey badger that gets excited about the notation of bottlenose dolphins instead of bottlenose dolphins.

@kyuurin525:2016-04-02 09:28:04
I opened something and became a honey badger on the 5th page.

@tsube_kii:2016-04-02 08:05:31
I didn't know what kind of animal the Lion Guard was with Kion at all, but the honey badger is an animal ...

When executed, it continues to be output to the console like this.

Recommended Posts

I don't tweet, but I want to use tweepy: just display the search results on the console
I want to output to the console coolly
I want to use Linux on mac
I want to use IPython Qt Console
I want to display the progress bar
I want to tweet on Twitter with Python, but I'm addicted to it
For the time being using FastAPI, I want to display how to use API like that on swagger
I don't want to use -inf with np.log
I want to use the activation function Mish
Save the search results on Twitter to CSV.
I want to display the progress in Python!
I want to use Python in the environment of pyenv + pipenv on Windows 10
I want to scroll the Django shift table, but ...
I want to use OpenJDK 11 on Ubuntu Linux 18.04 LTS / 18.10
I want to use the R dataset in python
I tried to scrape YouTube, but I can use the API, so don't do it.
I want to use the latest gcc without sudo privileges! !!
I want to use only the normalization process of SudachiPy
[Python] I want to use the -h option with argparse
I want to use the Ubuntu desktop environment on Android for the time being (Termux version)
I want to use Ubuntu's desktop environment on Android for the time being (UserLAnd version)
[Selenium] I want to display the browser by hitting the driver on the host OS from WSL
I don't want to search for high para because it is IQ1 (how to use lightgbm_tuner)
I want to use shortcut translation like DeepL app on Linux
I tried cross-validation based on the grid search results with scikit-learn
I want to use Ubuntu's desktop environment on Android for the time being (Termux version-Japanese input in desktop environment)
I want to display an image on Jupyter Notebook using OpenCV (mac)
Search for Twitter keywords with tweepy and write the results to Excel
I want to use the Qore SDK to predict the success of NBA players
I want to pin Spyder to the taskbar
I want to handle the rhyme part1
How to use Tweepy ~ Part 1 ~ [Getting Tweet]
I want to handle the rhyme part3
I want to use jar from python
I'm an amateur on the 14th day of python, but I want to try machine learning with scikit-learn
I want to handle the rhyme part2
I want to develop Android apps on Android
I want to handle the rhyme part5
I want to handle the rhyme part4
The file edited with vim was readonly but I want to save it
python I don't know how to get the printer name that I usually use.
I just want to add scipy, but it's a messy note. Ubuntu, Python 3.
I want to use PyTorch to generate something like the lyrics of Japari Park
I tried to display the infection condition of coronavirus on the heat map of seaborn
I want to be notified when the command operation is completed on linux!
[Writing] I want to display the variables used in the program on the server side in real time on the browser and update them.
I want to use MATLAB feval with python
I want to display multiple images with matplotlib.
I want to handle the rhyme part7 (BOW)
I don't want to take a coding test
I want to use Temporary Directory with Python2
I want to use ceres solver from python
I want to use ip vrf with SONiC
I want to do pyenv + pipenv on Windows
I want to log file I / O on Linux
I want to customize the appearance of zabbix
I use python but I don't know the class well, so I will do a tutorial
I want to use a python data source in Re: Dash to get query results
I want to plot the location information of GTFS Realtime on Jupyter! (With balloon)
I tried to understand how to use Pandas and multicollinearity based on the Affairs dataset.
[Don't refer to 04.02.17] Display the temperature sensor on a real-time graph with rasberry pi 3.