To access Twitter information from Python, use OAuth authentication and Exchange information through the Twitter API.
Below is a description of them and sample code.
Twitter uses an authentication method called OAuth authentication as the authentication method.
The details of how to create these are described on the following page, so refer to here to create four authentication keys. [Create twitter application](http://website-planner.com/ Create twitter application (consumer-key, consumer-secret, access-token, access-token-secret /)
This time, access is performed using the OAuth authentication library for Python called requests-oauthlib.
Execute the following command under the twi-py environment.
(twi-py)$ pip install requests requests_oauthlib
Operation check.
(twi-py)$ python
>>> from requests_oauthlib import OAuth1Session
>>>
OK if no error occurs.
Twitter API Basically, the following API is used to acquire Twitter data and send information from the program. https://dev.twitter.com/rest/public
The purpose of this time is to search with the specified keyword and get the necessary information. Use GET search / tweets. https://dev.twitter.com/rest/public/search
tweet_crawler.py
Sample code of a program that searches with the hashtag of #python and outputs the result.
Anyone who can read and write Python will know what they are doing in each function.
Basically, if you play around with url, params, oath.get () / oath.post (), Let's play with it because it can be operated using other Twitter APIs.
#!/usr/bin/env python
# -*- coding:utf-8 -*-
from requests_oauthlib import OAuth1Session
import json
### Constants
oath_key_dict = {
"consumer_key": "xxxxxxxxxxxxxxxxxxxx",
"consumer_secret": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
"access_token": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
"access_token_secret": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
}
### Functions
def main():
tweets = tweet_search("#python", oath_key_dict)
for tweet in tweets["statuses"]:
tweet_id = tweet[u'id_str']
text = tweet[u'text']
created_at = tweet[u'created_at']
user_id = tweet[u'user'][u'id_str']
user_description = tweet[u'user'][u'description']
screen_name = tweet[u'user'][u'screen_name']
user_name = tweet[u'user'][u'name']
print "tweet_id:", tweet_id
print "text:", text
print "created_at:", created_at
print "user_id:", user_id
print "user_desc:", user_description
print "screen_name:", screen_name
print "user_name:", user_name
return
def create_oath_session(oath_key_dict):
oath = OAuth1Session(
oath_key_dict["consumer_key"],
oath_key_dict["consumer_secret"],
oath_key_dict["access_token"],
oath_key_dict["access_token_secret"]
)
return oath
def tweet_search(search_word, oath_key_dict):
url = "https://api.twitter.com/1.1/search/tweets.json?"
params = {
"q": unicode(search_word),
"lang": "ja",
"result_type": "recent",
"count": "15"
}
oath = create_oath_session(oath_key_dict)
responce = oath.get(url, params = params)
if responce.status_code != 200:
print "Error code: %d" %(responce.status_code)
return None
tweets = json.loads(responce.text)
return tweets
### Execute
if __name__ == "__main__":
main()
(twi-py)$ python tweet_crawler.py
tweet_id: 531312631831732225
text:Shannon Lab holds regular python study sessions at the local Hachioji. Beginners of programming can also participate. Search for "Python study group Hachioji". ATND\
You can register from.#python
created_at: Sun Nov 09 05:09:37 +0000 2014
user_id: 426868502
user_desc:With creativity, change people and the world. Shannon Lab creates an innovative world through the fusion of mathematics, artificial intelligence and art. Programming study sessions and Android app products\
Tweet the information. Currently, we are planning a study session to make Android applications with Python. We are looking for internship engineers.
screen_name: shannon_lab
user_name: Shannon Lab
tweet_id: 531253415548944385
text:Looking at it like this, hands-on for beginners may be quite meaningful.
The story of a designer who met Python#PyLadiesTokyo by @uniq #python http://t.co/rZMEruT4yf @From SlideShare
created_at: Sun Nov 09 01:14:19 +0000 2014
user_id: 337419400
user_desc:Nichinichi engineer
screen_name: icoxfog417
user_name:Pixie
tweet_id: 531247311288946688
text:The story of a designer who met Python#PyLadiesTokyo by @uniq #python http://t.co/MwMFS5IFQ1 null
created_at: Sun Nov 09 00:50:03 +0000 2014
user_id: 1497018468
user_desc: Software Engineer
.NET/C#/HTML5/CSS3/JavaScript/jQuery/Perl/Ruby/Python
screen_name: teruru2013
user_name:Teru
...
https://dev.twitter.com/rest/public If you read the explanation of each API, you will find the following description.
Requests / 15-min window (user auth) 180
Requests / 15-min window (app auth) 450
This is the maximum number of times you can get information using the API per 15 minutes. The default is 180 times / 15min in this case.
Access from other than one Access token and Access token secret, If you use app auth, it will be 450 times / 15min.
Details of app auth are described at the following URL on the official page. https://dev.twitter.com/oauth/application-only
As soon as you collect information, you will get caught, so you need to consider a workaround. Basically, it is best to hold down the number of times that you do not hit the upper limit, If you need more search results than the limit in a short time, get multiple Twitter ids and When the response status reaches the upper limit of the key required for OAuth authentication There is a way to switch to the next key and continue acquisition (internally, it is a multiple Twitter application configuration). However, be aware that if you mass-produce your accounts too much, the risk of your accounts being frozen increases.
Recommended Posts