It is important to collect information on websites, Twitter, blogs, papers, etc. in order to know the latest topics, but it is very troublesome. This time, we have realized the following functions in Slack to improve the efficiency of information collection.
--Inject RSS into Slack --Inject emails (mainly Google Alerts) into Slack --Inject tweets of specific users into Slack
It is convenient that you can centrally manage the necessary information without checking the timeline of each site, and that the read history remains.
Building an automatic collection system for Twitter is the most difficult (although most can be copied), and technically it uses the Twitter API, heroku, and Python (Tweepy).
An RSS reader is provided as an app feature of Slack, and it is very easy to execute. RSS can be streamed on a channel-by-channel basis, and bookmarks and read items are provided as Slack functions, so you can use it in a similar way to Feedly.
There is already a lot of information on how to do this.
https://slack.com/intl/ja-jp/help/articles/218688467-Slack-に-RSS-フィードを追加する
https://ferret-plus.com/14641
You can also collect post information of specific tags of Qiita by RSS. It's easy because you don't have to look at the timeline every time.
I also collect information through various e-mail newsletters and Google Alerts, so I need to put it into Slack. There is already a lot of information here as well. There are several methods, but I do the following.
Other methods include: I tried various things, but the above was the best in terms of usability.
Zapier
--Send a specific email to a specific channel. --For details on how to do this, refer to https://qiita.com/mr_t_free/items/fd05ac8306fd5fcc1b37, etc. ――However, Slack does not support HTML in the first place, and the layout of Google Alert etc. is broken and it is not easy to use (If you transfer to a direct message, the email will be embedded and displayed neatly)
Slack Email App
――I haven't tried it because it is charged. However, it has the advantage that it is easy to set up and you can forward mail to a specific channel, so it seems convenient if you can pay for it.
Slack for Gmail
--Since the transfer is performed manually, the information collection was not automated.
Gripping implementation using API
――I haven't tried it because it seems to take time.
At first I thought it would be easy with IFTTT, but it wasn't easy to use and I gave up after all. I built a Twitter automatic collection bot. The general procedure is as follows.
The following two are very helpful. Rather, it's almost a copy.
https://qiita.com/yuhkan/items/805159f88dd0ad6b21c7
https://stefafafan.hatenablog.com/entry/2015/11/14/220601
Mainly uses Python. Tweepy is required as a package.
https://kurozumi.github.io/tweepy/
pip install tweepy
pip install requests
I also use pytz for detailed processing. I also need git.
Use Twitter API, Slack Webhook, heroku.
The Twitter API requires an access token.
https://qiita.com/kngsym2018/items/2524d21455aac111cdee#consumer-api-keysアクセストークン情報を使用したpythonスクリプト
Slack Webhook requires the URL of your channel, so make a note of it as well.
https://slack.com/intl/ja-jp/help/articles/115005265063-Slack-での-Incoming-Webhook-の利用
heroku uses the free version (no credit registration). However, I will not be able to operate at full capacity for a month, so I plan to register for credit in the future.
I'm using Streaming instead of Rest API. This is much easier to implement. Specifically, pass the list of user IDs you want to collect as query totweepy.StreamListener.filter (follow = query). Converting a username to a username is also easy with tweepy. In addition to monitoring specific users, you can also monitor keywords.
As a practical implementation of the Slack transfer part, in the ʻon_status method of tweepy.StreamListener`,
--Processing to filter unnecessary tweets such as retweets --Processing process to format data for Slack transfer --Process to transfer to Slack
I will bite you. Filtering for unwanted tweets looks like below, for example.
def is_invalid_tweet(self, status):
if isinstance(status.in_reply_to_status_id, int):
#True for reply
return True
if "RT @" in status.text[0:4]:
#True for RT
return True
return False
https://www.pytry3g.com/entry/twitter-api-seq2seq
Below is the formatting to json.
import json
from datetime import timedelta
def format_status(status):
channel = '#twitter_collection'
text = status.text
status.created_at += timedelta(hours=9) #In Japan time
username = str(status.user.name) + '@' + str(status.user.screen_name) + ' (from twitter)'
json_dat = {
"channel": channel,
"username": username,
"icon_url": status.user.profile_image_url,
"text": text
}
json_dat = json.dumps(json_dat)
return json_dat
Transferring to Slack is very easy, just use request.post.
def post_to_slack(json_dat):
url = SLACK_WEBJOOK_URL
requests.post(url, data=json_dat)
return
When using heroku,
You will need the file. Profile describes the main process at runtime. The required packages are listed in requirements.txt.
worker: python main.py
requirements.txt
tweepy==3.8.0
requests==2.22.0
pytz==2019.3
It's important to note that we are using worker as Dyno instead of the web. On the web, ʻError R10` occurs.
https://qiita.com/m_rn/items/9a580d04781b34f64693
It looks like the following.
heroku login
git init
git add .
git commit -m "test"
heroku create #Register the app
git push heroku master #Transfer app
heroku scale worker=1 #Run the app
The web process can be deployed with git push heroku master, but the worker process requires heroku scale worker = 1.
Log monitoring is
heroku logs -t
heroku ps
The stop is as follows.
heroku scale worker=0
https://qiita.com/naberina/items/da4a6d3c480aa7a62b06
Delete
heroku apps:destroy --app app name
It will be.
https://qiita.com/chihiro/items/5c3ff400f6cb99deb945
--Register heroku credits. If you do not register your credit card, you will not be able to operate all the time ...