I'll write it first, but I'm not good at writing sentences, so if it's hard to read, I'm sorry. Well, I'm very upset to know that I can't see the list of likes because I used likes instead of stock. It seems that many people are worried about twitter and Qiita. [^ 1] [^ 2] So, scrape with python to get a list of likes.
After installing python3, use the pip command to request, BeautifulSoup4, progressbar2 Please install
Get the article list and save it in the results.json
file.
# -*- coding:utf-8 -*-
from bs4 import BeautifulSoup
import requests
from time import sleep
import json
from progressbar import ProgressBar
#Login to Qiita
payload = {
'utf8': '✓',
'identity': 'lovemuffim114', #username
'password': 'tkhr2783' #password
}
# authenticity_Get token
s = requests.Session()
r = s.get('https://qiita.com')
soup = BeautifulSoup(r.text, "html.parser")
auth_token = soup.find(attrs={'name': 'authenticity_token'}).get('value')
payload['authenticity_token'] = auth_token
#Login
s.post('https://qiita.com/login', data=payload)
#Dictionary for saving results
results = dict()
searchId = 1000000
param = {"before": searchId, "type": "id"}
#Get the number of articles
maxId = s.get("http://qiita.com/api/public", params=param).json()[0]["id"]
bar = ProgressBar(min_value=0, max_value=maxId) #Set maximum value
try: #Error when there are no more articles to load
while True:
param = {"before": searchId, "type": "id"}
#Get article list
j = s.get("http://qiita.com/api/public", params=param).json()
#Add articles that you like and do not stock to results.
# "url"other than,"uuid"Can also be specified.
results.update({i["title"]: i["url"] for i in j if i["liked"] and not i["stocked"]})
searchId = j[-1]["id"]
bar.update(maxId - searchId) #Progress bar update
sleep(1) # sleep(1 second)You can use a decimal point.
finally:
print(results)
# results.Save to json
with open("results.json", "a") as f:
json.dump(results, f, ensure_ascii=False, indent=4, separators=(',', ': '))
that's all. Result is,
{
"[Narou4j] I made a novel acquisition library to become a novelist in Java": "863aa22a29db16463e52",
"Accelerate ListView without using ViewHolder": "28f8be64d39b20e69552",
"[Narou4j] Created Java wrapper library for Narurou API": "6c050593f45174056005",
"I made my own hydroponics set.": "5d60c14d560ecf518a4e",
"RxJava + Flux (+ Kotlin)Android app design by": "cbf304891daec87ba5b7",
"I made EventBus with RxJava": "a4ece37834446c9a39c8"
}
It will be saved as. For logging in to Qiita, I used Login to website in Python.
[^ 1]: [[Caution! ] Articles will not be stocked if you just press the "Like" button on Qiita! I wrote an extension! ]](Http://qiita.com/gimupop/items/be53044143a9a3e90a4b) [^ 2]: [Twitter search results for "qiita Likes-! -Breakthrough"](https://twitter.com/search?f=tweets&vertical=default&q=qiita%20%E3%81%84%E3% 81% 84% E3% 81% AD% E3% 80% 80% E4% B8% 80% E8% A6% A7% E3% 80% 80% 20-% EF% BC% 81% 20-% E7% AA% 81% E7% A0% B4 & src = typd)
Recommended Posts