Qiita's Advent Calender has a public ranking. Overall calendar ranking
At first, I took a look at it for the purpose of finding an interesting calendar, but when I looked closely, we were also ranked! : clap :: clap ::
However, even if the number of likes is 1, it is ranked, so it is natural.
It was the third day for the first time since I knew that it was ranked ** "What is the ranking today?" **. This is a hassle ...
I decided to automate the ranking acquisition, saying, "Since Qiita has an API, you can get the ranking with it."
Qiita has an API. Here → Qiita API v2 specifications However, even if you look at the documentation, there is no Advent Calender API: sob:
If there is no API, I decided to scrape with Python.
When I searched for Python scraping
, I found many Beautiful Soup, so I decided to use ** Beautiful Soup **.
The purpose is clear. To get the ranking of your company. I looked up the HTML tags.
To get the ranking numbers, go up two from the calendar link <a class="adventCalendarRankingListItem_calendarName">
and get the text.
I tried to make a code using Beautiful Soup.
from urllib import request
from bs4 import BeautifulSoup
targethref = '/advent-calendar/2019/fork'
def main():
url = 'https://qiita.com/advent-calendar/2019/ranking/feedbacks/all'
targetclass = 'adventCalendarRankingListItem_calendarName'
response = request.urlopen(url)
soup = BeautifulSoup(response,features="html.parser")
ranking = soup.find('a',class_=targetclass,href=targethref).parent.parent.contents[0].text
response.close()
print(ranking)
if __name__ == "__main__":
main()
That's it, it's that easy! ** BeautifulSoup ** Great! (It's my feeling because I don't know other libraries that can be scraped)
target href =
on the 4th line. *I run this logic on AWS Lambda once a day. The result of the execution was thrown to the in-house Chat tool to share the information.
It was a day when I realized that I could scrape even if the API wasn't published: sunny:
:fork_and_knife: FORK Advent Calendar 2019 The Advent Calender article I wrote is here
Recommended Posts