Create AtCoder Contest appointments on Google Calendar with Python and GAS

at first

The AtCoder contest is held irregularly, but I sometimes forget to schedule it and it seems to bother my family, so I decided to automate it (I don't know if I can do it). I used Google Colaboratory for execution. Being a beginner in programming, the code can be messy or wrong. sorry.

From AtCoder to Google Calendar

procedure

  1. Scraping from AtCoder using Python requests, Beautiful Soup, etc.
  2. Collect scraped data into Google Spread Sheets using Python's gspread etc.
  3. Create an appointment in Google Calendar using GAS from Google SpreadSheets

1. Scraping

Actual code Get only the name, date, and link of the AtCoder scheduled contest from the "Contest List" on the AtCoder homepage and make it a list.

from bs4 import BeautifulSoup #Import of Beautiful Soup
import requests #Import requests
import datetime
import re

url = "https://atcoder.jp/contests/"
response = requests.get(url).text
soup = BeautifulSoup(response, 'html.parser') #Beautiful Soup initialization
tags = soup.select("tbody a") #Select all a tags under tbody

l=[]
l_n=[]
l_link=[]
l_n1=[]

now=datetime.datetime.now().strftime('%Y-%m-%d %H:%M') #Get the current year, month, day, time

for i in tags:
  l.append(i.text) #Get the text in the a tag
  l.append(i.get("href")) #Get the link attached to the a tag
l.remove("practice contest") 
l.remove("/contests/practice")
l.remove("AtCoder Library Practice Contest")
l.remove("/contests/practice2")
l=[l[i:i + 4] for i in range(0,len(l), 4)]
for i in range(len(l)):
  l[i][0]=l[i][0][0:16]
  del l[i][1]
  if (l[i][0][0:4] > now[0:4]) or (l[i][0][0:4] == now[0:4] and l[i][0][5:7] > now[5:7])
 or (l[i][0][0:4] == now[0:4] and l[i][0][5:7] == now[5:7] and l[i][0][8:10] >= now[8:10]): #Remove past contests from list
    l_n.append(l[i])
for i in range(len(l_n)):
  l_link.append("https://atcoder.jp"+l_n[i][2]) #Change relative links to absolute links

It was difficult to get the start time, end time, rated and penalties of the contest, so I jumped to each link to get it.

for i in l_link:
  url_n=i
  response_n = requests.get(url_n).text
  soup_n = BeautifulSoup(response_n, 'html.parser') #Beautiful Soup initialization
  tags2=soup_n.select("span.mr-2")+soup_n.select("small.contest-duration") #Get Rated, Penalty, Holding Time, etc.
  for j in tags2:
    l_n1.append(j.text)
l_n1=[l_n1[i:i + 4] for i in range(0,len(l_n1), 4)]
for i in range(len(l_n1)):
  l_n1[i][1]=l_n1[i][1][13:]
  l_n1[i][1]=l_n1[i][1].replace("-","~")
  l_n1[i][2]=l_n1[i][2][9:]
  l_n1[i][3]=re.sub("\n","",l_n1[i][3])
  l_n1[i][3]=re.sub("\t","",l_n1[i][3])
  l_n1[i][3]=l_n1[i][3][54:60]
  del l_n1[i][0]
for i in range(len(l_n)):
  l_n[i]+=l_n1[i]

(Reference: https://dividable.net/programming/python/python-scraping)

2. Transfer data to spreadsheet

Paste the retrieved data into a spreadsheet.

from google.colab import auth
from oauth2client.client import GoogleCredentials
import gspread

auth.authenticate_user()
gc = gspread.authorize(GoogleCredentials.get_application_default())
worksheet = gc.open('AtCoderNewContestList').get_worksheet(0) #Specify the first sheet of a spreadsheet named AtCoderNewContestList

for i in range(len(l_n)):
  if worksheet.update_acell("B"+str(i+2),l_n[i][1]) in worksheet.range('B2:B10'): #Do not process if the contest has already been added
    continue
  else:
    worksheet.update_acell("B"+str(i+2),l_n[i][1])
    worksheet.update_acell("C"+str(i+2), l_n[i][0][0:4]+"/"+l_n[i][0][5:7]+"/"+l_n[i][0][8:10])
    worksheet.update_acell("D"+str(i+2),l_n[i][0][11:])
    worksheet.update_acell("E"+str(i+2),l_n[i][-1])
    worksheet.update_acell("F"+str(i+2),l_n[i][3])
    worksheet.update_acell("G"+str(i+2),l_n[i][4])
    worksheet.update_acell("H"+str(i+2),"https://atcoder.jp"+l_n[i][2])
  worksheet.update_acell("A"+str(i+2),"")

The result is below. image.png It's working pretty well.

3. Add appointment with GAS

Add appointments to Google Calendar using GAS. Due to various reasons, I decided to create a new Google account, access the created spreadsheet, extract data with GAS, and create a schedule. I can no longer access the account I created at the beginning. why? You can write GAS code by selecting "Script Editor" from "Tools" in the spreadsheet. (GAS can understand the contents of the code but cannot write it, so I copied and edited the code on the site below.) (Reference: https://qiita.com/cazimayaa/items/5fdfbc060dff7a11ee15)


function myFunction() {
  //Get the sheet of the currently selected spreadsheet
  var sheet = SpreadsheetApp.getActiveSheet();
  //Get the contents of the cell from the obtained sheet
  var values = sheet.getDataRange().getValues();
  var calendar = CalendarApp.getDefaultCalendar();
  //* The 0th part of var i is the header, so start from 1.
  for (var i = 1; i < values.length; i++) {
    var status = values[i][0];
    if (
      status != "Already" // 連携の欄がAlreadyになっていなかったら処理を行う
    ) {
      //scheduled date
      var date = values[i][2];

      //Start time
      var startTime = values[i][3];
      var startDateTime = new Date(date.getFullYear(),
                                   date.getMonth(),
                                   date.getDate(), 
                                   startTime.getHours(),
                                   startTime.getMinutes(), 0);
      //ending time
      var endTime = values[i][4];
      var endDateTime = new Date(date.getFullYear(),
                                 date.getMonth(),
                                 date.getDate(),
                                 endTime.getHours(),
                                 endTime.getMinutes(), 0);
      //title
      var title = values[i][1]+values[i][8];
      var options = {
        description: values[i][7]
      }
      // var event = calendar.createEvent(title, startDateTime, endDateTime);
      //Just add options to the argument
      var event = calendar.createEvent(title, startDateTime, endDateTime, options);
      //Register to calendar
      sheet.getRange(i + 1, 1).setValue("Already"); // 連携の欄をAlreadyにする
    }
  }
}

It may be fine as it is, but I want to link the calendar when the spreadsheet is updated, so set a trigger. (Reference: [https://auto-worker.com/blog/?p=1646](https://auto-worker.com/blog/?p=1646 https://auto-worker.com/blog/? p = 1646))) From "Trigger of current project", execute the function when the sheet is changed.

After execution calendar image.png Spreadsheet image.png I did it.

4. Supplement

I really wanted the Python code to run automatically on a regular basis, but I stopped it because it would be a hassle.

Recommended Posts

Create AtCoder Contest appointments on Google Calendar with Python and GAS
Create and edit spreadsheets in any folder on Google Drive with python
I tried updating Google Calendar with CSV appointments using Python and Google APIs
A memo with Python2.7 and Python3 on CentOS
Create and decrypt Caesar cipher with python
Deploy a Python app on Google App Engine and integrate it with GitHub
Solving with Ruby and Python AtCoder Tenka1 Programmer Contest C Cumulative sum
Create a Python3 environment with pyenv on Mac and display a NetworkX graph
Try running Google Chrome with Python and Selenium
Install OpenCV 4.0 and Python 3.7 on Windows 10 with Anaconda
Solve AtCoder 167 with python
Notes on HDR and RAW image processing with Python
Install selenium on Mac and try it with python
Automatic follow on Twitter with python and selenium! (RPA)
Solving with Ruby, Perl, Java and Python AtCoder diverta 2019 Programming Contest C String Manipulation
Get comments on youtube Live with [python] and [pytchat]!
PIL with Python on Windows 8 (for Google App Engine)
Ubuntu 20.04 on raspberry pi 4 with OpenCV and use with python
Email hipchat with postfix, fluentd and python on Azure
Automate Chrome with Python and Selenium on your Chromebook
Until you create a machine learning environment with Python on Windows 7 and run it
Create a decent shell and python environment on Windows
Have Google Text-to-Speech create audio data (narration) for video material (with C # and Python samples)
Solving with Ruby and Python AtCoder ARC 059 C Least Squares
Until you create Python Virtualenv on Windows and launch Jupyter
Solving with Ruby and Python AtCoder ABC178 D Dynamic programming
Create a list in Python with all followers on twitter
How to extract any appointment in Google Calendar with Python
Solving with Ruby and Python AtCoder ABC151 D Breadth-first search
Solve with Ruby and Python AtCoder ABC133 D Cumulative sum
Create an LCD (16x2) game with Raspberry Pi and Python
Let's create a PRML diagram with Python, Numpy and matplotlib.
IP spoof using tor on macOS and check with python
Test Python with Miniconda on OS X and Linux with travis-ci
Solving with Ruby and Python AtCoder AISING2020 D Iterative Squares
Solving with Ruby, Perl, Java and Python AtCoder ATC 002 A
Create youtube ad auto skip tool with python and OCR
Solving with Ruby and Python AtCoder ABC011 C Dynamic programming
Solving with Ruby and Python AtCoder ABC153 E Dynamic programming
Solving with Ruby and Python AtCoder ARC067 C Prime Factorization
Solving with Ruby, Perl, Java and Python AtCoder ATC 002 B
Solving with Ruby and Python AtCoder ABC138 D Adjacency list
Notes on deploying pyenv with Homebrew and managing Python versions
Deep Learning with Shogi AI on Mac and Google Colab
Operate limited shared Google Calendar with Lambda (Python) [cloudpack Osaka]
How to log in to AtCoder with Python and submit automatically
Programming with Python and Tkinter
Encryption and decryption with Python
Solve AtCoder ABC166 with python
Light blue with AtCoder @Python
Python and hardware-Using RS232C with Python-
Python on Ruby and angry Ruby on Python
Atcoder Acing Programming Contest Python
Create 3d gif with python3
Study Python with Google Colaboratory
python with pyenv and venv
Access Google Drive with Python
Create a directory with python
Solve AtCoder ABC 186 with Python
Atcoder Beginner Contest 152 Kiroku (python)
Works with Python and R