The story of making a university 100 yen breakfast LINE bot with Python

This article is the first day of Tokyo City University Advent Calendar 2019. https://adventar.org/calendars/4282

Introduction

Hello! My name is Ojie (@ 920oj) and I am studying information systems at Tokyo City University. This time, I built an advent calendar of Tokyo City University, so I myself Write "The story of making a LINE bot that informs you of a 100 yen breakfast at university with Python"!

background

Our school has a very profitable system where you can eat breakfast for 100 yen. Furthermore, since I am living alone in Tokyo from a rural area, it is perfect for running out of money.

image.png

This is 100 yen

This menu is posted on the cafeteria website every morning, but this website is a songwriter and you have to go through the app or the school portal to access it.

Therefore, I decided to automatically scrape the cafeteria website every morning and create a LINE Notify that will notify LINE of the 100 yen breakfast of the day.

Writer's environment

Python v3.7.1 pip 19.1.1 Windows 10 v1903

What was made

Tokyo City University Yokohama Campus Student Cafeteria 100 yen Breakfast notification bot https://github.com/920oj/TCU-YC-Breakfast-Notify-Bot

image.png

Like this, you will be notified every day at 7:30 in the morning (only if you have a 100 yen breakfast)!

Think about implementation

First, get the school cafeteria website with Beautiful Soup. Since you need to log in to browse the school cafeteria website, we have established a guideline to throw POST once, authenticate, and then knead using the session ID.

However, if you take a closer look, there is no such thing as authentication, and it seems that you just write the ID and password in plain text in the cookie to authenticate. (Is it okay?)

(Originally, I authenticate once (POST), and I think that the session ID is linked to the fact that the authentication passed, and the subsequent requests pass, but for some reason it works without POST. It's good because it's good) (It's not good originally)

Session relation

For the time being, when you go to the login page, it seems that a "session key" and a "session ID" are given, so it starts from getting this.

def get_sessionid():
    #Initial cookie acquisition process
    r = requests.get('https://livexnet.jp/local/default.asp')
    first_access_cookie = str(r.headers['Set-Cookie'])
    
    # "ASPSESSIONID+Any 8-digit uppercase letter"(24 uppercase letters)Get
    asp_session = str(first_access_cookie[first_access_cookie.find("ASPSESSIONID"):first_access_cookie.find("; secure")])
    asp_session_key = str(asp_session[0:asp_session.find("=")])
    asp_session_id = str(asp_session[asp_session.find("="):].replace('=',''))
    return asp_session_key, asp_session_id

The framework is like ASP.NET. As for ASPSESSIONID, it seems that an 8-digit uppercase letter is added at the end, so get this as well. The return value of this function is asp_session_key and asp_session_id, and two types are returned.

Set Cookies-Scraping

def get_breakfast_info(key,id):
    #Cookies are available (information may change in the future)
    site_cookies = {
        key: id, 
        'KCD': '02320', 
        'company_id': SITE_ID, 
        'company_pw': SITE_PASS, 
        'wrd': 'jp', 
        'dip': '0', 
        'ink': 'a', 
        'bcd': '02320', 
        'val': 'daily'
    }
    
    #Access menu / nutrition table page
    url = 'https://reporting.livexnet.jp/eiyouka/menu.asp?val=daily&bcd=02320&ink=a&col=&str=' + today_data
    r = requests.get(url, cookies=site_cookies)
    r.encoding = r.apparent_encoding
    
    #HTML parsing
    all_html = r.text.replace('<br>','')
    souped_html = BeautifulSoup(all_html, 'lxml')
    
    try:
        breakfast = souped_html.find('p', class_="img_comment6").string
        return breakfast
    except:
        return False

Use Chrome's developer tools to see what cookies are set.

After checking, prepare a cookie according to it and load it into beautifulsoup, so prepare a dictionary.

The key is the session key obtained earlier, and the id is the session ID. (This is not the name of the variable. Let's make it more descriptive)

As mentioned earlier, it seems that the authentication ID and password are read in plain text with SITE_ID and SITE_PASS (?), So specify it as it is.

(Isn't it meaningless to get this session ID ...? Please let me know if you have any details.)

All you have to do is load it into lxml, which parses HTML, and extract the class element "img_comment6" from it!

image.png

Fly to LINE (LINE Notify)

def post_line(result):
    post_data = 'today(' + today_data + ')100 yen breakfast is' + result + 'is.'
    
    line_api_headers = {"Authorization" : "Bearer "+ LINE_TOKEN}
    line_payload = {"message" :  post_data}
    
    r = requests.post(LINE_API_URL ,headers = line_api_headers ,params=line_payload)
    return r.status_code

All you have to do is skip to LINE Notify. With LINE Notify, you can put the authentication information and message content in the header and throw POST to the API endpoint to send the information to the preset talk.

Write the main process

def main():
    print('Tokyo City University 100 Yen Breakfast Menu Display Program by 920OJ')
    print('today' + today_data + 'is.')
    
    session = get_sessionid()
    session_key = session[0]
    session_id = session[1]
    
    print('Obtained initial credentials.' + session_key + 'Is' + session_id + 'is. Wait for 3 seconds ...')
    sleep(3)
    
    result = get_breakfast_info(session_key,session_id)
    
    if not result:
        print('Information could not be obtained. 100 yen Breakfast may not be available.')
        sys.exit()

    print('Today's 100 yen breakfast is' + result + 'is. Send a notification to LINE.')
    
    post_status = post_line(result)
    if post_status == 200:
        print('The LINE notification was successful. Exit the program.')
    else:
        print('LINE notification failed. The response is' + str(post_status) + 'is. Exit the program.')
    
if __name__ == "__main__":
    main()

After that, I will write the main process in the same way as assembling the function created earlier.

Finally, write ʻif name == "main": `to prevent the process from being executed arbitrarily if this program is imported somewhere. For the first time, I learned that the main () function is called when the names of the files being executed match.

operate

I host it on my borrowed Lightsail (VPS) and run it regularly with cron. As soon as I run it, I go to get breakfast for the day, so I run cron every morning at 7:30 to realize regular execution.

Finally

To tell the truth, this code was made in April, so when I look at it now, there are places where the variable names are not correct and the implementation is ambiguous. I would like to rewrite the code when I have a little more free time.


Tomorrow is an article by Mr. K (@ ke_odakyu9000)! Nice to meet you! https://adventar.org/calendars/4282

Recommended Posts

The story of making a university 100 yen breakfast LINE bot with Python
The story of making a question box bot with discord.py
The story of making a standard driver for db with python.
The story of making a module that skips mail with python
The story of making a Line Bot that tells us the schedule of competitive programming
The story of implementing the popular Facebook Messenger Bot with python
The story of making Python an exe
The story of blackjack A processing (python)
2016 The University of Tokyo Mathematics Solved with Python
Create a LINE BOT with Minette for Python
The story of making a lie news generator
The story of making a sound camera with Touch Designer and ReSpeaker
The story of making a mel icon generator
The story of creating a bot that displays active members in a specific channel of slack with python
The story of making a tool to load an image with Python ⇒ save it as another name
The story of making a web application that records extensive reading with Django
A story about making 3D space recognition with Python
The story of making a music generation neural network
A story about making Hanon-like sheet music with Python
I made a LINE BOT with Python and Heroku
[Super easy] Let's make a LINE BOT with Python.
A story about an amateur making a breakout with python (kivy) ②
Create a Twitter BOT with the GoogleAppEngine SDK for Python
The story of rubyist struggling with python :: Dict data with pycall
A story about an amateur making a breakout with python (kivy) ①
[Python] The first step to making a game with Pyxel
[Python] A story about making a LINE Bot with a practical manned function on its own without using Salesforce [Messaging API]
The story of Python and the story of NaN
The story of writing a program
A story stuck with the installation of the machine learning library JAX
I want to know the weather with LINE bot feat.Heroku + Python
Save the result of the life game as a gif with python
March 14th is Pi Day. The story of calculating pi with python
[python, ruby] fetch the contents of a web page with selenium-webdriver
A story that struggled to handle the Python package of PocketSphinx
Read the standard output of a subprocess line by line in Python
[LINE Messaging API] Create a BOT that connects with someone with Python
The idea of feeding the config file with a python file instead of yaml
Create a compatibility judgment program with the random module of python.
Check the existence of the file with python
Search the maze with the python A * algorithm
The story of making an immutable mold
[Python] [LINE Bot] Create a parrot return LINE Bot
The story of manipulating python global variables
[python] [meta] Is the type of python a type?
Let's make a Twitter Bot with Python!
A story that visualizes the present of Qiita with Qiita API + Elasticsearch + Kibana
The story of a Parking Sensor in 10 minutes with GrovePi + Starter Kit
[AtCoder explanation] Control the A, B, C problems of ABC182 with Python!
Calculate the shortest route of a graph with Dijkstra's algorithm and Python
A story about a beginner making a VTuber notification bot from scratch in Python
Python beginners decided to make a LINE bot with Flask (Flask rough commentary)
Get the number of searches with a regular expression. SeleniumBasic VBA Python
The story of having a hard time introducing OpenCV with M1 MAC
[AtCoder explanation] Control the A, B, C problems of ABC186 with Python!
[Introduction to Python] How to sort the contents of a list efficiently with list sort
[AtCoder explanation] Control the A, B, C problems of ABC185 with Python!
Calculate the probability of being a squid coin with Bayes' theorem [python]
Hit a method of a class instance with the Python Bottle Web API
Receive a list of the results of parallel processing in Python with starmap
The result of making the first thing that works with Python (image recognition)