I usually develop in Ruby, but as the number of big data analysis jobs has increased, it became necessary to play with python, so first of all, a system with a simple API-like role with a lightweight framework bottle I decided to make it.
It's insanely lightweight and easy, so you can use only the good parts of python and use the main language.
① Hit the API from the ruby application (2) Get the data in MySQL with the application written in python ③ Calculate with python ④ Return the calculation result to the ruby application
pip install virtualenv
virtualenv bottle_sample
cd bottle_sample
source bin/activate
pip install bottle
score_calculator.py
# -*- coding: utf-8 -*-
from bottle import route, run
import networkx as nx
db_name = {DATABASE name}
host = {DATABASE host}
username = {MySQL username}
passwd = {MySQL password}
@route('/update')
def update
g = nx.DiGraph()
import mysql.connector
connect = mysql.connector.connect(db=db_name, host=host, port=3306, user=username, passwd=passwd)
cur=connect.cursor()
cur.execute("select from_user_id,to_user_id from awesome_graph_data)
rows = cur.fetchall()
for row in rows:
if row[0] is not None:
g.add_node(row[0])
g.add_node(row[1])
g.add_edge(row[0],row[1])
cur.close()
connect.close()
pr=nx.pagerank(g,alpha=0.85,personalization=None, max_iter=500)
for id, score in pr.items():
print id,score #I will omit it this time, but you can save it in mysql here
return str(pr)
Now you can calculate the pagerank from the directed graph and save it in mysql.
score_calculator.py
@route('/score/<user_id>')
def score
import mysql.connector
connect = mysql.connector.connect(db=db_name, host=host, port=3306, user=username, passwd=passwd)
cur=connect.cursor()
cur.execute("select score from scoda_data where user_id = {{user_id}}")
score = cur.fetchall()
return score
Now you can get the score of a specific user_id.
A very simple application can be easily created with python + bottle. You can also easily create a view file to create a web supplement.
I thought it would be interesting to go out with various calculations that didn't go well with ruby.
Recommended Posts