[Ruby vs Python] Benchmark comparison between Rails and Flask

When introducing Flask, a lightweight web framework for Python, we compared it with Rails with DB access and template inheritance. Since the result was ** Rails is faster **, which is different from the external Web Framework Benchmarks, it is an article to identify the cause and resolve it.

Benchmark results

Since it is supposed to be used by individual PJs, I wrote and compared benchmarks specialized for each application. Results ** Rails ** was found to be ** 1.493 times faster than ** Flask ** **. Rails is really fast.

スクリーンショット 2015-12-14 18.44.24.png

Results different from Web Framework Benchmarks

The result is different from Web Framework Benchmarks. Flask is slow because there seems to be a problem on the code side, so I will verify it.

スクリーンショット 2015-12-15 8.10.35.png

Benchmark conditions

It is a benchmark test specialized for applications.

--DB access to get 1 record and display the result as a template --Be sure to access the DB from the O / R mapper --Templates always inherit the template once using the layout file (extends in Python) --Benchmark is obtained with Apache Bench. Get the average response speed 5 times when accessing 1000 times with a degree of parallelism of 10 --DB is mysql --Invalid because the session function is not used --All environments are built locally

Server start command


#In both cases, only one thread was started by the debug command.

# Rails
rails server

# Flask
python manage.py runserver

Version for each program

Classification Language ver Web framework ver
Flask Python 3.5.0 0.10.1
Rails ruby 2.0.0p451 Rails 4.2.5

Benchmark with Apache Bench

It is a command for measuring the performance of the WEB server that comes standard with Apache, and can be executed with the ** ab ** command (abbreviation for Apache Bench). ʻAb.exe -n -c `

ab command


#Access 1000 times with a degree of parallelism of 10
ab -n 1000 -c 10 http://127.0.0.1:3000/

--Quote: Crisp performance test with Apache Bench

My Flask can't be this slow

In Last Flask article, it responded with ** average 2ms ** without DB access and template inheritance. Template inheritance should be cached to zero cost, and DB access does not require 7ms. Shouldn't it be 1-2ms for DB access even if you estimate a lot? There must be some reason.

After that, as a result of print debugging and investigation, I found that DB connection pooling was disabled in sql alchemy (O / R Mapper) and fixed it.

db.py


#Before correction
engine = create_engine(db_path, encoding='utf-8')
#Revised
engine = create_engine(db_path, encoding='utf-8', pool_size=5)

#* On the bench, I wrote the code to reuse the connection by digging into ThreadLocalStorage.

Benchmark reacquisition result

After re-benchmarking with Flask's DB connection pooling enabled, Flask was the fastest on average request speed. When verifying an unfamiliar web framework, it makes sense to weigh the results of an external benchmark against the results at hand.

スクリーンショット 2015-12-14 17.10.02.png

スクリーンショット 2015-12-14 17.25.19.png

Classification Language ver Web framework ver O/R mapper
Django Python 2.7.5 Django==1.6.4 Django proprietary implementation
Flask Python 3.5.0 0.10.1 SQL Alchemy
Rails ruby 2.0.0p451 Rails 4.2.5 Active Record

What I think of Flask as a salaryman engineer

For me, who writes Python at work, I'm not very attracted to Flask, which is only twice as fast as Django on an execution speed basis. The more people who work in a team, the less likely they are to get rid of their existing tested assets and move on to paying the learning costs together. However, in personal PJ, I want to reduce the server cost by even 1 yen and light my nails, so I will actively use it. Money is important.

Summary

--TechEmpower's Web Framework Benchmarks Competent --Rails is fast enough --Flask, Pyramid users should check if SQL Alchemy DB Connection Pool is enabled --Django users should make sure they have DB Connection Pool enabled. --What you can do with Rails, you can do with Flask. And vice versa

Comparison of me and my bench and Web Framework Benchmarks

I tried to generate a comparison graph. I'll investigate if Django can't be this fast.

スクリーンショット 2015-12-14 16.26.29.png

The world of web frameworks on top

** revel ** in go language is ** 2.41 ** times faster than ** Flask **

スクリーンショット 2015-12-14 15.33.41.png

-Introduction of "fastest" full-stack web framework "revel"

Benchmark code

It's pretty fun to do each tutorial. Generating anything with the Rails rails generate command was hard to understand because it hides what's going on inside.

Rails

book_controller.rb


class BookController < ApplicationController
  def index
    @msg = 'Test message';
    # select
    book = Book.where("id = 1").all
    @book = book[0];
  end
end

rb:index.html.erb


<p>
    <%= @msg %><br/>
    <%= @book.id %><br />
    <%= @book.name %><br />
    <%= @book.description %><br />
</p>

Flask

index.html


{% extends "master.html" %}

{% block title %}index{% endblock %}

{% block body %}
<h1>this is index page</h1><br />
{{ book.id }}
{{ book.title }}
{{ book.publish }}
{% endblock %}

root.py


# -*- coding: utf-8 -*-
from flask import render_template, Blueprint

#The name of the first argument is the template url_Linked with the name when calling in for
from utils.db import get_db_session
from module.book import Book

app = Blueprint("index",
                __name__,
                url_prefix='/<user_url_slug>')


#Url when calling in template_for('index.index')
@app.route("/")
def index():
    book = get_db_session().query(Book).filter(Book.id=1).all()
    return render_template('root/index.html', book=book[0])

django

index.html


{% extends "master.html" %}

{% block content %}
{{ book.id }}
{{ book.name }}
{{ book.description }}
{% endblock %}

views.py


class TestView(TemplateView):
    """
    test
    """
    template_name = 'index.html'
    def get(self, request, *args, **kwargs):
        book = Book.objects.filter(id=1).get()
        response = self.render_to_response({
            'book': book,
        })
        return response

Rails for the first time

I used Rails for the first time, and the feeling of being completely exhausted was wonderful. I didn't find the Rails ecosystem so cozy. In particular, I was relieved that the settings that were output from the two systems for production and development were well thought out and had a dead design.

--No hesitation in setting. From the beginning, the appropriate production.rb and development.rb were generated. --View and associated test code generated with a single command --The asset pipeline automatically builds both css and js ――When you touch it lightly, ActiveRecord is now an ordinary O / R mapper. --Rake command is convenient because difficult parts are put together in one task. ――A lot of Japanese documents will come out if you go around ――We were able to build from scratch and obtain a benchmark in just 3 hours when the above factors overlapped. --High learning cost, worth it

Rails DB connection pooling situation

It seems that you can set the pool value in config / database.yml. However, if it is set to 0, an error will occur and it will not work. It seems that DB connection pooling must be enabled in Rails.

It's a problem that connection pooling can't be disabled in Rails

Enabling DB connection pooling will definitely speed things up, but it has side effects. DB server load. Even if you rent the highest DB instance on AWS, if you make 50,000 connections, the CPU will stick to 100% and fly with load. A DB connection is created for each thread of the server. Assuming 8 CPUs per server, it is common to start an HTTP child process with 8-48 threads, so even if pool = 1 is set, there are 1000 servers including the timeout. And DB will fly with load.

The person in the example company solved it

The person in the example recipe site solved it. I think that solving most problems after a while is one of the great advantages of a web framework with many users and a strong community.

-Disable connection pooling to database with Rails (ActiveRecord)

Your next line is "Sinatra" ... (・ ㅂ ・) و

Recommended Posts

[Ruby vs Python] Benchmark comparison between Rails and Flask
Difference between Ruby and Python split
Differences between Ruby and Python in scope
Encrypt with Ruby (Rails) and decrypt with Python
Differences between Ruby and Python (basic syntax)
Comparison of Python and Ruby (Environment / Grammar / Literal)
Ruby, Python and map
Python and Ruby split
Difference between Ruby and Python in terms of variables
[Ubuntu] [Python] Face detection comparison between dlib and OpenCV
Interprocess communication between Ruby and Python (POSIX message queue)
Comparison of CoffeeScript with JavaScript, Python and Ruby grammar
Python on Ruby and angry Ruby on Python
Install Python and Flask (Windows 10)
Java VS PHP VS Python VS Ruby
Python and ruby slice memo
Connection between flask and sqlite3
Ruby and Python syntax ~ branch ~
Python 3 sorted and comparison functions
Mandelbrot Benchmark (C, PHP, HHVM, Ruby, Python, PyPy, and Kinx)
Summary of Differences Between Ruby on Rails and Django ~ Basics ~
Difference between java and python (memo)
Difference between list () and [] in Python
Difference between == and is in python
Scraping with Node, Ruby and Python
Cooperation between python module and API
Difference in how to write if statement between ruby ​​and python
Differences between Python, stftime and strptime
Difference between python2 series and python3 series dict.keys ()
Speed comparison between CPython and PyPy
[Python] Difference between function and method
Summary of differences between Python and PHP (comparison table of main items)
File open function in Python3 (difference between open and codecs.open and speed comparison)
Python --Difference between exec and eval
[Python] Difference between randrange () and randint ()
[Python] Difference between sorted and sorted (Colaboratory)
msgpack deserialization performance comparison (C ++ / Python / Ruby)
Communicate between Elixir and Python with gRPC
Differences in authenticity between Python and JavaScript
difference between statements (statements) and expressions (expressions) in Python
Eating and comparing programming languages: Python and Ruby
Differences in syntax between Python and Java
Difference between PHP and Python finally and exit
POST variously with Python and receive with Flask
Difference between @classmethod and @staticmethod in Python
Difference between append and + = in Python list
Difference between nonlocal and global in Python
[Basic grammar] Differences between Ruby / Python / PHP
[Python] Difference between class method and static method
[Python3] Switch between Shift_JIS, UTF-8 and ASCII
[Python Iroha] Difference between List and Tuple
[python] Difference between rand and randn output
speed difference between wsgi, Bottle and Flask
Comparison of Python serverless frameworks-Zappa vs Chalice
Differences in multithreading between Python and Jython
Easy web scraping with Python and Ruby
Correspondence between Python built-in functions and Rust
Exchange encrypted data between Python and C #
Relationship between Homebrew + rbenv + ruby-build (Part 2) You can actually introduce ruby ​​and Rails!
Differences in string processing between Python, Ruby, JS, PHP (combination and variable expansion)
Difference in writing method to read external source code between Ruby and Python