speed difference between wsgi, Bottle and Flask

It's the season of ISUCON. ISUCON customarily uses a typical micro-framework for each language, but Python has been using Flask for now.

Flask certainly looks like a micro-framework when writing a simple sample app. However, structurally there are many hooks and signals, and it is a heavyweight design.

The combined Flask body and Werkzeug are tens of thousands of lines in size. Even a mere Hello World app has dozens of function calls behind the scenes.

Like Flask, Bottle is a framework that is multi-threaded, has a context stack that uses thread locals, and has extensions, but its structure is much simpler than Flask. The source code is 3000 lines per file, and the overhead of the framework is about half that of Flask.

I made a small measurement with the Hello app. Benchmark a single-threaded, single-process Hello app on a MacBook Air 2013 Mid (Core i5 1.3GHz) with wrk -t1 -c1. Python is CPython 3.4.1 and the web server is Meinheld (latest version on Github).

wsgi:

$ wrk -t1 -c1 http://localhost:6000/
Running 10s test @ http://localhost:6000/
  1 threads and 1 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   118.89us   46.59us   0.99ms   91.40%
    Req/Sec     7.57k   622.67     9.90k    67.44%
  71708 requests in 10.00s, 11.08MB read
Requests/sec:   7170.95
Transfer/sec:      1.11MB

Bottle:

$ wrk -t1 -c1 http://localhost:6000/
Running 10s test @ http://localhost:6000/
  1 threads and 1 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   221.92us  678.78us  18.72ms   99.81%
    Req/Sec     4.86k   418.08     6.44k    78.08%
  46110 requests in 10.00s, 7.74MB read
Requests/sec:   4611.09
Transfer/sec:    792.53KB

Flask:

$ wrk -t1 -c1 http://localhost:6000/
Running 10s test @ http://localhost:6000/
  1 threads and 1 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   476.59us  134.74us   1.71ms   79.35%
    Req/Sec     2.11k   194.96     3.10k    65.86%
  19967 requests in 10.00s, 3.35MB read
Requests/sec:   1996.71
Transfer/sec:    343.18KB

Looking at the difference in Avg Latency, about 100 μs for Bottle and about 350 μs for Flask are overheads for raw wsgi, which is more than double the total performance.

Bottle also supports Jinja templates, so even if your initial app was Flask, you should be able to switch to Bottle reasonably easily. In cases where you're hitting a lot of paths that just cache the HTML part and stick the data from Memcached back together, the performance of this base part should be a non-negligible difference.

Lastly, I will put the measured source code. Please try it.

import flask
import bottle

app = flask.Flask(__name__)
bottle_app = bottle.app()

@bottle_app.route('/')
@app.route('/')
def index():
    return b"Hello, World"

def wsgi(env, start):
    c = b"Hello, World"
    start("200 OK", [('Content-Type', 'text/plain'), ('Content-Length', str(len(c)))])
    return [c]

#starting method
# Flask:  gunicorn -k meinheld.gmeinheld.MeinheldWorker -b :6000 app:app
# Bottle: gunicorn -k meinheld.gmeinheld.MeinheldWorker -b :6000 app:bottle_app
# wsgi:   gunicorn -k meinheld.gmeinheld.MeinheldWorker -b :6000 app:wsgi