Use Numpy, Scipy, scikit-learn on Heroku

When you write an application that incorporates machine learning in Python, it often depends on scikit-learn, or even Numpy or Scipy.

These depend on various libraries, so deploying on Heroku is not straightforward. If you can set up your own server on AWS etc., you can set it up there, but it costs money and I want to do it on Heroku! I think that there are cases, so I will introduce a method for that.

Docker Container

Deployment using Docker Container is now possible on Heroku, so use this rather than editing the buildpack in a hurry Is more convenient. Below is an official sample, so please refer to it.

heroku-examples/python-miniconda

Now you won't have to install anything with apt-get or complain that your slag size is too big!

Conda buildpack Libraries such as Numpy and Scipy are usually quite difficult to install from pip, that is, to build from source code. There is an execution environment called Miniconda that can avoid this, that is, you can install compiled binaries, but a build pack that can be used on Heroku is provided. I am.

conda-buildpack

There are two ways to set up a buildpack on Heroku: specify it when you create it, or change it later.

** Specified at creation **

heroku create --buildpack https://github.com/kennethreitz/conda-buildpack.git

** Specify later **

heroku config:add BUILDPACK_URL=https://github.com/kennethreitz/conda-buildpack.git

To use it, just prepare conda-requirements.txt (requirements.txt in conda) for installation with conda. Those that are not provided by conda can also be installed using pip, so if you need to install with pip, you can still write them in requirements.txt.

The difficulty is that the environment created by conda conflicts with virtualenv and cannot be used together. Therefore, if you are looking to deploy on Heroku, you need to use conda to create the virtual environment. Please refer to here for the simple usage of the conda command.

Set the build pack on Heroku and prepare the configuration file for conda. Now you can run applications with machine learning libraries on Heroku.

I made a repository that I actually tried using conda-buildpack, so please refer to it. Since it has a Heroku Button, it can be deployed as it is.

number_recognizer

Heroku buildpack This is a build pack that I created because I didn't know the existence of conda-buildpack and deploys it based on pip as usual.

However, compiling scipy takes so long that it times out on Heroku (make sure you can build Numpy). The only way to overcome this is to use a better Dyno, so I think it's difficult with the free tier.

Heroku buildpack: Python with Numpy, Scipy, scikit-learn

It's not just compatible with Numpy and Scipy, it's designed to maintain its compilation environment, so it should work universally for things that depend on similar libraries (those that). I think it is provided by conda ...). If pip didn't work and you gave up when deploying Heroku, it might work for that too.

You can use heroku-buildpack-apt that can install the library with apt-get in the Heroku environment used here, or connect multiple build packs. heroku-buildpack-multi may be useful in other scenes as it may be useful to remember. heroku-buildpack-multi can be used when using bower etc. (process bower with the build pack for Node.js, and then run the process with the build pack for the main body).

Recommended Posts

Use Numpy, Scipy, scikit-learn on Heroku
Use Numpy, Scipy, scikit-learn on Amazon Linux
Use OpenBLAS with numpy, scipy
python + django + scikit-learn + mecab (1) on heroku
python + django + scikit-learn + mecab (2) on heroku
Use Numpy
Install CVXOPT, NumPy, SciPy on Travis CI
Use multithreaded BLAS / LAPACK with numpy / scipy
Redis on Heroku
If you want to use NumPy, Pandas, Matplotlib, IPython, SciPy on Windows
shimehari on heroku
Make SciPy, scikit-learn available on M1 chip Macbooks
Install and run Python3.5 + NumPy + SciPy on Windows 10
Until you use PhantomJS with Python on Heroku
Use pyvenv on Windows
Install numpy on Marvericks
How to use numpy
Use Ansible on Windows
Use QuTiP on Windows
Use pip on Windows
Use matplotlib on Ubuntu 12 & Python
Use music21 on Google Colaboratory
Use Github Desktop on Linux
Deploy masonite app on Heroku 2020
My reverse numpy / scipy memo
Use matplot libwidget on mac
Use sshpass on Amazon linux2
Use Python on Windows (PyCharm)
Use NeoPixel on Raspberry Pi
Use Linux on Windows 10 (WSL2)
Installation from python, numpy, scipy sources without root privileges on Linux