2015 began with the release of td-client-python (more precisely, treasure data from Python Pandas in the 2014 Advent Calendar). Hit](http://qiita.com/kiyoto/items/2d8d49e3a9b34f652e52)?), I think it was a year that could be called the first year of Python for Treasure Data.

I thought I'd write something about td-client-python, but it's not very interesting to write something that can be understood by reading the source, so if you don't understand even if you read the source, pyenv and pyenv- I will take this opportunity to explain some of the implementation of virtualenv. It has nothing to do with Treasure Data, but I also use pyenv for testing td-client-python and it should be acceptable to write it in Advent Caledar ...

What is pyenv

Around the summer of 2012, I suddenly thought about copying the tree of rbenv and tried sed -e's / RBENV / PYENV / g'-e's / RUBY / PYTHON / g'. In short, it's rbenv for Pythonista. Originally, the purpose was to make it easy to switch between multiple CPython versions and PyPy, so it supports installing not only CPython but also various flavors such as PyPy, Jython, Stackless, IronPython, etc. .. Although there are some parts where the specifications have been intentionally changed from rbenv, it should be possible to use it with the same usability as rbenv.

What is pyenv-virtualenv?

Until a few years ago, I think it was common to use virtualenvwrapper to manage multiple virtualenv, but I had a lot of thoughts so I made it possible to manage multiple virtualenv as a version of pyenv. is. When I first started making it, it was a script that just wrapped the virtualenv command, but after that, the automatic activation function of virtualenv was added, and I feel that it is becoming more convenient and fat little by little. I will.

There is also a plugin for managing virtualenv with pyenv called pyenv-virtualenvwrapper, which is also implemented quite hard, but the shell Virtualenvwrapper that works as a function and pyenv that runs with shims script are incompatible, so I wouldn't recommend it unless you are very fond of virtualenvwrapper.

Beginning of Anaconda / Miniconda support in pyenv

About two years ago, it was possible to select anaconda, anaconda3, miniconda, miniconda3 as the version of pyenv. Basically, it was a simple task of downloading and hitting .sh, which is distributed by Continuum Analytics as standard.

Problems with Anaconda / Miniconda support in pyenv

I tried to install Anaconda on pyenv properly, but when I actually installed it, I noticed that there were quite a few problems. For example

Anaconda hijacks curl, ʻopenssl`, etc.

Because Anaconda installs curl, ʻopenssl, sqlite3, redis`, etc. in its own prefix by default.

In the first place, curl is broken and doesn't work on Debian

Does not work unless the CA certificate is in /etc/pki/tls/certs/ca-bundle.crt

The attached ʻactivate` script cannot be used

pyenv hijacks $ PATH in shims scripts, so you can't source the script you expect by doing something like source activate foo

By the way, until now it was possible to install Anaconda with pyenv, but due to the above problem, it was difficult to use it regularly unless Anaconda was set to pyenv global. Personally, I rarely used Anaconda and only used Miniconda3.

About Anaconda / Miniconda support in pyenv

Even though the Python fever is increasing internally, I thought it would be a problem to leave the Anaconda support as it is, so I tried various things to fix it. To give you a rough idea,

Changed conda env to be treated as a version of pyenv like virtualenv (yyuu / pyenv # 471)
For curl, ʻopenssl, sqlite3, etc., change to use the one installed in system` instead of the one included in Anaconda / Miniconda ([yyuu / pyenv # 491]. (https://github.com/yyuu/pyenv/pull/491))
pyenv activate can now activate conda env as well as virtualenv (yyuu / pyenv-virtualenv # 104)
Prepared a ʻactivate script to enable conda env to be activated in a format like source activate foo` (yyuu / pyenv-virtualenv # 108 -virtualenv / pull / 108)))
When creating a virtualenv with pyenv virtualenv, the environment is created not only with virtualenv and pyvenv but also with conda create ([yyuu / pyenv-virtualenv # 125](https: // github) .com / yyuu / pyenv-virtualenv / pull / 125))

And so on, I've decided to do a lot of things ... For the time being, I think that the above-mentioned measures have almost solved the problems of using Anaconda with pyenv mentioned above.

Summary

Setting up Jupyter by _properly _ installing matplotlib etc. on top of CPython is quite difficult (especially when OS X requires --enable-framework etc.), so Anaconda / Miniconda I think it's a step forward to be able to handle it normally from pyenv. We will continue to add features such as Pandas and Jupyter that can withstand use in data analysis applications if necessary.

As of December 18, 2015 (Fri), the changes on the pyenv-virtualenv side have not yet been released to Homebrew etc., so in order to use the version to which the changes have been applied, https: // github. You need to get the latest changes from com / yyuu / pyenv-virtualenv. I would like to release it to Homebrew etc. within the year.

Finally

I feel like I'm going to get out of the number of Stars in fluentd in a little while, so I'm not sure if I'm an employee of Treasure Data, but I'd like to ask for the cooperation of Stars!

fluent/fluentd

yyuu/pyenv

About pyenv's support for Anaconda / Miniconda