Arul's Blog
Note to self

Setting up Python development environment - Pyenv, VirtualEnv, Conda

General, Tips · Published the 14 May 19
739 words · 4 min read

Setting up Python development environment - Pyenv, VirtualEnv, Conda

Python support two major versions right now Python 2 and Python 3. If you are starting new project, it is recommended that you start with Python 3. There are however cases where you want to have both Python 2 and Python 3 in your development machine. Further, different applications using the same Python version might need different set of libraries. In addition, Python is very popular for doing data science and machine learning and there are environments and libraries available for doing this work. In this article, we will see how we can get this all setup in our development environment.

I am using a Mac as my development environment so the instructions are targetted towards this platform

Installing Pyenv

The first we need to install is Pyenv using brew

$ brew install pyenv

Pyenv provides the ability to install different versions of python on our computer.

Once Pyenv is installed, add pyenv init to your shell to enable shims and autocompletion. Please make sure eval “$(pyenv init -)” is placed toward the end of the shell configuration file since it manipulates PATH during the initialization.

$ echo -e 'if command -v pyenv 1>/dev/null 2>&1; then\n  eval "$(pyenv init -)"\nfi' >> ~/.bash_profile

Now we are ready to install different versions of python in our development machine.

You can see the list of version that you can install with the following command

arulmac:~ arulk$ pyenv install --list
Available versions:
  2.1.3
  2.2.3
  2.3.7
  2.4
  2.4.1
  2.4.2
  2.4.3
  2.4.4
  2.4.5
  2.4.6
  2.5
  2.5.1
  ....

Installing specific Python distribution

Next, let’s install some python version. I would like to install latest version of Python 3 and Ananconda3 versions. I can do that using the following commands

arulmac:~ arulk$ pyenv install anaconda3-5.0.0
Downloading Anaconda3-5.0.0-MacOSX-x86_64.sh...
-> https://repo.continuum.io/archive/Anaconda3-5.0.0-MacOSX-x86_64.sh
Installing Anaconda3-5.0.0-MacOSX-x86_64...
Installed Anaconda3-5.0.0-MacOSX-x86_64 to /usr/local/opt/pyenv/versions/anaconda3-5.0.0

In the above command, install parameter tells pyenv that we would like to install a new version and the parameter following that tells pyenv what version we would like to install.

Let’s do the same command but now we will install python 3 version

arulmac:~ arulk$ pyenv install 3.6.3
python-build: use openssl from homebrew
python-build: use readline from homebrew
Downloading Python-3.6.3.tar.xz...
-> https://www.python.org/ftp/python/3.6.3/Python-3.6.3.tar.xz
Installing Python-3.6.3...
python-build: use readline from homebrew
Installed Python-3.6.3 to /usr/local/opt/pyenv/versions/3.6.3

VirtualEnv

virtualenv is a tool to create isolated Python environments.

The basic problem being addressed is one of dependencies and versions, and indirectly permissions. Imagine you have an application that needs version 1 of LibFoo, but another application requires version 2. How can you use both these applications? If you install everything into your platform’s standard location, it’s easy to end up in a situation where you unintentionally upgrade an application that shouldn’t be upgraded.

Or more generally, what if you want to install an application and leave it be? If an application works, any change in its libraries or the versions of those libraries can break the application.

Also, what if you can’t install packages into the global site-packages directory? For instance, on a shared host.

In all these cases, virtualenv can help you. It creates an environment that has its own installation directories, that doesn’t share libraries with other virtualenv environments (and optionally doesn’t access the globally installed libraries either).

You can install virtualenv seperately but since we are using pyenv, we want virtualenv tool to play nicely with pyenv. Luckily there is a pyenv plugin created for virtualenv.

The easy way to install this on mac would be to use brew manager again as follows

arulmac:~ arulk$ brew install pyenv-virtualenv

We need to add pyenv virtualenv-init to our shell to enable auto-activation of virtualenvs.

$ echo 'eval "$(pyenv virtualenv-init -)"' >> ~/.bash_profile

Using pyenv virtualenv with pyenv

To create a virtualenv for the Python version used with pyenv, run pyenv virtualenv, specifying the Python version you want and the name of the virtualenv directory. For example,

$ pyenv virtualenv 3.6.3 my-virtual-env-3.6.3

will create a virtualenv based on Python 3.6.3 under $(pyenv root)/versions in a folder called my-virtual-env-3.6.3

Create virtualenv from current version

If there is only one argument given to pyenv virtualenv, the virtualenv will be created with the given name based on the current pyenv Python version.

$ pyenv version
3.6.3 (set by /Users/arulk/.pyenv/version)
$ pyenv virtualenv venv36

List existing Virtualenvs

you can list all the existing virtualenvs using the following command

``` $pyenv virtualenv

I will add more use cases as I use them.

Arul Kumaravel
Thank you for reading my blog. I write these blogs as note to myself but if you find it useful, please share your comments