Skip to main content

KHE

Better Local Python

Kenneth H. East [view page source]

For many years I have managed my local Python installs using a combination of macports, virtualenvs, and pip. I've fought many issues along the way and have finally reached the tipping point. As the TV commercials say, "There has to be a better way!" Whether it is a better way or not remains to be seen, however, I am cutting over to using pyenv for local python installs. These are my notes on using pyenv.

Existing Configuration

Each time I wish to install a python version locally, without affecting the macOS default python, I install a number of macports packages. For example, for Python 3.7, I install python37, py37-pip, py37-virtualenv, py-virtualenvwrapper, and py-gnureadline. Additionally, I set WORKON_HOME=~/.virtualenvs.

To globally set a default python, I run sudo port select for python, pythonXY, pip, and virtualenv. I have a script in my ~/bin that changes all of these at once. New virtualenvs are created via mkvirtualenv newenv or mkvirtualenv --python=3.7 newenv.

Installing pyenv

I use the installer described here.

curl -L https://github.com/pyenv/pyenv-installer/raw/master/bin/pyenv-installer | bash

# Add the following to your environment:
export PATH="/Users/khe/.pyenv/bin:$PATH"
eval "$(pyenv init -)"
eval "$(pyenv virtualenv-init -)"

This will install pyenv in the default location $HOME/.pyenv. It also updates $PS1, which you may or may not want, depending upon your existing virtualenv configuration.

Managing Python Installs with pyenv

pyenv install --list will list all available python versions. I was surprised/pleased to see that, in addition to the standard python from python.org, jython, miniconda, pypy, and others, were available for install. The current dev versions are available as well.

To try it out, I installed 2.7 and 3.7:

pyenv install 2.7.15
pyenv install 3.7.1

pyenv first downloads and installs openssl and readline from pyenv.github.com. Then it downloads, compiles, and installs the specified pythons from python.org. The python versions are installed to $PYENV_HOME/versions.

At this point, if you still have other pythons installed, as I did, pyenv honors whatever your existing default python is. For example, I still had macports 3.7.0 as my default. Because pyenv is at the front of my $PATH, python executes ~/.pyenv/shims/python, which, in turn calls /opt/local/bin/python3.7, the macports 3.7.0.

Your global python can be set either via export PYENV_VERSION=3.7.1 or by pyenv global 3.7.1. The former obviously inspects the environment for the global version. The latter stores the specified version in $PYENV_HOME/version. If set, the environment overrides the contents of the version file.

Once the global python is specified, simply running python at the bash prompt will start the specified python. This is much nicer than the macports mechanism of port select across multiple related macports packages.

If a new version of python is installed, or if a package that installs binaries is installed, run pyenv rehash to refresh all the pyenv shims.

Documentation for all pyenv commands can be found here.

bash, fish, and zsh completions are available in $PYENV_HOME/completions. pyenv --init - automatically enables the appropriate completions.

I ended up adding the following to my .bash_profile:

export PYENV_HOME="$HOME/.pyenv"
export PATH="$PYENV_HOME/bin:$PATH"
export PYENV_VERSION=3.7.1
eval "$(pyenv init -)"

Virtualenvs

pyenv supports virtualenvs via its pyenv-virtualenv plugin, which is installed by default. If you are an existing user of virtualenvwrapper, you may wish to install the pyenv-virtualenvwrapper plugin. This allows one to continue with the previous workflow using workon, mkvirtualenv, etc., just as they did before pyenv.

git clone https://github.com/pyenv/pyenv-virtualenvwrapper.git \
    $(pyenv root)/plugins/pyenv-virtualenvwrapper

I then updated my .bash_profile so that it contained:

export PYENV_HOME="$HOME/.pyenv"
export PATH="$PYENV_HOME/bin:$PATH"
export PYENV_VERSION=3.7.1
eval "$(pyenv init -)"
eval "$(pyenv virtualenv-init -)"
pyenv virtualenvwrapper_lazy

Interestingly, all of my old virtualenvs using macports pythons continued to work properly. Of course, any virtualenvs created after this, used the proper python from pyenv. In all, this is already much less trouble-prone than my previous method.

Farewill macports python

Things were working so well at this point that I was emboldened enough to go ahead and remove all of my macports python installs.