Updating Python using Pyenv and Poetry

This isn't rocket science but is something I do infrequently enough to forget steps.

In code blocks, lines beginning $ are commands and lines without show output. There are a few simple bash commands (e.g. cd).

It's assumed that you already use Pyenv and Poetry and want to change a project's Python version.

If you just want the steps, skip to all the code.

Optional: Store virtual environment in project

By default, Poetry creates virtual environments outside of projects, in Poetry's own cache directory. (Usually something like /home/ellen/.cache/pypoetry/virtualenvs.)

I've started to prefer venvs in their project directories. There's something tidy about having everything in the project directory. If I've messed something up and want to delete the venv, I don't have to remember which directory it was created in. VSCode also notices a venv created in a project and asks if I want to start using it, which means a little less configuration.

One time change:


$ poetry config virtualenvs.in-project true

Install the preferred version of Python

If Pyenv was installed a while ago, there are probably new Python patch releases and maybe even a new minor version. The code uses Python 3.10 as an example. 


$ pyenv update  # Update Pyenv and the list of available Python versions
$ pyenv install --list | grep "3.10"  # Check which versions of 3.10 are available
$ pyenv install 3.10.2  # This will take a while

$ cd a_project_directory
$ pyenv local 3.10.2  # Creates file .python-version, telling Pyenv which version to use when in the directory

Replace virtual environment

The existing venv was created using the old version of Python and should be replaced. It's also worth telling Poetry which version of Python to use because Poetry doesn't always grab the right one. (The behaviour can depend on things like which Poetry installer you used and whether you installed Poetry or Pyenv first. More info in GitHub issues 5077 and 5190.)

The code example assumes the existing virtual environment was created in the project directory. 


$ rm -rf .venv
$ poetry env use $(pyenv which python)
$ poetry install

A note on shims and debugging

Pyenv uses shims to select and run the expected version of Python. In other words, Pyenv intercepts calls related to Python and decides which Python executable to use. That's why you can type python in one directory and expect a Python 3.8.12 shell, while expecting that python in another directory will start a Python 3.10.2 shell. That's incredibly useful but adds a little complexity.

When getting an unexpected version of Python, it can be useful to check both the system's builtin which command and Pyenv's which command. 


$ which python
/home/ellen/.pyenv/shims/python
# defers to Pyenv

$ pyenv which python
/home/ellen/.pyenv/versions/3.10.2/bin/python
# the Python exectuable that will be used

It can also be useful to check the virtual environment's configuration file pyvenv.cfg 


$ cat .venv/pyvenv.cfg
base-executable = /home/ellen/.pyenv/versions/3.10.2/bin/python
# plus other info about the configuration of the venv

If several things are trying to set the version (e.g. environment variable and .python-version file), refer to Pyenv's documentation about how it chooses which version to use.

All the code

Assumes you have a virtual environment in the project directory.


$ pyenv update
$ pyenv install --list | grep "3.10"
$ pyenv install 3.10.2

$ cd a_project_directory
$ pyenv local 3.10.2

$ rm -rf .venv
$ poetry env use $(pyenv which python)
$ poetry install