Use per-directory Poetry environments to run Jupyter kernels. No need to install
a Jupyter kernel per Python virtual environment!
The idea behind this project is to allow you to capture the exact state of your
environment. This means you can email your work to your peers, and they’ll have
exactly the same set of packages that you do! Reproducibility!
Why not virtual environments (venvs)?
Virtual environments were (and are) an important advancement to Python’s package
management story, but they have a few shortcomings:
- They are not great for reproducibility. Usually, you’ll create a new virtual
environment using arequirements.txt
which includes all the direct
dependencies (numpy, pandas, etc.), but not transient dependencies (pandas
depends on pytz for timezone support, for example). And usually, even the
direct dependencies are specified only as minimum (or semver) ranges (e.g.,
numpy>=1.21
) which can make it hard or impossible to accurately recreate the
venv
later. - With Jupyter, they usually require that the kernels be installed globally.
This means you’ll need need to have a separate kernelspec for every venv you
want to use with Jupyter.
Poetry uses venvs transparently under the hood by constructing them from the
pyproject.toml
and poetry.lock
files. The poetry.lock
file records the
exact state of dependencies (and transient dependencies) and can be used to more
accurately reproduce the environment.
Additionally, Poetry Kernel means you only have to install one kernelspec. It
then uses the pyproject.toml
file from the directory of the notebook (or any
parent directory) to choose which environment to run the notebook in.
Shameless plug
The reason we created this package was to make sure that the code environments
created for running student code on Pathbird exactly match your development
environment. Interested in developing interactiv