python virtual environments - absolute basics

Python’s virtual environments are a way to avoid conflicting dependencies.

For example - imagine you have two Python projects that both use pytest to provide testing. One, however, was built before and is using v3.x and one is using the latest and greatest (as of today 5.x).

Virtual environments allow us to have both installed so that we can run each project as it was written without having keep all of our project dependencies in sync (a nightmare scenario if I’ve ever heard of one).

Let’s see how that can work.

Below is intended to demonstrate the absolute basics - see the official documentation for more information or look at related solutions like pipenv and poetry for alternatives.

Step 0: Project setup

First, make sure that you’re in the project:

$ pwd
/Users/stephen/code/my-new-project

If I look at my project directory, it’s pretty bare bones right now:¹

$ tree
.
├── app.py
├── requirements-dev.txt
└── requirements.txt

If we look at the requirements.txt, we’ll see a few production dependencies:

$ cat requirements.txt
pg8000
pyyaml

And requirements-dev:

$ cat requirements-dev.txt
flake8
mypy
pylint
pytest
pytest-pylint
coverage

Great - so we know we need to install some packages, but how do we scope them?

Step 1: Create the virtual environment

Before we can use the virtual environment for its sandboxing, we need to create it:

python -m venv my-venv

This should create a new directory in your project with a few sub-directories, including a new site-packages (Python’s equivalent to a node_modules for the node world):

$ tree -d -L 4
.
└── my-env
    ├── bin
    ├── include
    └── lib
        └── python3.8
            └── site-packages

Step 2: Activate the virtual environment

Now that the virtual environment’s present, we need to activate it.

source ./my-env/bin/activate

This is a shell script that sets everything up (you can read its contents with cat ./my-env/bin/activate).

You’ll know that it worked when you see the virtual environment name (my-env) in parentheses to the left of your shell line:

(my-env) $

Step 3: Managing packages with pip

At this point we can install various packages using pip. In our case, however, we have a requirements.txt and requirements-dev.txt. That being the case, we can read those in and install from them directly:

(my-env) $ pip install -r requirements.txt requirements-dev.txt

Step 4: Exiting the virtual environment

One of the methods that’s included in the activate script is a deactivate. This is what will tear down the virtual environment when you’re done.

To use it, all you need to do is run the command:

(my-env) $ deactivate
$

When the (my-env) has disappeared, you’ll know that you’re no longer in the virtual environment.

Wrap Up

With these few commands we can now isolate our Python dependencies - an important step to being able to develop python applications with confidence!

Footnotes

• ¹ I will be using the tree package to print out my directories. It’s one of my favorite little utilities. I wrote about it here.