Python Virtual Environments & Dependency Management⚓

When developing a python project, you might need specific dependencies (i.e external python libraries). Those dependencies might clash with the ones installed by your Operation System for its normal operations (especially on Linux-based OS). Furthermore, it might clash with dependencies you have for your other python projects or cause side-effects.

To avoid breaking or impacting existing systems, other projects or even the one you are working on, you should isolate their environment as much as possible (i.e each project only has its own dependencies).

Thankfully, python is packaged with a virtual environment tool, along with tools to manage and install those dependencies. And third-party tools also exist, expanding or combining them.

Virtual Environments⚓

A virtual environment in python is an isolated set of python libraries, that is linked to a python interpreter and a specific python project. Depending on the tool used to create and manage them, its creation and setting will vary. Its usage may also become more or less seamless.

The official python tool for virtual environments is the native module venv.

venv⚓

As previously mentioned, this tool is packaged with python since version 3.6. When invoked, it creates a directory containing the definition of the virtual environments, its commands and sources.

Creation⚓

# From your project root directory
python -m venv VENV_FOLDER  # e.g VENV_FOLDER is called venv for clarity

Note

On Debian/Ubuntu, the venv tool might not be installed by default. Then the command will fail with a suggestion on how to install it.

Activation⚓

A virtual environment already exists. To use it, you need to call its activate command.

On Windows:

1	`venv\Scripts\activate`

On Linux/MacOS:

1	`source venv/bin/activate`

You should then see the name of the virtual environment appear in the beginning of your command line.

Note

If you did not give it a name during creation, it will simply use its folder name (e.g venv). If you work on multiple projects with each their own venv, we recommend you to give them a name (e.g your project name) when creating them, to easily differentiate them:

python -m venv venv --prompt VENV_NAME  # replace VENV_NAME with the actual name

From now on, every python command you are using will be called from within this virtual environment. This includes calls to the python interpreter, its utilities, python libraries installed in the venv, etc.

Deactivation⚓

When you are done working on a project, you can simply close the terminal as the virtual environment activation only applied to it. If you want to keep the terminal session opened (e.g to make tests on a different environment), you can simply call its deactivate script.

1	`deactivate`

This will deactivate the virtual environment and set back the system environment.

Removal⚓

Removing a virtual environment is as simple as deleting its folder, i.e the venv folder.

Other tools⚓

There are other third party tools to manage and create virtual environments.

pyenv & virtualenv⚓

This tools manage virtual environments and python versions together in a centralized manner. You need to set it up once (upgrade it from time to time to get the new python versions), then call the pyenv command.

All virtual environments and python interpreter's versions will be located in a single directory generally in your home folder. It has a built-in way to automatically activate a virtual environment when navigating to its directory.

These are actually two tools you need to install from their repository:

pyenv: the tool managing multiple python versions
pyenv-virtualenv: a plugin adding virtual environments to pyenv

poetry⚓

This tool manages project dependencies, and isolate them from other projects. It can also be used to build & package your python project and distribute it.

See Poetry for more information.

uv⚓

This tool is an extremely fast python package and project manager. It can replace all previously mentioned tools, even pip, and has the following non-exhaustive list of features:

Manage project dependencies
Manage python versions
Manage virtual environments
Build and package projects
Distribute projects
Manage project version number

See more on uv.

Python dependencies⚓

A python project might depend on other projects to function, those are called dependencies. These dependencies might be libraries you import and call from within your code, programs you need to execute your code, CLI tools you use during development.

It is important to be conservative in your use of external dependencies, as every dependency add code and content making installation bigger, adding potential bugs and security issues. Though you need to balance this need to include as few dependencies as possible with the time you have on hand, and not always reinvent the wheel. A good mantra is to use "as much dependencies as necessary, as few as possible".

Python comes with a handy tool to install/manage dependencies: pip.

Manage dependencies⚓

pip is a command line interface (CLI), which you can call to manage dependencies.

Note

The exact pip version used depends on the current active environment. This means that once you activated a virtual environment, all calls to pip will manage dependencies of this virtual environment. When not in a virtual environment, the system dependencies will instead be managed. That latter way to manage system dependencies is actively discouraged by python, and you should use your OS package manager to install system requirements instead of pip.

pip comes with a bunch of subcommands.

Note

If you are using a tool with its own commands to manage dependencies (e.g poetry or uv), then you should use the corresponding commands instead of pip.

Warning

Depending on the way pip was installed, it might not be directly available as a command. In that case, you need to prefix all calls to the pip utility with python -m pip instead. For example:

python -m pip install numpy

Install libraries⚓

This command install the numpy library

1	`pip install numpy`

This command will download and install the latest version of numpy found on the Python Package Index (PyPI). Example for numpy. It will also install its dependencies.

Note

You can (and if possible should) control the exact version of a library you want to install. This could be done with the following modified command:

pip install numpy==2.3.2  # install numpy at version 2.3.2

Upgrade library⚓

This command will upgrade the numpy library to the latest version available:

pip install --upgrade numpy

Note

pip is also a python library, and can be upgraded for better performances or compatibility:

pip install --upgrade pip

List installed libraries⚓

This command will list all libraries installed in your current environment and their version:

1	`pip list`

Here is an example output of freshly installed venv, with numpy installed:

Package    Version
---------- -------
numpy      2.2.6
pip        22.0.2
setuptools 59.6.0

Note

Your versions of these libraries may vary, as the latest numpy version is installed, and pip/setuptools are usually the same versions than your system by default, so may differ.

Uninstall a library⚓

You can uninstall any library such as numpy with the following command:

1	`pip uninstall numpy`

Note

This will only remove the selected libraries, their own dependencies won't be removed unless listed in the command.

pip has more commands and usages, visit its documentation for more: pip.

Dependencies Declaration⚓

Your project dependencies must be specified in order for other persons to use or contribute to your project. This ensures they are using the exact same dependencies you use which means they should expect the same behaviour from your code on their machine, avoiding the classic "It works on my machine" trap.

There are multiple ways to declare dependencies, we will show you the most widespread.

Requirements file⚓

This declaration of dependencies consist in listing the complete set of dependencies of your project (library and exact version) inside a requirements file, generally called requirements.txt and located at the root of the project.

An example of such file for a fresh project with only numpy installed might be the following:

1	`numpy==2.2.6`

Using this requirements file makes the synchronization of dependencies rather easy.

You can use the following pip command to install the requirements contained in the file:

pip install -r requirements.txt

You can also use this pip command to override your requirements file with the uptodate installed dependencies list:

pip freeze > requirements.txt

This requirements file should therefore be included in your project whenever you are sharing it.

Pyproject file⚓

Another way to declare project dependencies is to include them in your project metadata files called pyproject.toml.

This method is more advanced and will be covered in the Packaging and Distribution guide. TODO

Types of dependencies⚓

When working on a python project, you may have multiple dependencies required for different usage.

You have your software core dependencies (i.e all the library imported by your code and/or any tool/program needed to execute it), which are necessary for it to function.

You have optional dependencies, which are not necessary to use the core functionalities of your software, but may enhance them (e.g plotting dependencies to unlock plotting utilities).

You may also have utilities necessary to test your code (e.g pytest and/or coverage), or utilities to develop your code, format it correctly, etc. Those are generally called development dependencies, and are needed for potential contributors wanting to work on your package but not for people simply installing and using it.

You should therefore separate those different types of dependencies, so users or contributors can easily install the minimum set of dependencies for their need. With requirements file, you can include multiple requirements file with names indicating their use, and documentation precising it. With pypyoject.toml, you can also do this separation of requirements.

Python Version management⚓

Python is not a fixed language, it constantly evolves, offering new features and ways to write python code. At the time of writing, the latest stable release of python is the 3.13.5 and the 3.14 and 3.15 are respectively in prerelease and development status.

If you keep to the python 3 version, you should not expect any breaking change when upgrading the version of python for your project. However, downgrading it might break your project, if it uses new features introduced after the downgraded python version was released. For example, structural pattern matching was introduced in Python 3.10 and is therefore not present in python<3.10.

At any rate, you should always document the version of python you are using for development. You should also document on what versions of python your code can be executed, and should in principle be tested on every supported different python version (see Python Versions). In general though, as python versions are backward compatible, you may simply need to document the minimal python version necessary to run your project.

Another good practice is to align your python version policy on numpy (see NEP 29). Their policy is widely adopted across the scientific python ecosystem and recommend support for python versions in a time window manner (i.e every python version released on the last 42 months).