Python project structure

Python projects have a typical folder structure, which depends on the type of software project you are engaged in.

That structure organizes every file such project must contain:

• source code
• test files
• documentation
• notebooks
• license information
• project information (requirements, metadata, etc.)

All those files and folders must be contained in your root project folder, usually named as the project name (for example cookie-factory).

When using specific framework (e.g web development frameworks such as django, flask), they might modify or adapt the usual folder structure to their needs. In that case, follow the instructions on how to setup a project on the framework documentation.

Source code

You need to organize your source code, so as to simplify its development and maintenance. To that end, you should strive towards a modular design, i.e cleanly separate your features in different python files (modules). Those file names must be properly formatted so python can execute/import them: they must start with a letter, then only contain letters, numbers and _, and finish with the python file extension .py.

Python package structure

Those modules need to be organized, especially if your project grows and you have more than a single python file. To that end, python allows to form (potentially) deep hierarchy of modules, organized in multiple folders (packages), ans subfolders (subpackages). In order for those subpackages to be usable from other directory (e.g implement some functionalities in a directory, import and use them in another), you need to indicate to python the folder is a package/subpackage by adding a __init__.py file in it. When nesting multiple subpackages together, every folder must contain this file.

This hierarchy of source files can then be contained within a single parent folder. Different layouts exist to setup this single parent source code folder.

Python main script (__main__.py)

Another special python file exists, __main__.py, it is meant to contain the main code of the project (i.e the application main script), it is called by default when executing a python package.

python -m cookie_factory  # will call cookie_factory/__main__.py file if present

Note

This is a rather niche python file, as it is also possible to define entry function/module to be run when executing a package.

Private/public API

There is another naming convention for python files, functions, classes, methods. If they start with a _ character, it means they are private and should not be used directly by a user (though your project might still use it). This allows you to split code between internal private parts, necessary for the project to work correctly but not to be used by the end-user directly, and the public parts, which the end-user can use. This is especially useful when building an API.

Note

Python namespaces (modules, functions, classes, etc) starting with a single _ character can still be imported with import _module or from module import _function lines, this is a naming convention, not a strict encapsulation feature. Though it is a python standard and best practice, broadly shared by the python developers' community.

Layouts

Every python source file, be it module or scripts, must be found somewhere in your project hierarchy. It is more practical to gather the source code in one single folder, but there are multiple variants which we will present.

Src-layout

In that layout, all source code is located in a src folder located at the root of the project. If you are developing source code there that need to be imported elsewhere and/or installable, it should be gathered inside a package folder, directly located in that src folder.

For example, if you are developing a project named cookie-factory, you would need to put your source code inside a folder named, for example, src/cookie_factory. Let's see below an example of such structure:

cookie-factory
├─── src
|    ├─── cookie_factory
|    |    ├─── __init__.py
|    |    ├─── __main__.py
|    |    ├─── cookie.py
|    |    └─── factory.py

Flat-layout

In that layout, all source code is located in a folder named as the package located at the root of the project. If you are developing source code there that need to be imported elsewhere and/or installable, it should be gathered inside a package folder, directly located in that folder.

For example, if you are developing a project named cookie-factory, you would need to put your source code inside a folder named, for example, cookie_factory. Let's see below an example of such structure:

cookie-factory
├─── cookie_factory
|   ├─── __init__.py
|   ├─── __main__.py
|   ├─── cookie.py
|   └─── factory.py

Tests

Your tests should be contained inside a single directory, called generally tests and located in the root of the project (e.g cookie-factory/tests).

We will present the different types of tests in another guide: Testing Your Python Code.

Documentation

The minimal documentation for a project is the readme file. This file is located at the root of the project, and is generally in Markdown format. It is generally called README.md or simply README. For instance cookie-factory/README.md. This file is the entrypoint for your project to anyone wishing to use or contribute to it.

If your documentation is a little more complex, containing multiple pages, it should be contained inside a single directory, called generally docs or doc, located in the root of the project (e.g cookie-factory/docs).

We will talk in more details about how to document a project in another guide.

Notebooks

Your notebooks should be contained inside a single directory, called generally notebooks and located in the root of the project (e.g cookie-factory/notebooks).

We will present notebooks in another guide.

Other files

This guide being already quite long, we will stop there. Other files/folders will be explained in their corresponding guides.

Here is a recap of a typical package structure for a src-layout python package:

cookie-factory
├─── README.md
├─── src
|    ├─── cookie_factory
|    |    ├─── __init__.py
|    |    ├─── __main__.py
|    |    ├─── cookie.py
|    |    └─── factory.py
├─── notebooks
├─── docs
└─── tests