Advanced Usage#
Automatic Testing#
Use Tox as the runner#
Tox is a great tool for testing against multiple Python versions or dependency sets.
You can configure a tox.ini like the following to integrate your testing with PDM:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 | |
To use the virtualenv created by Tox, you should make sure you have set pdm config python.use_venv true. PDM then will install
dependencies from pdm lock into the virtualenv. In the dedicated venv you can directly run tools by pytest tests/ instead
of pdm run pytest tests/.
You should also make sure you don't run pdm add/pdm remove/pdm update/pdm lock in the test commands, otherwise the pdm lock
file will be modified unexpectedly. Additional dependencies can be supplied with the deps config. Besides, isolated_build and passenv
config should be set as the above example to make PDM work properly.
To get rid of these constraints, there is a Tox plugin tox-pdm which can ease the usage. You can install it by
1 | |
Or,
1 | |
And you can make the tox.ini much tidier as following, :
1 2 3 4 5 6 7 8 9 10 11 12 | |
See the project's README for a detailed guidance.
Use Nox as the runner#
Nox is another great tool for automated testing. Unlike tox, Nox uses a standard Python file for configuration.
It is much easier to use PDM in Nox, here is an example of noxfile.py:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 | |
Note that PDM_IGNORE_SAVED_PYTHON should be set so that PDM can pick up the Python in the virtualenv correctly. Also make sure pdm is available in the PATH.
Before running nox, you should also ensure configuration item python.use_venv is true to enable venv reusing.
About PEP 582 __pypackages__ directory#
By default, if you run tools by pdm run, __pypackages__ will be seen by the program and all subprocesses created by it. This means virtual environments created by those tools are also aware of the packages inside __pypackages__, which result in unexpected behavior in some cases.
For nox, you can avoid this by adding a line in noxfile.py:
1 | |
For tox, PYTHONPATH will not be passed to the test sessions so this isn't going to be a problem. Moreover, it is recommended to make nox and tox live in their own pipx environments so you don't need to install for every project. In this case, PEP 582 packages will not be a problem either.
Use PDM in Continuous Integration#
Only one thing to keep in mind -- PDM itself requires Python 3.10 or newer, so if your project is to be tested on older Python versions, you have to make sure PDM is installed on a supported Python version, which can be different from the target Python version the particular job/task is run on.
Fortunately, if you are using GitHub Action, there is pdm-project/setup-pdm to make this process easier. Here is an example workflow of GitHub Actions, while you can adapt it for other CI platforms.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 | |
TIPS
For GitHub Action users, there is a known compatibility issue on Ubuntu virtual environment.
If PDM parallel install is failed on that machine you should either set install.parallel to false or set env LD_PRELOAD=/lib/x86_64-linux-gnu/libgcc_s.so.1.
It is already handled by the pdm-project/setup-pdm action.
Note
If your CI scripts run without a proper user set, you might get permission errors when PDM tries to create its cache directory. To work around this, you can set the HOME environment variable yourself, to a writable directory, for example:
1 | |
Caching in CI#
PDM doesn't run pip under the hood, so caching pip's cache directory has no effect. To speed up installs on other CI platforms,
cache these instead:
- The PDM cache directory, shown by
pdm config cache_dir. It holds the HTTP cache, package metadata, the wheels built from source distributions and the package hashes, which saves downloading and building them again in later jobs. Its location can be changed with thePDM_CACHE_DIRenvironment variable, which is useful when the CI platform only caches paths inside the project directory. - Optionally, the project's virtualenv (
.venvby default). It must be keyed on the operating system, the architecture, the Python version and the content ofpdm.lock, otherwise a stale environment may be restored.
To install from the committed lock file in CI, use pdm install --check, which fails if pdm.lock is missing or outdated, or
pdm sync, which installs the locked packages without resolving or updating the lock file. Setting PDM_CHECK_UPDATE=false
also skips PDM's own update check.
Here is an example for GitLab CI:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 | |
Use PDM in a multi-stage Dockerfile#
It is possible to use PDM in a multi-stage Dockerfile to first install the project and dependencies into __pypackages__
and then copy this folder into the final stage, adding it to PYTHONPATH.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 | |
Use PDM to manage a monorepo#
With PDM, you can have multiple sub-packages within a single project, each with its own pyproject.toml file. And you can create only one pdm.lock file to lock all dependencies. The sub-packages can have each other as their dependencies. To achieve this, follow these steps:
project/pyproject.toml:
1 2 3 4 5 6 | |
packages/foo-cli/pyproject.toml:
1 2 | |
packages/foo-app/pyproject.toml:
1 2 | |
Now, run pdm install in the project root, and you will get a pdm.lock with all dependencies locked. All sub-packages will be installed in editable mode.
Look at the đ Example repository for more details.
Hooks for pre-commit#
pre-commit is a powerful framework for managing git hooks in a centralized fashion. PDM already uses pre-commit hooks for its internal QA checks. PDM exposes also several hooks that can be run locally or in CI pipelines.
Export requirements.txt#
This hook wraps the command pdm export along with any valid argument. It can be used as a hook (e.g., for CI) to ensure that you are going to check in the codebase a requirements.txt, which reflects the actual content of pdm lock.
1 2 3 4 5 6 7 8 | |
Check pdm.lock is up to date with pyproject.toml#
This hook wraps the command pdm lock --check along with any valid argument. It can be used as a hook (e.g., for CI) to ensure that whenever pyproject.toml has a dependency added/changed/removed, that pdm.lock is also up to date.
1 2 3 4 | |
Sync current working set with pdm.lock#
This hook wraps the command pdm sync along with any valid argument. It can be used as a hook to ensure that your current working set is synced with pdm.lock whenever you checkout or merge a branch. Add keyring to additional_dependencies if you want to use your systems credential store.
1 2 3 4 5 6 | |