How to avoid version conflicts in lock files compiled with pip-tools

Many projects I work on maintain two requirements lock files:

• requirements.txt for remote, production-like environments;
• requirements.dev.txt for development and testing environments;

compiled using pip-compile from pip-tools.

The development requirements should always be a superset of the production ones but it’s possible for them to fall out of sync if you’re not careful in updating both lock files at the same time.

.in files

If you declare packages in requirements.in and requirements.dev.in files then starting requirements.dev.in with:

-r requirements.txt

will avoid conflicts when compiling.

pyproject.toml

If you declare packages in a pyproject.toml file then it’s best to install both lock files in development environments:

pip-sync requirements.txt requirements.dev.txt

If there’s a conflict between the two lock files then pip-sync will error.

pip-sync doesn’t support installing packages in editable directly - you can’t run:

pip-sync requirements.txt requirements.dev.txt -e .

but you can work around this by creating a requirements.local.txt file with contents:

-e .

and then install your development packages with:

pip-sync requirements.txt requirements.dev.txt requirements.local.txt

to ensure all packages are installed in one invocation and any conflicts are brought to light.

Hat-tip to Hynek Schlawack for a useful post on this topic.