On , I learnt ...

How the new argument to mock.patch is shared between parameterized tests

Consider this contrived test:

from unittest import mock
import pytest

def get_config():
    """Return a config object with a 'salutation' attribute."""
    pass

def say_hello(name):
    """Return a greeting."""
    return get_config().salutation + " " + name

@mock.patch(
    __name__ + ".get_config",
    new=mock.Mock(return_value=mock.Mock(salutation="Hello"))
)
@pytest.mark.parametrize("name", ("Alan", "Barry", "Calum"))
def test_say_hello(name):
    assert say_hello(name) == f"Hello {name}"

We are testing the say_hello function three times with different arguments and use mock.patch to stub the response of the get_config function, passing in the replacement version at compile/collection time using the new parameter.

All three tests pass:

test_patch_state.py::test_say_hello[Alan] PASSED
test_patch_state.py::test_say_hello[Barry] PASSED
test_patch_state.py::test_say_hello[Calum] PASSED

Test pollution via shared state

But there is potential test pollution as the same mock.Mock instance is used to replace the get_config function in each parameterised test.

To see this, consider mutating the get_config mock in the test body:

@mock.patch(
    __name__ + ".get_config", new=mock.Mock(return_value=mock.Mock(salutation="Hello"))
)
@pytest.mark.parametrize("name", ("Alan", "Barry", "Calum"))
def test_say_hello(name):
    assert say_hello(name) == f"Hello {name}"

    # Mutate the `get_config` return value.
    get_config.return_value.salutation = "Hi"

Now we see two failures:

test_patch_state.py::test_say_hello_1[Alan] PASSED
test_patch_state.py::test_say_hello_1[Barry] FAILED
test_patch_state.py::test_say_hello_1[Calum] FAILED

...
E AssertionError: assert 'Hi Barry' == 'Hello Barry'

...
E AssertionError: assert 'Hi Calum' == 'Hello Calum'

The latter two tests fail as the change to the return value of get_config in the first test pollutes the second and third tests.

This is a potential cause of flakey tests.