Switching from nose to py.test at Mozilla

Switching to py.test


  • What is py.test
  • Why py.test instead of unittest/nose
  • How to switch to py.test


  • Test: piece of code written to assert that another piece of code is behaving as expected.
  • Test runner: gathers tests, runs them, reports success or failures.
  • Test suite, build: (full) collection of tests to run.
  • CI, Continuous Integration: running the tests automatically on each change.
  • Fixture: can be a data fixture (django) or dependency injection (py.test)


What is py.test

By Holger Krekel, who also wrote (some parts of) Tox, pypy, devpi

Equivalent to unittest and nose, highly customizable, with an excellent API.

Still actively maintained, and well documented.

Installation: pip install pytest


Test writing with no boilerplate:

  • With py.test: assert 1 + 1 == 2
  • With unittest: self.assertEqual(1 + 1, 2)

py.test can run your current code: unittest, nose, doctest, autres


  • plugin system and advanced customization
  • 10 slowest tests: --duration=10
  • Stop on first fail: -x (ou --maxfail=n)
  • Drop to pdb on failure: --pdb
  • Post to pastebin: --pastebin=failed
  • Modify tracebacks: --showlocals --tb=long
  • Output capture by default: only display output on failures


  • Mark tests: @pytest.mark.functional
  • Only run selected tests
    by name: py.test -k test_foo
    by path: py.test tests/bar/
    by module: py.test test_baz.py
    by mark: py.test -m functional ou py.test -m "not functional"

awesome failure reports:

        def test_eq_dict(self):
        >       assert {'a': 0, 'b': 1, 'c': 0} == {'a': 0, 'b': 2, 'd': 0}
        E       assert {'a': 0, 'b': 1, 'c': 0} == {'a': 0, 'b': 2, 'd': 0}
        E         Omitting 1 identical items, use -v to show
        E         Differing items:
        E         {'b': 1} != {'b': 2}
        E         Left contains more items:
        E         {'c': 0}
        E         Right contains more items:
        E         {'d': 0}
        E         Use -v to get the full diff

Dependency injection

Depencendy injection using fixtures, instead of (setUp|tearDown)(function|class|module|package)

        def test_view(rf, settings):
            settings.LANGUAGE_CODE = 'fr'
            request = rf.get('/')
            response = home_view(request)
            assert response.status_code == 302


Write the test once, run it with multiple inputs

        @pytest.mark.parametrize(("input", "expected"),
                                 [("3+5", 8), ("2+4", 6)])
        def test_eval(input, expected):
            assert eval(input) == expected

Configuration: pytest.ini

Globally, in the pytest.ini file: mainly to (optionnally) declare marks and default parameters

        addopts = --reuse-db --tb=native
        markers =
            es_tests: mark a test as an elasticsearch test.

Configuration: conftest.py

Locally, for the current folder and children: declare fixtures, plugins, hooks

        import pytest, os
        def pytest_configure():
            # Shortcut to using DJANGO_SETTINGS_MODULE=... py.test
            os.environ['DJANGO_SETTINGS_MODULE'] = 'settings_test'
            # There's a lot of python path hackery done in our manage.py
            import manage  # noqa


There's many included plugins, but also community plugins:



pytest-django: reuse DB

  • Reuse the DB: --reuse-db
  • Force the creation of a new DB: --create-db
  • Set it by default in the pytest.ini file
        addopts = --reuse-db

Explicitely give access to the DB

        def test_with_db(self):  # function/method
        class TestFoo(TestCase):  # class
        pytestmark = pytest.mark.django_db  # module

Always give access to the DB

in the conftest.py file

        def pytest_collection_modifyitems(items):
            for item in items:
                item.keywords['django_db'] = pytest.mark.django_db


py.test is pythonic

Use simple asserts

No camelCase, eg self.assertEqual

  • assert 1 == 2 vs self.assertEqual(1, 2)
  • assert True vs self.assertTrue(True)
  • assert not None vs self.assertIsNotNone(None)

py.test is actively maintained

Ok, but why change?

Why not take advantage of something that

  • is better maintained
  • is compatible with the current tests
  • will be better for all the future tests

Naming is hard

nose vs python + test

Why not change?

No dependency injection (and not parametrization) for nose classes

But we can still use the autouse fixtures (fixtures on steroids)

No (data) fixture bundling ( django-nose feature), so slower test runs, but Django 1.8 test case data setup and possibility to run Travis builds in parallel


Test files

find . -name test* | wc -l



ag "class Test" | wc -l



ag "def test_" | wc -l



ag "self.assert|ok_|eq_" | wc -l



py.test runs your current tests.

the proof

Out of those, is for a new data fixture (code cleanup), unrelated to the switch.

Running tests with py.test



REUSE_DB=1 python manage.py test --noinput --logging-clear-handlers --with-id

Give access to the DB

Added @pytest.mark.django_db on our base TestCase

Added pytestmark = pytest.mark.django_db in 35 files that needed it

        def pytest_collection_modifyitems(items):
            for item in items:
                item.keywords['django_db'] = pytest.mark.django_db


Missing calls to super() in the (setUp|tearDown)(class), leading to code/data leaks in the following tests.
Ugly PYTHONPATH hacks in the manage.py file.

        import manage.py
        @pytest.fixture(autouse=True, scope='session')
        def _load_testapp():
            installed_apps = getattr(settings, 'INSTALLED_APPS')
            setattr(settings, 'INSTALLED_APPS', installed_apps + extra_apps)
            django.db.models.loading.cache.loaded = False


tox randomizes the PYTHONHASHSEED by default on each run. And suddenly you realize your tests were expecting the dicts to be ordered! Need to fix tests:

  • dicts
  • urls built with parameters (that are stored in dicts)
  • Dump or load of json
  • Elasticsearch and its queries
  • csv.DictReader

Bonus: parallel builds

With py.test, tox and travis, it's a piece of cake

split builds

From more than 50 minutes down to less than 20


pip install pytest && py.test

learn py.test: http://www.merlinux.eu/~hpk/testing.pdf

slides: http://mathieu.agopian.info/presentations/2015_06_djangocon_europe/