Brian Okken's pytest: write a test suite the client will dare to run after you leave

When an engagement ends, you leave and the integration code stays. A few months later, an engineer on the client side may open the repo, change one line and run the tests. Whether they dare to merge depends on how readable and how trustworthy the test suite you left behind is.

That is why a book about a testing tool is still worth an FDE’s time. The second edition of Brian Okken’s Python Testing with pytest was published by Pragmatic Bookshelf in February 2022 and runs to 272 pages. For an FDE, the biggest gain from the book is learning to write a test suite that whoever comes next can understand on first reading.

What does the book teach, and how?

The publisher describes it as a complete rewrite covering assert, fixtures, parametrize, markers and plugins, and the book is built around a small database application.

Okken presents it as the “gentlest and most complete” tutorial on pytest, written so readers learn step by step. In his words, the second edition grew from 7 to 17 chapters. Parametrize gets a chapter of its own, while coverage, mocks and CI are treated in more depth.

The table of contents includes a dedicated chapter on mocks, covering mocking attributes, classes and methods, along with supporting plugins. Another chapter covers tox and CI, including running tox with GitHub Actions.

What Okken sets out to show is that, thanks to fixtures, pytest works for small tests and complex functional tests alike. For an FDE, that means there is no need for separate toolkits for unit tests and integration tests.

Idea one: broken setup is not a broken test

Perhaps the most valuable advice in the book is to move setup code out of the test body and into fixtures, so that a setup failure can be told apart from a test failure. Imagine you are handing over a connector that syncs orders from a client’s API into a data warehouse.

If each test opens its own connection and creates its own temporary tables, then when the token expires, every test reports FAILED. Your successor goes hunting for a bug in the mapping logic, when that logic is perfectly fine.

Move that work into a fixture and pytest reports ERROR at the setup stage instead of FAILED at the assert. One look at the report tells you whether the problem is the environment or the code. According to the official pytest documentation, with a yield fixture all teardown code goes after the yield:

@pytest.fixture(scope="module")
def staging_db():
    conn = connect_staging()   # setup
    yield conn
    conn.drop_temp_tables()    # teardown
    conn.close()

The fixtures chapter also covers scope and how to share fixtures through conftest.py. At handover, a tidy conftest.py is worth as much as a page of documentation: everything the tests need in order to run sits in one place.

Idea two: one test, lots of real data

The book presents parametrize as a way to run many test cases from a single test. The pytest documentation confirms that @pytest.mark.parametrize lets you declare several sets of arguments right at the test function. The chapter goes beyond the decorator, too: it also covers parametrizing fixtures and the pytest_generate_tests hook.

Apply this to the connector above: client data is rarely clean. Imagine the order date column arriving in three different formats. Instead of writing three test functions, you write one:

@pytest.mark.parametrize("raw_date, expected", [
    ("2024-03-01", date(2024, 3, 1)),
    ("01/03/2024", date(2024, 3, 1)),
    ("1 Mar 2024", date(2024, 3, 1)),
])
def test_parse_order_date(raw_date, expected):
    assert parse_order_date(raw_date) == expected

Pytest reports three separate results, so if only the second format breaks, the report points to exactly that line. Each time a new kind of bad data turns up in the wild, you add a line to the list. By handover, that list has become a record of every kind of bad data you met at this client.

Parametrized fixtures help when the same logic has to run against two backends, say staging and a local copy. You write the test once and let the fixture decide which environment it runs on.

Two traps for newcomers to fixtures

The first trap is turning conftest.py into a dumping ground. A fixture used by only one test file belongs in that file. If conftest.py runs to several hundred lines, whoever takes over has to read all of it to work out which test depends on what, and it stops serving as documentation.

The second trap is choosing the wrong scope. Setting scope="module" on the staging_db fixture above makes sense, because opening a connection is slow. But if a fixture that creates sample data the tests will modify is given an equally wide scope, earlier tests leave data behind for later ones, and the suite turns green or red depending on run order.

There is a simple rule of thumb: anything expensive and read-only (connections, clients) gets a wide scope; anything the tests write to or modify gets function scope and is cleaned up after yield. Skip the clean-up and temporary tables pile up on the client’s staging environment, and it is their team that will have to clear them.

Idea three: tests that don’t run automatically will break sooner or later

The book also shows how to extend pytest with plugins, hook it into a CI system, use it alongside tox, mocks and coverage, and run it with existing unittest tests. That last point sounds minor but is very useful: if the client’s repo already has tests written with unittest, you can still run pytest without rewriting them from scratch.

When you are not allowed to call the client’s production systems during tests, you will need mocks. And tox running on GitHub Actions keeps the suite running every time someone pushes code, even long after you have left the project.

Who should read it, and in what order?

If you already write Python but your tests are still a handful of loose assert functions, read in this order: the early chapters on assert, then fixtures, then parametrize. Save mocks, tox and CI for when you have a real repo to hand over.

If you are already fluent in pytest, you can go straight to fixture scope, conftest.py and pytest_generate_tests. That is what separates someone who merely uses pytest from someone who can build a whole test suite.

Career-wise, “wrote tests” on a CV impresses nobody. A line such as “handed over a connector with a pytest suite, including fixtures for the staging environment, run automatically with tox on GitHub Actions” is different: it shows you think about the people who come after you.

If the FDE job descriptions you are targeting mention handover or maintaining code for clients, have this example ready to tell in interviews.

The book came out in 2022, but what it teaches has not aged. A good test suite gives the client’s team the confidence to change the code after you have gone.