Advanced Pytest Patterns: Harnessing the Power of Parametrization and Factory Methods

When working with complex software systems, writing comprehensive and maintainable test suites is crucial. In our previous blog post, we shared how Pytest plays a central role in our testing strategy, helping us deliver quickly and reliably through the use of Pytest fixtures. In this post, we will discuss additional techniques: parametrization and factory patterns.

Let’s explore what these patterns are, how to use them, and the benefit we gain from them.

1. Pytest Parametrization

While Pytest fixtures are often used for setup and teardown processes, they can be enhanced via  parametrization. Parametrization allows a single test function to run with multiple sets of arguments, reducing redundancy and improving readability. By dynamically injecting varying input values, parametrization boosts test reusability, promotes modularity, and streamlines the overall testing process, making code more organized and efficient. This approach is particularly valuable for testing a function’s behavior across various inputs or validating multiple test cases.

Example: Simple Fixture Parametrization

In the following example, we test a mathematical operation using different sets of numbers:

# File - conftest.py

import pytest

@pytest.fixture(params=[(2, 3), (5, 7), (10, 0)])
def numbers(request):
    return request.param

In this case, the numbers fixture is parametrized to return different pairs of numbers ((2, 3), (5, 7), and (10, 0)) for each test run. This approach allows you to test the same functionality with multiple inputs without duplicating test code.

In the example below, the numbers fixture is parametrized with three sets of input values. The test function uses these parameters to validate the behavior of a simple addition function:

# File - math_operations.py

def add_numbers(num1, num2):
    return num1 + num2


# File - test_math_operations.py

def test_addition(numbers):
    num1, num2 = numbers
    result = add_numbers(num1, num2)
    assert result == num1 + num2

By associating the numbers fixture with the test_addition function, Pytest will run the test three times — once for each set of input values provided by the fixture. This ensures the additional logic is tested across multiple cases without redundant test code.

Flexible Testing with Indirect Parametrization in Pytest

Directly passing parameters to fixtures can be limiting, as it may only address a narrow range of test scenarios. Indirect parametrization provides greater flexibility by allowing parameters to be passed to fixtures directly from the test function. This approach simplifies test design while enhancing adaptability, making it easier to define parameters alongside the test function and accommodate a wider variety of testing requirements.

# File - conftest.py

import pytest

@pytest.fixture()
def numbers(request):
    return request.param


# File - test_math_operations.py

@pytest.mark.parametrize("numbers", [(2, 3), (5, 7), (10, 0)], indirect=True)  
def test_addition(numbers):
    num1, num2 = numbers
    result = add_numbers(num1, num2)
    assert result == num1 + num2

In this example, the @pytest.mark.parametrize decorator indirectly passes values to the numbers fixture. Pytest then runs the test_addition function three times, each with a different set of input values, all while keeping the test design clean and adaptable.

Dynamic Parametrization Using Markers

For greater flexibility in parametrization, Pytest provides the pytest.mark.parametrize marker. This allows you to define multiple sets of parameters directly within the test function, making it easier to manage and scale tests.

# File - test_math_operations.py

import pytest

@pytest.mark.parametrize("num1, num2, expected", [(2, 3, 5), (5, 7, 12), (10, 0, 10)])
def test_addition(num1, num2, expected):
    result = add_numbers(num1, num2)
    assert result == expected

This approach improves flexibility and readability, especially when handling larger sets of parameters, as it allows parameters to be declared clearly — right at the test function. It also makes it easier to add or modify test cases as needed.

2. Factory Patterns for Dynamic Test Generation

While parametrization works well for testing a range of predefined input values, factory patterns extend this capability by dynamically generating tests based on specific conditions. This approach is especially useful in complex testing scenarios or when the exact number of test cases is unknown ahead of time, ensuring more flexible and comprehensive test coverage.

Simple Factory Pattern for User Model Creation

Let's explore a scenario where we want to test the creation of a User model using a factory pattern. In this example, we have a simple User model with three attributes: username, email, and active.

# File - model.py

from dataclasses import dataclass

@dataclass
class User:
    username: str
    email: str
    active: bool = False

Now, let's create a factory to dynamically generate User instances for testing. The user_factory fixture serves as a simple factory, allowing us to dynamically create User objects with different attributes in our tests.

# File - conftest.py

import pytest

@pytest.fixture
def user_factory():
    def _create_user(username, email, active):
        return User(username, email, active)
    return _create_user

Here’s how you can use this factory in a test case:

# File - test_user.py

import pytest

def test_user_creation(user_factory):
    user = user_factory(username='user1', email='user1@fiddler.ai', active=True)
    assert isintance(user, User)

In this test, the user_factory fixture dynamically creates a User object, and the test verifies that the created object is an instance of the User class. Acting as a simple factory, the user_factory fixture allows us to generate user objects with different configurations, reducing redundancy and making the tests more flexible while also abstracting aspects that are not core to the test itself (like user creation in this case).

By using the factory pattern in combination with Pytest fixtures, you can easily create multiple variations of objects for your tests, ensuring flexibility, maintainability, and better coverage across different scenarios.

A More Complex Factory Pattern

For more complex scenarios where the creation of test inputs involves intricate logic, external libraries like factory_boy provide a flexible and expressive way to define factories for generating complex objects. Here's an example:

# File - conftest.py

import pytest
from factory import Factory, LazyAttribute, Sequence

class UserFactory(Factory):
    class Meta:
        model = User

    username = Sequence(lambda n: f'user{n}')
    email = LazyAttribute(lambda obj: f'{obj.username}@fiddler.ai')

In this setup, UserFactory dynamically generates User objects with unique attributes. The Sequence function generates usernames like user1, user2, etc., while LazyAttribute ensures that the email is based on the generated username.

Now, we can dynamically generate User objects in our test:

# File - test_user_creation.py

import pytest

@pytest.fixture
def user_factory():
    return UserFactory()

def test_user_creation(user_factory):
    user = user_factory
    assert isinstance(user, User)
    assert user.username.startswith('user')
    assert user.email == f'{user.username}@fiddler.ai'
    assert user.active is False

In this test, the UserFactory is used to generate a User object with unique attributes for each test run. The test verifies that the generated User object has the correct username and email format.

This showcases the power of factory patterns, especially when generating complex and unique test scenarios. Because the Python ecosystem is rich with supporting libraries such as factory_boy, there are many ways to efficiently create dynamic, flexible test data without writing repetitive code.

Using Fixtures via Parametrize 

Bringing this all together, you can pass fixtures themselves as parameters via parametrize. This method allows you to run tests with different fixture configurations, improving flexibility and reducing duplication.

# File - conftest.py

@pytest.fixture
def active_user():
    return UserFactory(active=True)


@pytest.fixture
def inactive_user():
    return UserFactory(active=False)

In this setup, we define two fixtures: active_user and inactive_user. These fixtures create User objects with different active statuses.

Now, we can pass these fixtures as parameters in our test:

# File - test_user_creation.py

import pytest

@pytest.mark.parametrize("fixture, expected", [("active_user", True), ("inactive_user", False)])
def test_user_status(fixture, request):
    user = request.getfixturevalue(fixture)
    assert isinstance(user, User)
    assert user.active is expected

In this test, the getfixturevalue method is used to retrieve the value of the fixture by its name (passed as a parameter). The test is executed twice: once with the active_user fixture and once with the inactive_user fixture, ensuring both scenarios are covered.

This approach adds flexibility to our test suite by dynamically injecting fixture values, allowing us to test different object configurations without writing duplicate test functions.

Conclusion: Enhancing Test Flexibility with Pytest Techniques at Fiddler

At Fiddler, we leverage many Pytest techniques, such as fixture parametrization and factory patterns, to build test suites that are flexible, scalable, and maintainable. These methods streamline our testing process and enhance the reliability of our AI Observability platform. By adopting these techniques, we handle complex test scenarios efficiently, improve test coverage, and consistently deliver an enterprise-grade AI Observability platform to the world’s leading companies.

Happy testing!

Interested in helping Fiddler define AI Observability? Join our team!