Dependency

Due to its inherent nature, load testing amplifies usage. By doing so, it may help surfacing implementation issues. Previously, we studied an application which leverages a service provider, namely httpbin.org, to perform some action.
Every single time our endpoint will be exercised results in a call to the underlying service provider. It could quickly become an issue, regarding either cost or conduct, as load testing obviously implies load. Imagine now you have to pay for the said service, acquire a license token upstream or simply end up disturbing production flow by flooding traffic. Even if you do not fall into previous use cases, leveraging underlying service for testing is not acceptable. Because you are not testing in isolation and thus cannot stick to determinism trait your tests should obey. Let’s see how to properly deal with such use cases.

Problem statement

We would like to test the following FastAPI path operation.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12


@api_version(1)
@httpbin_router.get("/random")
async def httpbin_random_bytes(
    numbytes: Annotated[int | None, Query(gt=0, le=50)] = 20
) -> HttpbinRandomResponse:
    """
    Returns a base64 encoded string made of 'numbytes' random bytes generated online at httppbin.org
    """
    res = requests.get(f"https://httpbin.org/bytes/{numbytes}")
    return HttpbinRandomResponse(
        random_bytes=b64encode(res.content).decode("ascii"),
    )

Before engaging, we notice multiple issues that may complicate our journey:

We are tightly coupled with the underlying service provider httpbin.org.
Moreover, httpbin.org is an external service that may complicate its customization. External refers to something out of the scope of our application. We should consider every dependencies this way, even or especially - YMMV - company ones.
We mixed concerns as path operation should ideally only ingest/shape parameters and digest/shape outcome while offloading process.
We may question how data shaping is done, i.e., if encoding could be relocated outside of the HttpbinRandomResponse constructor.

Process complexity is by no mean correlated to the code length used to trigger the call. In fact, it is a perfect example of seeing the forest as the trees. This one-liner call hide a lot of implicit variables, such as response time or failure rate, that make building resilient tests against very difficult. In fact, smart APIs often surface one-liner or fluent flavors, which is damn good but in the meanwhile deactivates our natural threshold of code splitting. May this simple call translates into ten lines instead would have lead us to consider relocating them outside of the method sooner.

Pattern

We took the decision to refactor the code base applying well-known design patterns, namely:

Dependency Inversion by introducing an abstraction to replace the tightly coupling.
Dependency Injection to inject an implementation of the abstraction from the outside.
Separation of Concerns by dispatching code pieces to the layer they belong to.
Mock implementation to be exercised by unit testing to enforce isolation.

Refactoring

There are multiple ways to achieve this refactoring especially regarding the sequence of actions. We advocate to start with interfaces because they allow to gently scale (code remains correct, parallelization becomes an option, …). Once in place, there is no specific order to follow.

Introduce service interface

1
2
3
4


class IService(ABC):
    @abstractmethod
    def get(numbytes: int) -> str:
        pass

Good pattern is to split concerns. We decided to keep IService agnostic of the web calling context, especially deciding to use str instead of HttpbinRandomResponse for return signature, considering it as a responsibility for path operation instead.

You may notice we still use primitives, int and str here. Good pattern would have been to craft and use the value object design pattern. See Core boundaries for details.

Introduce injection interface

1
2
3
4
5
6


def get_service() -> IService:
    return (
        MockService
        if os.getenv("IS_LOAD_TESTING", False)
        else Service
    )

Good pattern would have been to retrieve IS_LOAD_TESTING value upstream instead and inject it there. Once again, look at Core boundaries for details.

Swap

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12


@api_version(1)
@httpbin_router.get("/random")
async def httpbin_random_bytes(
    service: IService = Depends(get_service),
    numbytes: Annotated[int | None, Query(gt=0, le=50)] = 20) -> HttpbinRandomResponse:
    # Morph input if any (likely to be done by pydantic)

    # delegates
    res = service.get(numbytes)

    # Shape result leveraging pydantic
    return HttpbinRandomResponse(random_bytes=res)

We leverage built-in FastAPI dependency mechanism via depends to inject our dependency and remove the tightly coupling we had before. We then use this injected dependency to offload process. We also amend HttpbinRandomResponse constructor taken into account design choice of relocating data encoding within service.

Create a first implementation to mock process

1
2
3
4
5


class MockService(IService):
    @classmethod
    def get(numbytes: int) -> str:
        time.sleep(1)
        return ["Fake data."]

Good pattern would have been to extract the 1 scalar and to promote it as a parameter. We are likely to play with this parameter to show case impact of such variable on the overall system, e.g., does a slower process impact the flow and thus the UX? On the other hand, how would the system react to an over-performing service, e.g., drastically reducing it - up to the the 0 passthrough - could surface some race conditions…

Current implementation is a good mock to start with, replacing a call to an external service with something we have full control over. This said, we took some freedom with the reality. Not all calls last for a second. Not all calls even succeed in the real world. We just crafted a Reduced Order Model, aka ROM, to fake the underlying service and mitigate our dependency. Nothing new, as most of us are using ROM for physics simulation. Coding is no exception.
This approach is really good as we can now foresee how we could extend our ROM capabilities. Sticking within time dimension, we could instead provide a baseline for the duration and weight it using delta value (e.g., 3s ± 0.5s) or percentage (e.g., 2s ± 20%). If we want to introduce some failure to assess how the overall workflow will accommodate such issues, we can complicate the algorithm, adding some failures using timer (e.g., mock fails every 3s) or counter (e.g., mock fails every 10 calls). Obviously, failure parameters become new variables we can play with, adding some randomness for example (e.g., mock fails every n calls, n being a random value between 5 and 15, reset every time it is triggered). By doing so, you can emulate the mocked service. This said, keep in mind mock should serve downstream testing campaign. By adding too much randomness, you also sacrifice determinism and thus make comparison between two testing campaign more difficult to gauge. Trade-off once again. Trade-off everywhere.

Create a second implementation to effectively perform process

1
2
3
4
5
6
7
8


class Service(IService):
    @classmethod
    def get(numbytes: int) -> str:
        """
        Returns a base64 encoded string made of 'numbytes' random bytes generated online at httppbin.org
        """
        res = requests.get(f"https://httpbin.org/bytes/{numbytes}")
        return b64encode(res.content).decode("ascii")

Up to you here to decide if it makes sense to surface implementation details such as the encoding patterns. I would argue that it is not that obvious compared to the previous mock scalar. But as always, it is a contextual decision.

Closing

Starting from a tiny snippet, we show case how one could think about unit testing it. A quick analysis helps to surface some issues that may be easier to deal with upstream via a proper refactoring instead of downstream pushing all this extra complexity at the testing stage.
We also see how to identify and combine well-know design patterns to achieve our goal, stressing along the way the shortcuts we still decided to live with. Keep in mind a refactoring does not mean the resulting code will be the best one, only it will fix some identified issues or prepare for another round. It is a good practice to stage a refactoring if this one becomes too complex to be addressed at once. Here for example, I would advocate to start this way and write a unit test prior to engage the value object yard. This way, it would be easier to assess breaking changes if any as you will have a test baseline to refer to. Do not underestimate what staging implies regarding both final target but also as achievement rewards. Software development is not a speedrunning game, you can and you probably should have healthier and more regular rests.