Testing asynchronous systems
I'm spending these days kicking off end-to-end tests. By end-to-end tests, I mean tests that interact with a set of IT systems as if the set was a big black box. For example, one test sends a REST API request to system A, waits until the data propagates through systems B and C to system D, and checks the result in system D via another REST API call. Now, how to implement the waiting part for this asynchronous test?
The first option is to wait for a fixed period of time, such as 10 seconds or 1 minute. But with the period being too long, it's cumbersome to develop such tests; I want fast feedback loops when developing. On the other hand, with the period being too short, the tests can fail; and I need tests to be stable to trust them.
Another option is to set up a service that provides visibility into the asynchronous request. So we can periodically poll the status of the request from tests. However, building such a service can be a mound of engineering work across multiple systems. Such a service could not be even feasible to build.
The third option is to write tests that check periodically that the expectation is met until a given timeout is reached. For example, we can check every second that the result in system D is present; and we can keep checking up until one minute. I find the third option to be the sweet spot: minimal engineering work, minimal waiting time, and fast feedback loops.
It turns out there are libraries built for such periodic checks (thanks Peter for introducing me to these). For my current project based on Javascript/Typescript and jest, there's wait-for-expect. For JVM-based languages, such as Scala, there's awaitility.
- ← Previous post: Generating tax reports with Python, Jinja, and Click
- → Next post: Contractor didn't deliver: how to resolve?
This blog is written by Marcel Krcah, an independent consultant for product-oriented software engineering. If you like what you read, sign up for my newsletter