by: Shamim Ahmed
In my previous blog posts on test data management (TDM), I have discussed principles for continuous TDM and outlined how TDM for microservices-based applications differs from traditional applications.
In this blog post, the first in a two-part series, I will combine both of these concepts to discuss key approaches for applying continuous TDM to microservices. In my second post, Continuous Test Data Management for Microservices, Part 2, I detail the key steps for applying continuous TDM in various phases of the delivery lifecycle.
As I have discussed in my previous blog posts, applying TDM to microservices is quite challenging. This is due to the fact that an application may have many services, each with its own underlying diverse data store. Also, there can be intricate dependencies between these services (see the “spaghetti architecture” depicted below).
For these systems, TDM for end-to-end system tests can be quite complex. However, it lends itself very well to the continuous TDM approach. As part of this approach, it is key to align TDM with the test pyramid concept (see figure below).
Let’s look at the TDM approaches for tests in the various layers of the pyramid.
Unit tests test the code within the microservice and at the lowest level of granularity. This is typically at a function or method level within a class or object. This is no different than how we do unit testing for other types of applications. Most test data for such tests should be synthetic. Such data is typically created by the developer or software development engineer in test (SDET), who uses “as-code” algorithmic techniques, such as combinatorial. Through this approach, teams can establish a high level of test data coverage. While running unit tests, we recommend that all dependencies outside the component (or even the function being tested) are stubbed out using mocks or virtual services. See figure below.
This step is key for TDM of microservices, since the other tests in the stack depend on it. In these tests, we prepare the test data for testing the microservice or component as a whole via its API.
There are various ways of doing this (see figure below), depending on the context:
Regardless of the approach we take, our view is that in most cases (especially approaches b through d above) test data fabrication for a microservice must be prepared by the developer or producer of the microservice, and made available as part of service definition. Specifically, additional APIs should be provided to set up the test data for that component. This is necessary to allow for data encapsulation within a microservice. It is also required because different microservices may have various types of data stores, often with no direct access to the data.
This also allows the TDM of microservices applications to re-use test data, which enables teams to scale tests at higher layers of the pyramid. For example, a system or end-to-end test may span hundreds of microservices, with each having its own unique encapsulated data storage. It would be very difficult to build test data for tests that span different microservices using traditional approaches.
Again, for a single component API test, it is recommended that all dependencies from the component be virtualized to reduce the TDM burden placed on dependent systems. See figure below.
These tests validate the interaction between microservices based on behaviors defined in their API specifications (see figure below).
The TDM principles used for such testing are generally the same as for the process for API testing described previously. The process goes as follows (see figure below):
In this type of test, we have to support a chain of API calls across multiple services. For example, this type of test may involve invoking services A, B, and C in succession (see figure below).
The TDM approach for supporting this type of test is essentially the same as that for a single API test described above—except that we need to set up the test data for each of the services involved in the transaction, for example, for services A, B, C, and D.
However, an additional complexity is that we also need to ensure that the test data setup for each of these services (and the underlying services they depend on) are aligned, so the test can be successfully executed. Data synchronization across microservices is largely a data management issue, not specific to TDM per-se, so we need to ensure that our microservices architecture sufficiently addresses this requirement. There are many approaches available to address this (see here for one example).
Assuming data synchronization between microservices is in place, we recommend the following approaches to make test management easier:
The TDM approach for these tests is similar to that for system tests described above, since user actions map to underlying API calls. Such tests are likely to span more components (see figure below).
Many customers prefer to use real components, rather than virtual services, for user acceptance testing, which means that the TDM burden can be significant. As before, the key to reducing TDM complexity for such tests is to reduce the number of tests to the bare minimum, using techniques like change-impact testing, which was discussed above. We also recommend you use the change-impact approach to decide whether to use real components or their virtual services counterparts. If a set of components has changed as part of the release or deployment, it makes sense to use the actual components. However, if any dependent components are unchanged, and their test data has not been refreshed or is not readily available, then virtual services can be considered.
In this post, I’ve detailed approaches for applying TDM for microservices testing in the various layers of the pyramid. In my next post, Continuous Test Data Management for Microservices, Part 2, I’ll examine the key steps for applying continuous TDM across the delivery lifecycle.