<img height="1" width="1" style="display:none;" alt="" src="https://px.ads.linkedin.com/collect/?pid=1110556&amp;fmt=gif">
February 16, 2022

Continuous Test Data Management for Microservices, Part 1: Key Approaches

by: Shamim Ahmed

Introduction 

In my previous blog posts on test data management (TDM), I have discussed principles for continuous TDM and outlined how TDM for microservices-based applications differs from traditional applications. 

In this blog post, the first in a two-part series, I will combine both of these concepts to discuss key approaches for applying continuous TDM to microservices. In my second post, Continuous Test Data Management for Microservices, Part 2, I detail the key steps for applying continuous TDM in various phases of the delivery lifecycle.

Revisiting the Continuous TDM Approach in the Context of Microservices Applications

As I have discussed in my previous blog posts, applying TDM to microservices is quite challenging. This is due to the fact that an application may have many services, each with its own underlying diverse data store. Also, there can be intricate dependencies between these services (see the “spaghetti architecture” depicted below). 

Broadcom Enterprise Software Academy – Continuous Test Data Management for Microservices, Part 1: Key Approaches

For these systems, TDM for end-to-end system tests can be quite complex. However, it lends itself very well to the continuous TDM approach. As part of this approach, it is key to align TDM with the test pyramid concept (see figure below).

Broadcom Enterprise Software Academy – Continuous Test Data Management for Microservices, Part 1: Key Approaches

Let’s look at the TDM approaches for tests in the various layers of the pyramid. 

1. TDM Approach for Supporting Microservices Unit Tests 

Unit tests test the code within the microservice and at the lowest level of granularity. This is typically at a function or method level within a class or object. This is no different than how we do unit testing for other types of applications. Most test data for such tests should be synthetic. Such data is typically created by the developer or software development engineer in test (SDET), who uses “as-code” algorithmic techniques, such as combinatorial. Through this approach, teams can establish a high level of test data coverage. While running unit tests, we recommend that all dependencies outside the component (or even the function being tested) are stubbed out using mocks or virtual services. See figure below. 

Broadcom Enterprise Software Academy – Continuous Test Data Management for Microservices, Part 1: Key Approaches

2. TDM Approach for Supporting Microservices Component or API Tests

This step is key for TDM of microservices, since the other tests in the stack depend on it.  In these tests, we prepare the test data for testing the microservice or component as a whole via its API.

There are various ways of doing this (see figure below), depending on the context: 

  1. Generate simple synthetic test data based on the API specs. This is typically used for property-based testing or unit testing of the API.
  2. Generate more robust synthetic test data from API models, for example, by using a test modeling tool like Broadcom Agile Requirements Designer. This enables us to do more rigorous API testing, for example for regression tests.
  3. Generate test data by traffic sniffing a production instance of the service, for example, by using a tool like Wireshark. This helps us create more production-like data. This approach is very useful if for some reason it isn’t possible to take a subset of data from production instances. 
  4. Generate test data by sub-setting and masking test data from a production instance of the service, or by using data virtualization. Note that many microservice architectures do not allow direct access to the data store, so we may need special data access APIs to create such test data.  

Broadcom Enterprise Software Academy – Continuous Test Data Management for Microservices, Part 1: Key Approaches

Regardless of the approach we take, our view is that in most cases (especially approaches b through d above) test data fabrication for a microservice must be prepared by the developer or producer of the microservice, and made available as part of service definition. Specifically, additional APIs should be provided to set up the test data for that component. This is necessary to allow for data encapsulation within a microservice. It is also required because different microservices may have various types of data stores, often with no direct access to the data. 

This also allows the TDM of microservices applications to re-use test data, which enables teams to scale tests at higher layers of the pyramid. For example, a system or end-to-end test may span hundreds of microservices, with each having its own unique encapsulated data storage. It would be very difficult to build test data for tests that span different microservices using traditional approaches.   

Again, for a single component API test, it is recommended that all dependencies from the component be virtualized to reduce the TDM burden placed on dependent systems. See figure below. 

Broadcom Enterprise Software Academy – Continuous Test Data Management for Microservices, Part 1: Key Approaches

3. TDM Approach for Supporting Microservices Integration and Contract Tests

These tests validate the interaction between microservices based on behaviors defined in their API specifications (see figure below). 

Broadcom Enterprise Software Academy – Continuous Test Data Management for Microservices, Part 1: Key Approaches

The TDM principles used for such testing are generally the same as for the process for API testing described previously. The process goes as follows (see figure below): 

  1. For contract definition, we recommend using synthetic test data, for example, based on the API specs, to define the tests for the provider component. 
  2. The validated contract should be a recorded virtual service based on the provider service. This virtual service can then be used for consumer tests. Note that in this case, a virtual service recording forms the basis of the test data for the consumer test. 

Broadcom Enterprise Software Academy – Continuous Test Data Management for Microservices, Part 1: Key Approaches

4. TDM Approach for Supporting a Cross-service System Test or Transaction Test at the API Level 

In this type of test, we have to support a chain of API calls across multiple services. For example, this type of test may involve invoking services A, B, and C in succession (see figure below).

Broadcom Enterprise Software Academy – Continuous Test Data Management for Microservices, Part 1: Key Approaches

The TDM approach for supporting this type of test is essentially the same as that for a single API test described above—except that we need to set up the test data for each of the services involved in the transaction, for example, for services A, B, C, and D. 

However, an additional complexity is that we also need to ensure that the test data setup for each of these services (and the underlying services they depend on) are aligned, so the test can be successfully executed. Data synchronization across microservices is largely a data management issue, not specific to TDM per-se, so we need to ensure that our microservices architecture sufficiently addresses this requirement. There are many approaches available to address this (see here for one example). 

Assuming data synchronization between microservices is in place, we recommend the following approaches to make test management easier: 

  1. As mentioned before, use model-based testing to describe the cross-service system tests. This allows us to specify test data constraints for the test uniformly across affected services, so that that initial setup of test data is correct. This is done using the test data setup APIs we discussed above.
  2. Since setting up test data definition across services is more time consuming, we recommend minimizing the number of cross-service tests, based on change impact testing. We run transaction tests only if the transaction, or any of the underlying components of the transaction, have changed. Again, this is a key principle of continuous testing that’s aligned with the test pyramid. 
  3. If there have been no changes to a participating component or underlying sub-component, we recommend using a virtual service representation of that component, for example, services C and D referenced above. This will help to further reduce the TDM burden for that component. 

5. TDM Approach for Supporting End-to-End Business Process or User Acceptance Tests 

The TDM approach for these tests is similar to that for system tests described above, since user actions map to underlying API calls. Such tests are likely to span more components (see figure below). 

Broadcom Enterprise Software Academy – Continuous Test Data Management for Microservices, Part 1: Key Approaches

Many customers prefer to use real components, rather than virtual services, for user acceptance testing, which means that the TDM burden can be significant. As before, the key to reducing TDM complexity for such tests is to reduce the number of tests to the bare minimum, using techniques like change-impact testing, which was discussed above. We also recommend you use the change-impact approach to decide whether to use real components or their virtual services counterparts. If a set of components has changed as part of the release or deployment, it makes sense to use the actual components. However, if any dependent components are unchanged, and their test data has not been refreshed or is not readily available, then virtual services can be considered. 

Conclusion

In this post, I’ve detailed approaches for applying TDM for microservices testing in the various layers of the pyramid. In my next post, Continuous Test Data Management for Microservices, Part 2, I’ll examine the key steps for applying continuous TDM across the delivery lifecycle. 

Explore More Posts

View All Blog Posts
June 13, 2022

Optimize Continuous Delivery of Microservices Applications with Continuous Performance Testing

Here is a look at the the full lifecycle implementing end-to-end continuous performance testing for micro-services. Read Now
May 25, 2022

What's New in Test Data Manager 4.10?

This latest release lets customers deploy TDM at scale, facilitate migrations of SAP applications and operate across a wide range of databases. Read Now
April 22, 2022

Register Today for Our "What's New in Test Data Manager 4.10" Webinar

Learn how the new release lets you deploy TDM at scale, facilitate migrations of SAP applications, and operate across a wide range of databases. Read Now
February 21, 2022

Continuous Test Data Management for Microservices, Part 2: Key Steps

This post, the second in a two-part series, outlines the key steps for applying continuous test data management in microservices environments. Read Now
February 16, 2022

Continuous Test Data Management for Microservices, Part 1: Key Approaches

This post, the first in a two-part series, looks at key concepts for applying continuous test data management (TDM) to microservices environments. Read Now
January 24, 2022

Continuous Service Virtualization, Part 1: Introduction and Best Practices

In this blog, the first of a two-part series, we will describe some best practices for establishing continuous service virtualization as part of DevOps. Read Now
January 24, 2022

Continuous Service Virtualization, Part 2: Steps for Optimizing DevOps

In this second post in our two-part series, we discuss four key steps for leveraging continuous service virtualization to optimize the DevOps lifecycle. Read Now
November 3, 2021

Employing Agile Requirements Designer to Support API Testing: 5 Use Cases

With Agile Requirements Designer, teams can create API tests that can be executed with their test automation framework. This post describes five use cases. Read Now
September 28, 2021

Optimizing Continuous Testing with Continuous Test Data Management

This blog describes some best practices for establishing continuous test data management (TDM) as part of continuous testing. Read Now
September 8, 2021

Security Testing in an Agile Development World

This blog post presents different security testing methods and provides a few tips for conducting a more secure code review. Read Now