by: Shamim Ahmed
In my prior blog, Continuous Service Virtualization, Part 1: Introduction and Best Practices, we offered an introduction to continuous service virtualization (SV) and discussed some key best practices. In this, the second and final post in the series, we will discuss the continuous SV lifecycle and how it helps to optimize DevOps and the continuous integration/continuous delivery (CI/CD) pipeline.
The following figure summarizes the different activities in a typical continuous SV process across the different stages of the CI/CD pipeline.
Figure 1: Continuous SV lifecycle
These activities are summarized below:
The SV activities during this stage need to support the requirements of agile development (Step 2). Following are some of the key steps:
The SV activities at this stage support the needs of the subsequent stages of the CI/CD lifecycle. Following are a few of the most critical efforts:
Typically orchestrated by the CI engine, this is an automated step that triggers the build of the necessary software components and the running of a variety of tests, including build verification or smoke tests, regression, component integration, and so on. This automation uses the virtual services that were created in step 2.
The tests along the CD lifecycle (from X-team integration to pre-production tests) use different types of virtual services based on the principle of progressive virtualization. In this way, tests get progressively more realistic as they move towards the right. Also note that the mix of virtual services will be different in each stage of the lifecycle as will the scope of the tests. Another interesting and evolving approach is one in which site reliability engineers (SREs) use service virtualization for performance canary testing, chaos engineering/testing, and other forms of negative testing in production or near-production environments. For example, using virtual services, it is easier to emulate significant service degradation, or even failure, as opposed to adjusting a real service.
Even within traditional COE models, the use of service virtualization provides significant benefits in the context of testing. (These benefits are well documented here.) This includes enablement of agile parallel development and productivity improvements, support for the shift-left of testing, better application quality, easier management of test environments, and more.
Continuous SV, on the other hand, is an integral supporting pillar of continuous testing (CT) and CD (as part of a continuous everything philosophy) and is a key success factor for both. This continuous SV approach offers these key benefits:
Progressive virtualization described above is a key tenet for supporting CT/CD. It ensures that appropriate virtual services are created, updated, and re-used in every part of the CI/CD lifecycle, by different stakeholders, and not just during testing. For example, if a virtual service for a dependent endpoint is not already available, developers can create and use lightweight synthetic virtual services for unit and component tests. Testers can then enrich those virtual services with additional transaction support and test data. Another key tenet is the integration with continuous test data management (see my blog on that subject here). Such integration ensures that virtual services can not only help with the creation of test data, but also can be created “fit for purpose” with appropriate test data and transaction support for different types of testing needs. One can argue that for most applications it is not practical to do CT or CD without continuous SV.
Continuous SV requires active participation from, and collaboration between, various federated stakeholders across application teams—including architects, developers, SDETs, testers (both functional and performance), test data engineers, release/deployment engineers, and Site Reliability Engineers (SREs) to name a few. This is different from traditional SV COE models in which virtual services were primarily created and maintained by testers. With limited collaboration, these earlier approaches failed to scale in a rapidly changing application landscape.
Continuous SV ensures that virtual services are an integral part of the agile design and development process. Virtual services are continuously updated (in the context of application changes), re-used, easily deployed with automation, and synchronized with real-world data from production. This active “whole team” collaboration—a key tenet of DevOps—also means that it improves the productivity of all these different stakeholders, not just testers alone.
As a corollary to the traditional COE approaches outlined above, the “whole team” federated approach to continuous SV makes it easier to scale in large enterprises with multiple applications undergoing frequent changes. In this approach, virtual services are treated like first-class “software” objects with a disciplined software engineering approach. For example, virtual services can be tracked and updated using a source code change management process (and versioned in source control repositories) by various stakeholders. They can be packaged for deployment along with applications, and they can be shared and accessed using a centralized service repository. This software engineering (or some call it “as-code”) approach to continuous SV enables better scalability.
While use of SV helps to make it simpler for testers to do testing, that alone may not significantly impact the overall delivery cycle time. Whereas continuous SV—by virtue of the fact that it helps to accelerate cycle time in every step of the CI/CD lifecycle—has a much more significant impact.
Continuous reliability is an SRE discipline that seeks to pro-actively assure reliability of applications in the context of frequent changes deployed to production. A continuous SV approach allows us to emulate virtual services more realistically with appropriate SLOs as part of continuous reliability testing across the lifecycle. Where SLOs are not defined (for non-critical components), SREs can obtain SLI data from such components in production and emulate their performance correctly in pre-production. In addition, as mentioned above, SREs take advantage of virtual services to perform negative and chaos testing in production or near-production environments.
Continuous SV is meant to be practiced in conjunction with CT and CD. Various resources offer insights into evolving to continuous testing. If you are already practicing CT/CD and want to move to continuous SV, our recommendation is to proceed as follows:
Regardless of the approach you take, you need to also make some organizational, cultural, and skills adjustments. These are related to changes you need to make for supporting CT/CD overall. Here are some of the changes that relate specifically to continuous SV:
Generally speaking, these organizational and cultural changes are the most difficult aspect of transitioning to continuous SV. Luckily, organizational change management processes in DevOps are now becoming a more established discipline. These same principles will help with continuous SV as well.
Until such time, my friends, stay well, and may all your SV efforts be continuous!
