<img height="1" width="1" style="display:none;" alt="" src="https://px.ads.linkedin.com/collect/?pid=1110556&amp;fmt=gif">
Skip to content
    July 9, 2025

    Auto Scaling of Broadcom TDM Masking on Kubernetes

    8 min read

    Key Takeaways
    • Optimize cloud infrastructure for Broadcom Test Data Manager (TDM) masking workloads to improve both performance and cost efficiency.
    • Automate Kubernetes resource provisioning, ensuring scalable and responsive operations that align with real-time business demand.
    • Implement automated scaling to align technology investments with business objectives and enhance return on cloud spending.

     

    Overview

    Broadcom Test Data Manager (TDM) offers advanced Al data profiling algorithms for automatic discovery of potentially sensitive information that is stored enterprise wide. TDM enables organizations to protect and manage large, complex data sets efficiently—even as they scale—while maintaining data integrity and preserving complex relationships. The result is high-quality test data that mirrors production without exposing sensitive content. As customer data volumes grow exponentially, TDM users increasingly depend on scalable, on-demand compute resources to meet performance and compliance demands.

    Orchestration platforms such as Kubernetes help customers reduce costs by enabling on-demand, scalable compute resources. Customers can manually scale out and scale in their Kubernetes compute resources as needed.

    Scaling a Kubernetes deployment enables dynamic adjustment of application instances to meet varying demand, ensuring optimal performance and resource utilization.

    Historically, scaling of TDM Kubernetes masking containers was handled manually, resulting in increased operational overhead and slower response times.

    By implementing auto-scaling, we reduce infrastructure and labor costs while boosting agility—enabling faster responses to market shifts and customer demands, and ultimately strengthening our competitive edge.

    Auto-scaling is the process of automatically adjusting compute resources to meet a system's performance requirements. As workloads grow, systems require additional resources to sustain performance and handle increasing demand. Conversely, when demand decreases, an automatic scaling service can reduce costs by scaling in unutilized compute resources.

    Auto-scaling eliminates the need for manual human intervention in the scaling process, reducing the probability of incorrect resource allocation and saving cost by automatically allocating the exact compute power that is required.

    To deliver a truly scalable and cost-efficient masking solution, TDM supports automated scaling of masking pods. This includes the ability to seamlessly scale out by monitoring demand and launching additional pods as needed, and to scale in by identifying and shutting down idle pods.

    Such automation enhances operational efficiency, optimizes resource usage, and ensures the platform can adapt quickly to changing workloads.

    In this post, we will show you how you can use Kubernetes Event Driven Autoscaling (KEDA) to easily auto-scale the TDM masking service based on your own masking workloads.

    KEDA is a Kubernetes-based, Event-Driven Autoscaler. With KEDA, you can drive the scaling of any container in Kubernetes based on custom application metrics that are external to Kubernetes.

    Prerequisites

    You will need the following to complete the steps in this post:

    The following diagram shows the complete setup that we will walk through in this post:

    ESD_FY25_Academy-Blog.Auto-Scaling-Broadcom-TDM-Masking-Kubernetes.Figure-01

     

    TDM masking auto-scaling setup

    Step 1: Before you begin TDM Kubernetes deployment

    Before you deploy TDM in Kubernetes, ensure that these prerequisites are met.

    Step 2: Install KEDA on the TDM Kubernetes cluster

    KEDA should be installed and available on the Kubernetes cluster. TDM uses KEDA for the automatic scaling of the masking service pods. For more information on KEDA deployment, see KEDA Documentation.

    You can use the following Helm commands to install KEDA:

    helm repo add kedacore https://kedacore.github.io/charts

    helm repo update

    helm install keda kedacore/keda --namespace keda --create-namespace

     

    Step 3: Configure values.yaml file for auto-scaling

    You should configure the values of TDM Helm Chart according to your Kubernetes system setup in the values.yaml file.

    After configuring, use the updated values.yaml file for the TDM deployment on your Kubernetes cluster using Helm Chart.

    Please verify that autoscaling.scaledObject.enabled helm value is true.

    For this simple demonstration, please set the masking.service.poolSize helm value to “1”.

    ESD_FY25_Academy-Blog.Auto-Scaling-Broadcom-TDM-Masking-Kubernetes.Figure-02

     

    Step 4: Deploy TDM on Kubernetes using Helm Chart

    After you complete the pre-deployment tasks, you can deploy TDM on the Kubernetes cluster using the Helm Chart following these steps.

     

    Auto-scaling in action

    The external KEDA ScaledObject is now periodically polling the TDM Portal web service (over gRPC protocol) for the value of the "numberOfActiveMaskingTasks" metric. Based on the value and the target size (masking.service.poolSize helm value) for this metric, KEDA is scaling the TDM masking deployment.

    The default value of the masking.service.poolSize helm value is "4". That is, each masking pod should not run more than "4" masking tasks.

    For this simple demonstration, we will set the masking.service.poolSize helm value to “1”.

    For example, when the "numberOfActiveMaskingTasks" metric is "5", KEDA should automatically scale the masking deployment to run (5 / 1 =) "5" masking pods.

    The default polling interval (autoscaling.scaledObject.pollingInterval helm value) is set to 30 seconds in helm values.yaml.

    The Kubernetes logs of the keda-operator pod (in the keda namespace) should show that the “maskingScaledObject in the “tdm” namespace has detected the “masking” deployment in the “tdm” namespace:

    ESD_FY25_Academy-Blog.Auto-Scaling-Broadcom-TDM-Masking-Kubernetes.Figure-03

    The TDM Portal web service is splitting each masking job to one or more masking tasks. Each masking task is executed by a dedicated FastDataMasker process (running concurrently inside the masking pod). The default value of the tdmweb.TDMMaskingService.tableTaskRowThreshold property is 1000000. That is, when running a masking job on multiple database tables, every database table that has more than 1,000,000 rows will be assigned to its own dedicated masking task (within this masking job).

    Let’s put some load on the TDM masking service by running a masking job of 5 large (>1,000,000 rows) SQL database tables with masking.service.poolSize set to 1.

    You may use the following SQL commands for creating 5 large (>1,000,000 rows) SQL database tables (or use your own large database tables).

    ESD_FY25_Academy-Blog.Auto-Scaling-Broadcom-TDM-Masking-Kubernetes.Figure-04

    Immediately after the status of this masking job is RUNNING, TDM will respond to KEDA with the current value of the "numberOfActiveMaskingTasks" metric. The reported value should reflect the current number of active masking tasks. If the masking job is masking 5 large database tables, then the reported metric value is ‘5’.

    If the masking.service.poolSize helm value is set to “1”, then the reported target-size is ‘1’.

    You may check the TDMMaskingService.tdmweb.log log file (in the $TDM_HOME/logs/ folder inside any of the pods) for the following log message:

    ESD_FY25_Academy-Blog.Auto-Scaling-Broadcom-TDM-Masking-Kubernetes.Figure-05

    So since the reported "numberOfActiveMaskingTasks" is "5", KEDA should now automatically scale the masking deployment to run (5 / 1 =) "5" masking pods.

    You may now run the following commands and watch KEDA auto-scaling in action:

    ESD_FY25_Academy-Blog.Auto-Scaling-Broadcom-TDM-Masking-Kubernetes.Figure-06

    You may see KEDA auto-scaling in action by watching the number of TDM masking pods automatically scaling out:

    ESD_FY25_Academy-Blog.Auto-Scaling-Broadcom-TDM-Masking-Kubernetes.Figure-07

    Once each of the masking tasks is completed, the reported value for the “numberOfActiveMaskingTasks” metric will gradually decrease from ‘5’ back to ‘0’. KEDA will now gradually automatically scale in the number of TDM masking pods.

    ESD_FY25_Academy-Blog.Auto-Scaling-Broadcom-TDM-Masking-Kubernetes.Figure-08

    This example is scaling the number of masking pods (replicas) based on the value of an application TDM metric in real time.

    The TDM Portal pod is running a gRPC server that is responding to the periodic getMetrics requests that are generated by the external KEDA ScaledObject.

    The external KEDA ScaledObject is dividing the returned value of the “numberOfActiveMaskingTasks” metric by its target size (masking.service.poolSize helm value) and sets the number of masking pods (replicas) accordingly.

    For example, metricValue: "5" and targetSize: "1" will set the masking replicas to 5 ( "5" masking tasks require 5 concurrently running masking pods). Each masking pod should not run more than "1" (targetSize) masking tasks.

    Clean up

    Use the following commands to delete resources created during this post:

    helm uninstall tdm -n tdm

    Helm uninstall tdmpvc -n tdm

    helm uninstall tdmzk -n tdm

    helm uninstall keda -n keda

    kubectl delete storageclass tdm

    kubectl delete namespace tdm

    kubectl delete namespace keda

     

    References

    Ofer Yaniv

    Ofer leads the Kubernetes Architecture track within Broadcom AppDev. With over 20 years of experience in designing and developing large-scale applications, he plays a key role in driving innovation and modernization of AppDev solutions through cutting-edge technologies. He is at the forefront of implementing IMS...

    Other posts you might be interested in

    Explore the Catalog
    icon
    Blog June 17, 2025

    Broadcom's Approach to Testing Mainframe Applications

    Read More
    icon
    Blog May 16, 2025

    Auto Scaling of Kubernetes Workloads Using Custom Application Metrics

    Read More
    icon
    Blog August 16, 2024

    How Broadcom Test Data Manager Helps Enterprises Meet Fast-Changing Global Data Security and Privacy Requirements

    Read More
    icon
    Blog May 31, 2024

    Revolutionizing Healthcare Demands a New Approach to Software Development

    Read More
    icon
    Blog December 20, 2023

    Broadcom Software Academy Wins Silver in Brandon Hall Group’s Excellence in Technology Awards

    Read More
    icon
    Blog October 20, 2023

    The Broadcom Approach to Test Data Management for the Mainframe

    Read More
    icon
    Blog August 1, 2023

    How ARD Helps DevOps Realize Its Full Potential

    Read More
    icon
    Blog April 20, 2023

    Revitalize Your Testing With Continuous Everything Practices to Meet DevOps Goals

    Read More
    icon
    Blog March 16, 2023

    Better Together: Optimize Test Data Management for Distributed Applications with Service Virtualization

    Read More