Installation and Upgrade Enhancements Delivered in DX Platform 23.3

On November 7, 2023, the AIOps and Observability team announced general availability of DX Operational Intelligence 23.3 and DX APM 23.3 for on-premises deployments. While the announcements and Release Notes cover all the important enhancements, several new capabilities deserve additional attention—especially those for installing and upgrading the DX Platform.

These enhancements offer the following benefits:

• Streamline the deployment process and reduce the risk of human error.
• Extend options for performance testing and hardware validation testing.
• Provide greater control for configuration, before and after an upgrade.
• Allow organizations to upgrade directly from 23.1 to 23.3, skipping 23.2.
• Simplify charting customizations for developers using the DX Platform.

Below, you’ll see seven enhancement areas involving installation and upgrades. Each section provides short descriptions of what’s changed in DX Platform 23.3, and the benefits of these changes.

1. HELM charts replace conventional YAML

Beginning with DX Platform 23.2, the installer transitioned from relying on YAML files to utilizing Helm Charts for component deployment.  

Previously, to deploy components, operators and developers were required to configure and manage an individual YAML file for each component. With Helm Charts and their support of parameterization through value files, a single customized chart can be used to configure multiple Kubernetes components. This reduces effort and risk of error for operators and developers, without sacrificing flexibility.

With the release of DX Platform 23.2, customizations are delivered via custom value files. Now upgrades subsequent to 23.2 will automatically use customizations from the prior Helm-based installation.

Pre-installation changes can now be done to files inside this folder:  <installer_dir>/custom_values. For post-installation changes, navigate to <nfs_folder>/update/helm folder and make changes there.

2. Enhanced update command

The Installer 23.3 supports upgrading from the immediate prior release (23.2) and directly from 23.1 to 23.3. The Update Command also runs a larger number of validations, which mirrors the behavior of the Install Command.  

3. –dry-run flag for the install and upgrade commands

In DX Platform 23.2, a new flag was added that can be used to generate a Helm Chart file and individual yaml files—prior to performing the actual install or upgrade—without modifying the cluster. Operators and developers can review and validate these files in advance.

4. Enhanced configuration of OpenShift Route

Introduced in DX Platform 23.2, OpenShift installations now exclusively create Routes (not Ingresses) by utilizing default OpenShift certificates. DX Platform 23.3 extends customization capabilities so operators and developers can configure OpenShift Routes with custom certificates and key files in addition to the default OpenShift certificates. Here are a couple advantages:

1. Enabling custom certificates allows the implementation of robust encryption mechanisms, which can be tailored to support security specifications.
2. Enabling the use of custom certificates also helps operators ensure that OpenShift Routes align with internal security policies and promote a cohesive and compliant infrastructure.

5. Optimized deployment strategies

Beginning with DX Platform 23.2, deploying Elasticsearch, Kafka, and Zookeeper no longer relies on Elasticsearch labels. This change eliminates the need to label Elasticsearch nodes. With 23.3, for a fresh installation in a Cluster Administrator scenario with local storage enabled, the installer will show prompts for each node name and path for ElasticSearch and Kafka PVs.

6. NFS IOPS performance check tool

To ensure optimal performance, the installer includes scripts for conducting NFS IOPS performance checks. Executed manually and prior to installation, these checks are vital to verify minimum hardware and software requirements are met and to prevent potential runtime failures. By comparing these performance metrics collected by the scripts to the recommended IOPS for the product, operators can assess whether the system meets performance standards.

1. In DX Platform 23.3, several scripts to run IOPS tests have been updated.  This script (dxi-fio-test.sh), which is located in the Tools directory, runs fio tests. If you install fio on the cluster node (or host where you can mount NFS), you can run the script to test disk performance on that host using the script. If you want to run tests in pods, dxi-fio-docker.sh can be used for this, but you need to push the image to your registry first. This script creates daemonset/pod and runs fio checks in it.  dxi-prepare-baseimage.sh used to push the image with the fio utility into the selected registry.
2. The utilities dxi-fio-test.sh and dxi-fio.docker.sh can be used to assess the performance of NFS. The scripts perform benchmark tests on Kubernetes persistent disk volumes, measuring key parameters, such as read/write IOPS, bandwidth in MB/s, and latency. Upon completion of the testing process, the scripts record NFS performance metrics, specifically IOPS, in a file named test_report_<date>.txt. They also generate a detailed log file named test_summary_<date/time>.txt.
3. The scripts can be executed to test either a single pod on a specified node or, when deploying a daemonset, to initiate pods on all nodes. They can also be used in a pre-existing namespace. PVC and UID\GUID can be specified during script execution which will open a path for using script as namespace admin.  

7. New script to generate the YAML for PV creation

To create PVs manually, operators can use the dxi-generate-pvs.sh script (available in the Tools directory) to generate a dxi-pvs.yaml file with PVs.

With yaml files, a small mistake can lead to misconfigurations and result in failures or unnoticed issues. This script helps reduce these risks by prompting operators to enter specific values from which the script generates PV templates. One set of the prompts is responsible for adding dxi/dxi-backup PVs to the template. A second set of prompts is responsible for adding jarvis es/kafka/zookeeper PVs. This structure makes it possible to generate multiple templates to support either installations or upgrades.

Note: It is required to re-create jarvis PVs when upgrading to 23.3. After generation, yaml can be verified and, later, applied.

For additional information, refer to these technical articles:

• Configure the DX Platform Installation
• DX Platform Scripts

These DX Platform Installer advancements reflect our team’s commitment to streamline the deployment process, offer greater configurability, and ensure optimal performance. Customers benefit from a more tailored, efficient, and robust installation experience, while minimizing complexities and enhancing the platform's reliability.