Why OpenTelemetry and Observability Are Important for APM

It used to be that application performance monitoring (APM) tools were introduced from a center of excellence and standardized so that everyone used the same tool. Then organizations underwent digital transformation initiatives, adopting DevOps, open source, new tech stacks, microservices, and containerization, and observability became far more complex.

It’s not unusual today to have teams choosing and using their own monitoring tools so they can deploy and iterate faster, as well as choosing when in the production lifecycle to bake observability in. Likewise, open source usage is increasing in most enterprises, with teams choosing a variety of open source monitoring solutions for their observability needs.

For teams, this freedom is great. But for the enterprise as a whole, having multiple monitoring tools in play makes it difficult to gain full-stack observability and glean data insights to reduce MTTR and achieve operational excellence.

Consolidating Data Sources

To solve this, enterprises need to consolidate monitoring approaches to gain a holistic view and extract data insights. For open-source technologies, enterprises are specifically turning to OpenTelemetry (OT), now in beta, which is an observability framework for cloud-native software.

Essentially, OT is a collection of tools, APIs, and SDKs that can be used to instrument, generate, collect, and export telemetry data for analysis to understand performance and behavior. It’s attempting to standardize the open-source monitoring world.

But, we still have most enterprises working in a hybrid model of technologies, and further, we’re still struggling to glean useful insights from the data to support the business’s success.  I would suggest that it doesn’t matter so much how you collect the data, as what you do with it once you do.

Enterprises are challenged with questions such as:  How do we compare the data? What do we do with the data? How can we glean performance and operations insights from it?

This issue of siloed monitoring tools in a hybrid world is a real challenge to solve.  

Can APM Solve It?

APM solutions provide observability for the enterprise so that they can have a holistic view of performance across the organization. With so many enterprises using open-source technology in their tech stacks, APM solutions must account for those sources as well.

In light of this, traditional APM solutions become inadequate and cannot keep up with new and emerging technologies and their complexity and alert to issues quickly. APM solutions need to include AIOps capabilities with robust analytics to identify anomalies quickly, predict behavior, and inform automatic corrective actions.

They should interface efficiently with the observability systems that are built into many modern applications. Because increasingly, applications provide native functionality for generating monitoring data through tracing frameworks (such as OpenTracing and Zipkin) and metrics generation tools (like Prometheus).

APM tools must be able to take full advantage of these interfaces, even if the frameworks used to implement them vary from one application to another. Equally important is the ability of APM tools to fill in the gaps by supplementing these types of native application performance data with other data sources.

At Broadcom, we support the implementation of open standards and realize the value they bring to extend observability. We included open source monitoring in our DX APM solution before OpenTelemetry came on the market.

We built DX APM to ingest data from all monitoring sources, including Open Telemetry, Opentracing Zipkin, etc, and to account for the fact that teams will use different tools. DX APM focuses on collecting and correlating data from all monitoring tools and with powerful AIOps analytics, provides actionable performance insights enabling teams to rapidly remediate problems before they start impacting the business.

Let us take an example (see Figures 1 & 2 below) of a real-world cloud-native application based on a microservices architecture. The application, Tixchange, is comprised of several different microservices such as Tixchange Web, Tixchange Services, and a Pricing Service.

Each of these microservices is managed by separate teams, utilizing different technology stacks and also utilizing different observability stacks. In this example, Tixchange Web and Tixchange Services are Java services and are instrumented by DX APM. The pricing service is a nodeJS service, and the team managing the pricing service decides to instrument it with OpenTelemetery. From a monitoring approach, you must correlate performance across all these disparate sets of data for single-pane-of-glass monitoring and faster root cause analysis.

How does Broadcom do this? The center of our tech is our topology. We automatically construct a multidimensional data model of the customer’s environment that includes all aspects of how the applications and services are calling each other and how they’re tied to the infrastructure.

All of that data creates the model and then is associated within this model. Since everything changes all the time, an understanding of the topology (really, the model IS the topology) allows every other part of the system to stay current to reality and understand exactly where the information being analyzed fits in the overall scheme of things (the application infrastructure and architecture at any given moment).

We believe our customers will continue to be hybrid, and we will continue to embrace new technology stacks and stay active in participating in OpenTelemetry groups to drive standardization there. From our perspective, it doesn’t matter how performance data is collected, it’s what you do with it, and our goal is to take the data and solve the actual performance problems.

Visit Enterprise Software Academy to learn more about Broadcom and our APM resources.