Scaling From 2,000 to 60,000 Remote Users? How One Enterprise Made It Happen

There’s the old adage we’ve all heard countless times: “The more things change, the more they stay the same.” While this may be true in many cases, it certainly does not apply to the network operations teams of today.

The reality is that the nature of networks and network operations have been changing radically for several years, and the pace of change only seems to keep accelerating.

In years past, network teams had a data center and a few branch offices connected by a T1. By and large, they knew what was in their networks, and what was happening on them.

Now, the reality is fundamentally different. Here are just a few ways the nature of network operations has changed:

• Modern architectures introduced visibility gaps. Organizations around the world are now reliant upon a mix of SaaS offerings and private and public clouds. They rely on an array of ISPs and other third parties to handle the connections with these distributed environments. These network delivery paths introduce significant visibility gaps for network operations teams. Now, these teams may have to troubleshoot issues for a user, while only having visibility into a small fraction of that user’s network path.

• Dynamic and software defined environments increased complexity, alarm noise. Teams continue to employ new network technologies, such as software-defined wide area networks (SD-WAN) and software-defined data centers (SDDC). These technologies make on-the-fly decisions, for example on routing. Consequently, they present much more rapid change, and in the process, can create a lot of alarm noise.

• Business and security requirements intensified demands for scalability and collaboration. As businesses continue their digital transformations, the reliance on network connectivity keeps growing. In addition, in the wake of escalating security threats, organizations continue to employ advanced security technologies and approaches like zero-trust architectures. In these environments, more requests are being filtered through centralized access points, creating potential bottlenecks. This requires increased collaboration and information sharing between security teams and the network operations center.

For all these reasons, the old way of doing things is no longer sufficient.

Solution: Network Observability by Broadcom

Network Observability by Broadcom offers the advanced, comprehensive capabilities that enable teams to address the imperatives posed by modern networks. Following are some of the key advantages the solution delivers:

• Gain comprehensive, end-to-end visibility. The solution provides coverage of logs, flow, events, performance, telemetry, and more. For teams running SD-WAN, the solution offers complete coverage of SD-WAN metrics and it offers underlay and overlay correlation. The solution features active monitoring that can mimic different types of traffic. It can track a broad range of metrics, including capacity, throughput, jitter, voice loss, voice jitter, mean opinion score, and more. Further, the solution aggregates this intelligence, so teams can run sophisticated analytics.
• Eliminate traditional visibility gaps. Network Observability by Broadcom enables teams to gain complete visibility into all network delivery paths, including those that span third-party ISP and cloud environments. Operators can see the entire end-to-end path, including each network hop, even those paths that traverse environments that internal teams don’t manage. For example, an internal administrator can pinpoint the culprit device at a specific IP address in an ISP’s network.
• Promote standard operating procedures. With the solution, teams can move away from relying on disparate point tools and establish unified visibility of dynamic, multi-tech, and multi-vendor environments. This unified coverage is vital in enabling teams to establish standard operating procedures and improved consistency and operational efficiency.
• Harness continuous coverage. The solution can run tests every minute, while consuming minimal overhead. This gives teams an effective baseline and enables administrators to spot issues immediately.
• Leverage flexible implementation, coverage. With the solution, teams can leverage global monitoring points that can be used to test general availability. Teams can also deploy points in desired environments. The solution offers bi-directional support, enabling testing of download and upload performance over a given path.
• Reduce noise. Network Observability by Broadcom can minimize the noise teams have to sift through, ensuring they’re receiving timely, targeted, and actionable insights. The solution offers advanced noise reduction and alarm suppression, so, for example, thousands of events can be rolled up into one alarm.
• Access intuitive insights. The solution provides color-coded dashboards that enable administrators to see status and quickly determine if users’ experiences are affected. For example, if they view a dashboard with a red icon displayed, they know users are experiencing an issue. The solution features a network path overview page. Through this page, users can use a filter to view paths from specific monitoring points, whether a hardware device in a cloud provider or ISP environment, devices deployed on equipment in the organization’s premise, or some other elements in other environments. With the solution, administrators can click on any point in the path to view details on that specific system.

Benefits

By harnessing Network Observability by Broadcom, teams can realize these significant benefits:

• Gain granular visibility to speed triage, remediation, and escalation workflows. Network Observability by Broadcom provides all the data today’s network operations teams need, so they can work more intelligently, proactively, and quickly. Teams can see what the entire path looks like, from where the user is located to the destination they’re trying to reach. They can dive deeper, for example, into packet loss on WAN links. With this actionable visibility, teams can reduce mean time to repair.
• Speed mean time to innocence. When issues arise in external environments, internal teams can quickly identify that their internal systems aren’t the culprit, saving staff time. In addition, they can hold external service providers accountable for issues and SLA breaches.
• Boost team productivity. The solution equips level one and level two operators with insights to address more issues, without escalating to senior team members.

Example: Oil and Gas Company

For many network operations teams, there’s clear demarcation in time: before the pandemic and after. For these teams, virtually overnight, realities changed dramatically and irrevocably.  

The network operations team at a large oil and gas exploration company was no exception. The following sections offer a brief overview of the specific challenges the team faced, and how Network Observability by Broadcom was proven to help.

Challenge

In a matter of weeks, the network operations team had to move from supporting 2,000 remote employees to supporting 60,000. Given each user had their own Wi-Fi network, residential ISP, and so on, this in effect meant that the network operations team had to go from managing a few networks to managing tens of thousands.  

To ensure remote workers continued to have a seamless experience, network operations teams needed to make changes in a range of areas to accommodate new scalability demands. For example, before, they had limited numbers of VPN licenses and VPN concentrators, limited support for concurrent connections, and so on—and all these areas needed to be scaled substantially.  

Solution

Network Observability by Broadcom delivered the actionable insights that helped the team scale their operations and environments. The solution delivered a global view, so teams could see where users were, where VPN concentrators were located, and so on. Teams could also quickly pinpoint where issues were located, so they knew where to focus triage efforts. With the solution, teams could track the VPN user experience, see current connections, and assess how VPN concentrators were performing. They could also quickly track the health of interfaces and interface utilization metrics.

Results

Network Observability by Broadcom helped the IT operations group remain efficient and effective during a complex time with unprecedented challenges. Consequently, the solution played an essential role in keeping the business running effectively in spite of significant disruption.

With the solution, the team was able to ensure users had a quality experience, so they could continue to be productive in their jobs. This initiative was appreciated by staff and executives, and was the recipient of the organization’s annual award for most innovative IT project.

Since the early days of their implementation, they have continued to expand their use of Network Observability by Broadcom. The solution has fueled ongoing enhancements and has proven invaluable as the organization pursues its network observability strategy.

Conclusion

When the COVID pandemic emerged, network operations teams had to react rapidly to some fundamental changes—but the changes didn’t stop. Since then, the nature of networks and network operations have continued to see radical transformation. To thrive in these new realities, network operations teams need to establish advanced network observability.

To learn more, see our Small Bytes webcast, Scaling Network Delivery for 60k Remote Oil and Gas Workers. This webcast features a solution demonstration and it offers details on the oil and gas services company’s implementation. In addition, you can gain more information by reviewing the oil and gas services company case study.