Troubleshooting Microsoft Teams Performance Issues

When it comes to the relative merits of hybrid work approaches, there can be a range of opinions; some are strong advocates, some see significant downsides, and many fall somewhere in between. About the one thing pretty much everyone can agree upon is that the rise in hybrid work has made the jobs of IT operations teams much more challenging.

On any given day, a user may be working in the office, from their home kitchen, a neighborhood café, or just about anywhere else. Even when employees are in the office, they may be using shared desks, and they may opt to join virtual meetings from their office desk rather than congregating in a conference room.

The upshot of these modern approaches and environments is that users are now reliant upon a complex mix of networks—including ISP, cloud, Wi-Fi, and others—that IT doesn’t have visibility into or control over. Further, various on-premises networks may be supporting many more simultaneous virtual conferences and calls than they were originally designed to.

Consequently, it keeps getting more difficult to manage service levels and troubleshoot issues. These challenges are particularly pronounced when it comes to real-time communication apps like Microsoft Teams. Today, many organizations and users are employing Teams. In 2022, more than 270 million users relied upon Microsoft Teams, a number that grew from 20 million in 2019.

Compared to other employee apps, performance and availability of Teams is even more critical. While a user may not notice that emails are taking longer to load, they’ll absolutely notice if a connection to a Teams meeting is dropping packets.

Challenges: monitoring a black box

In effect, when trying to track performance, Teams can present administrators with a black box. In some ways, this is by design. Microsoft has taken steps to limit external teams’ ability to monitor specific systems, which is understandable as this monitoring traffic can affect how services perform.

Further, visibility is also clouded by the globally distributed Azure network. Within the Microsoft global network, there are around 40 different regions that can host Teams calls. For a typical call or virtual conference, the host network will be the one that the first user connects to. Subsequent users will initially be connected to the point of presence (PoP) that has the lowest latency, and that PoP will direct them to the host network.

To monitor Teams performance, teams are given an Anycast IP address. (Anycast is a technology that enables multiple routing paths to a group of endpoints, which are assigned the same IP address.) While tracking the performance of this Anycast IP address will provide some idea of performance, it won’t register performance of the backbone or of the full round trip response time users are actually experiencing. This approach also will not take into account the specific PoP routes a users’ call will take.

These environments result in IT teams having a lack of ownership and visibility over availability and connectivity associated with their users’ critical business apps. This creates a number of challenges. When issues arise, administrators struggle to determine the scope and impact and they have a hard time identifying the specific domain in which the problem is occurring. When users encounter issues, their productivity gets diminished, and IT teams struggle to troubleshoot what’s happening and where.

Solution

The team at Broadcom has worked closely with Microsoft and Teams customers to establish effective Teams monitoring. AppNeta by Broadcom offers differentiated monitoring coverage that enables more effective, efficient monitoring of Teams services.

With its patented TruPath technology, AppNeta can measure performance every minute from every location. The solution offers granular visibility and enables objective, metric-based comparisons across regions and users. 
AppNeta can provide a complete view of performance, both upload and download. It provides visibility into the end-to-end delivery path, including traffic running on the WAN, ISP, and Wi-Fi networks that are used to get to the Azure region. This visibility is vital in enabling more intelligent analysis and troubleshooting.

AppNeta give teams the concrete data they need to identify whether the issue is arising in the Teams environment, versus elsewhere in the network or application delivery path.

Flexible, automated views of Azure regions

AppNeta has Global Monitoring Targets (GMTs) that are located in 30 cities that host Azure regions. These GMTs are highly available. Further, they enable teams to establish dynamic targets. With the solution, operators can automatically target the Azure region that a user would access, including those they’d connect through to get to a host PoP.

Based on where a user is, AppNeta can test connectivity to their lowest-latency region, and determine full end-to-end performance of interactions. This is important to underscore: With AppNeta, teams can track performance delivered by the specific region that’s actually doing the hosting, versus just tracking the standard Anycast IP address or the region the user initially connects to.

Metrics coverage

With AppNeta, administrators can track essential metrics, including:

• Capacity. With AppNeta, operators can monitor capacity to ensure there’s adequate bandwidth to handle the extra load associated with Teams traffic.
• Two-way latency. This metric is vital in enabling administrators to understand when differences between ingress and egress traffic may be to blame for performance issues.
• Voice-specific loss and jitter. Data and voice packets behave differently over the wire. AppNeta enables teams to track voice traffic, which has distinct requirements and characteristics. The solution sends real voice packets to measure voice performance. This helps administrators identify when delays and spikes are occurring.
• Mean opinion score (MOS). While tolerances can vary by region, AppNeta’s ability to track MOS helps teams establish a uniform baseline across global environments.

Network and App Views

Network

With AppNeta, operators can track multiple network paths and filter their views. For example, they can focus on a subset of the environment, such as a specific region or user location.

An example of an application quality dashboard, which helps administrators quickly determine which locations are experiencing issues.

At a glance, administrators can view hop-by-hop transmissions, so they can quickly identify which area is experiencing an issue. For IT teams, this means they can immediately determine whether their environment is responsible, and, if not, follow up with third-party service providers or partners, and determine potential workarounds.

Administrators can also easily drill down into specific areas of a network path. The solution will trigger diagnostics, which provide hop-by-hop details, gathering as much information as possible from the time an issue occurred.

Application

With AppNeta, administrators can do synthetic testing of Teams services. AppNeta features capabilities for running scripts based on Selenium. The solution offers basic, pre-packaged scripts, such as for issuing a “get post” command. With the solution, teams can also do deeper synthetic interactions, employing milestones.

This ability to get more detailed visibility is vital. For example, simply doing testing of a login function may not be enough. This functionality could rely upon a completely different service and infrastructure than other parts of the application. This means login functionality may be working fine, but other major capabilities may be completely unresponsive.

Alternatively, if there’s only an issue with the login functionality, users already logged in may not be experiencing any issues. With AppNeta’s scripting capabilities, teams can test all the different interactions users may have, and track their performance over time.

Advantages

By employing AppNeta in Teams environments, network operations staff can:

• Gain visibility from the end-user perspective.
• Quickly isolate the error domain to speed mean time to innocence (MTTI) and mean time to resolution (MTTR).

Ultimately, with AppNeta, teams can be much more proactive. They can find out about issues with third-party networks immediately, and provide users with workarounds for known issues. For example, they can guide employees to use a different messaging service if there’s an issue with Teams or suggest they reschedule an upcoming meeting. This visibility and approach can strengthen IT operations’ support of the organization and boost employees’ confidence in IT.

Watch our presentation to learn more

To learn more, be sure to watch our Small Bytes presentation How do I troubleshoot Microsoft Teams issues for my work from home users? Find out how to establish the monitoring coverage your teams need to be successful with your Microsoft Teams usage. Find out how to quickly spot the cause of issues, no matter where users are based or which networks they’re relying upon.