3 Steps to Get DX NetOps Events in Slack and Google Chat

Network operations centers (NOCs) play a critical role in any organization’s operational and business continuity. To meet their vital charters, NOC teams must constantly strive to maintain uninterrupted network availability and to minimize the business impact of network issues.

Within the NOC, effective collaboration is essential for quick troubleshooting and resolution of network issues. Today, with chat and instant messaging solutions, this collaboration can happen in real time between departments, vendors, customers, and team members.

These collaboration platforms provide a number of key benefits:

• They enable easy integration with voice assistants, chatbots, ticketing systems, dashboards, and alerting applications.
• They can produce KPIs and analytics to measure the duration of incidents and the volume of alarms handled by time, operator, and engagement level.
• They provide segregation of duties by group chats or channels, based on projects or areas.
• They offer filtering for efficiency and noise reduction.

These collaboration tools have become so popular that many organizations use them as the primary interaction point for their NOC.

This practitioner blog will detail how to integrate DX NetOps by Broadcom with two collaboration tools: Slack and Google Chat. Through this integration, your organization can benefit from real-time notifications on network events that are having an impact on your business.

1. Configure a new AlarmNotifier instance

In this first step, it is important to have a clear understanding of the target audience and which events we want to integrate with our collaborative tools.

You will need to know in advance which channel or group from Slack or Google Chat is going to receive the notifications, and the minimum amount of information that must be shared in those messages.

Remember, do not overload the message with too many fields. Often, it is sufficient to add a hyperlink to the originating solution (DX NetOps in this case) and only send data from key fields, such as severity, impact, troubleshooter, alarm message, and business/service impacted.

You will start by configuring a new instance of the AlarmNotifier, the DX NetOps tool that acts like a gateway for outbound integrations. All the following steps will take place in Spectro Server:

1. Make a copy of$SPECROOT/lib/SDPM/partslist/ALARMNOTIFIER.idband rename it to indicate the integration type (i.e. ALARMNOTIFIER-GCHAT.idb or ALARMNOTIFIER-SLACK.idb).
2. Edit the copied file and update the variables below (This example is for Google Chat, but you can simply replace “gchat” with “slack” if you are setting up Slack integration):
   PARTNAME;ALARMNOTIFIER-GCHAT;
APPNAME;AlarmNotifier-gchat;
WORKPATH;$SPECROOT/Notifier;
LOGNAMEPATH;$WORKPATH/NOTIFIER-GCHAT.OUT;
ARGV;$SPECROOT/Notifier/AlarmNotifier<CSEXE> -r $WORKPATH/.alarmrc-gchat -n AlarmNotifier-gchat;
3. Make a copy of $SPECROOT/Notifier/.alarmrc and rename it to indicate the integration that is being supported (i.e. .alarmrc_gchat or .alarmrc_slack).
4. Edit the copied file and update the variables below:
   EXTRA_ATTRS_AS_ENVVARS=0x11F57,0x12022,0x12bf7,0x12bf6
SET_SCRIPT=/usr/Spectrum/Notifier/SetScript-gchat
CLEAR_SCRIPT=/usr/Spectrum/Notifier/ClearScript-gchat
UPDATE_SCRIPT=/usr/Spectrum/Notifier/UpdateScript-gchat
APPLICATION=AlarmNotifier-gchat
   Note we are adding four extra attributes 0x11F57 as trouble_shooter, 0x12022 as trouble_ticket_id, 0x12bf7 as ServiceImpact, and 0x12bf6 as CustomerImpact. We have also specified and created custom Set, Clear, and Update scripts.

To recap, on this step we have created a parts list for each integration:

[root@oanoss1 partslist]# ls -lrt ALARM*
-rw-r--r--. 1 root root 435 May  3 20:12 ALARMNOTIFIER.idb
-rw-r--r--. 1 root root 473 May  3 20:12 ALARMNOTIFIER-SLACK.idb
-rw-r-----. 1 root root 453 Jul 10 11:27 ALARMNOTIFIER-GCHAT.idb

And a config .alarmrc file for each integration too:

[root@oanoss1 Notifier]# ls -lrt .a*
-rw-r-----. 1 admin admin 772 May  3 20:11 .alarmrc
-rw-r-----. 1 admin admin 772 Nov 12  2022 .alarmrc-slack
-rw-r-----. 1 admin admin 805 Jul 10 11:42 .alarmrc-gchat

These steps are available out of the box and additional information can be found in the official documentation.

2. Define SetScript to invoke the integration

This is the most important step of the integration process. Here, we will define the action (API call in this case) and the data shared (API payload) between DX NetOps and the collaboration solution.

We need to define this information in the SetScript that will be executed to invoke the integration. If not done yet, make a copy of the SetScript, ClearScript, and UpdateScript that are stored in $SPECROOT/Notifier folder (i.e name them like SetScript-gchat or SetScript-slack).

Now, edit the custom SetScript and make the following changes:

1. Add the variable ENABLEINTEGRATION right below the SENDEMAIL variable and set it to TRUE. 
   ENABLEINTEGRATION=True             #True or False
2. Add the troubleshooter, trouble ticket, ServiceImpact, and CustomerImpact fields to the list of attributes consumed by SetScript:
    echo "TroubleShooter:  " $SANM_0x11F57
 echo "TroubleTicket:   " $SANM_0x12022
 echo "ServiceImpact:   " $SANM_0x12bf7
 echo "CustomerImpact:  " $SANM_0x12bf6
3. Note that each collaborative tool (Slack or GChat) will require a different payload to create a message. Review Slack and Google documentation for information on how to enrich or modify the webhook payload. We provide two examples below. Add this section in the end of the SetScript for Slack:
   if [ "$ENABLEINTEGRATION" = "True" ]
then
  SUMMARY=`echo "$PCAUSE" | head -1 -`

  curl -X POST   'https://hooks.slack.com/services/<SLACK-CHANNEL-ENDPOINT>' \
  --header 'Content-Type: application/json' \
  --data '{"channel": "#netops-openaccess-alarms", "username":  "NetOps Notification","text": ":broadcom: *Message:* <http://netops.forwardinc.biz:8181/pc/desktop/page?mn=2000193&pg=2000156&parentid=2000184&GroupID=1&GroupPathIDs=1|'"$SUMMARY"'>\n*Date:* '"$DATE"' '"$TIME"'\n*Severity:* '$SEV'\n*Device:* '"$MNAME"'\n*Troubleshooter:* '"$SANM_0x11F57"'\n*Service Impacted:* '"$SANM_0x12bf7"'\n*Customer Impacted:*'"$SANM_0x12bf6"'\n", "icon_emoji": ":broadcom:"}'

else
   echo_info
fi
   Basically, if integration is enabled, it will trigger a curl POST to a Slack endpoint to receive messages. Note that fields in red are subject to change based on your environment, while $SUMMARY, $DATE, $TIME, $SEV, $MNAME, $SANM_0x11F57, $SANM_0x12bf7 and $SANM_0x12bf6 are alarm fields from DX NetOps. The Slack message received with this payload is displayed in the following image:

Note we have decorated it with some icons and custom formatting, and included hyperlinks to the DX NetOps Alarm Console.

For Google Chat integration, just change the curl payload to the below example. Again, pay attention to fields in red, which must be customized:
   curl -X POST     'https://chat.googleapis.com/v1/spaces/<GCHAT-GROUP-ENDPOINT>' \
--header 'Content-Type: application/json' \
--data '{"text": "\u26a0\ufe0f *Message:* <http://netops.forwardinc.biz:8181/pc/desktop/page?mn=2000193&pg=2000156&parentid=2000184&GroupID=1&GroupPathIDs=1|'"$SUMMARY"'>\n*Date:* '"$DATE"' '"$TIME"'\n*Severity:* '"$SEV"'\n*Device:* '"$MNAME"'\n*Troubleshooter:* '"$SANM_0x11F57"'\n*Service Impacted:* '"$SANM_0x12bf7"'\n*Customer Impacted:*'"$SANM_0x12bf6"'"}'

This payload will produce a message in Google Chat similar to the following image:

Next, we are ready to move to the final step, where we will decide which alarms will be sent as notifications to the collaboration tool.

3. Associate a policy with your AlarmNotifier instance

The final step is to decide which DX NetOps alarms will be sent to Slack, Google Chat, or both for subsequent processing or action. At this point, our recommendation is to start small, measure the effectiveness of the integration, and iterate to keep improving.

If no policy to filter alarms has been created, you might want to create a new one by following these steps. Basically, you will need to decide which alarms will be processed and sent over to Google Chat or Slack. Then, you can just link the AlarmNotifier instance created in previous steps to the defined policy, as shown below:

Moving forward, the alarms matching the policy will be sent as messages to your collaboration tool.

Next steps

We hope this blog has proven valuable as you look to integrate DX NetOps with your messaging solution. Surfacing the affected customer and business service in Google Chat or Slack messages is extremely useful in prioritizing and routing these notifications. Through this approach, you can enable your team to collaborate more effectively and react faster to network events.

As next steps, there are multiple ways to take your integration to the next level. Here are some ideas:

• Integrate Google Chat/Slack messages with a chatbot, voice assistant, or a generative AI solution to enrich alarms and facilitate the work of NOC operators.
• Enrich the Google Chat or Slack messages with hyperlinks to CMDBs, to alarm actions (clear, escalate...), to run a knowledgebase search, or even to execute a health check on the affected device.
• Enrich tickets with the troubleshooting thread from Google Chat or Slack.

Get to know more about DX NetOps by Broadcom and how it can help your business to optimize the new enterprise network.