SEE DEMO

Global Entertainment Company Saves Hundreds of IT Hours Every Year with
Forward Networks

Read below or download the case study >>

The Company

A near century-old entertainment and media company knows a few things about resilience, agility, and the importance of generating the right kinds of headlines while preventing the bad ones.

Every day, this company entertains enough guests to populate a city the size of Buffalo, New York. Almost every visitor is carrying a cell phone that could lead to any mishap going viral, making predictability key to their operation.

The behind-the-scenes logistics of running such an operation are staggeringly complex. Maintaining this environment takes several hundred highly trained technical experts, managing a system that is comprised of tens of thousands of networking devices spread across an area roughly the same size as Manhattan. This network has been evolving for nearly 50 years and includes devices from every major networking vendor and connects to multiple public clouds.

The Environment

Tens of thousands of networking devices
Millions of endpoints
Hundreds of IT professionals
Tens of thousands of acres

The Challenge

Over time and with the introduction of new technologies, the network had become difficult to manage. Seemingly simple tasks like locating devices and conducting path searches took prohibitively long. Like many companies with large networks, they lacked an accurate, in-depth understanding of network topology. These limitations inhibited efforts to modernize the network and caused an unnecessary number of expensive outages.
The goals of the technical leadership team included:

Achieving a better understanding of the network to support modernization efforts.

Increasing the overall efficiency of the IT department by automating laborious and time-consuming manual tasks.

Continuously ensuring that the network configurations remain in-policy.

The Solution

The team turned to Forward Networks to help them achieve their goals. Specifically, they wanted to automate path searches and reduce Mean-Time-to-Resolution (MTTR) while enhancing situational awareness. The Forward Networks feature set easily delivered on these goals while creating a path for further modernization and optimization of the network.

Achieving a Better Understanding
of the Network

The first step in utilizing Forward Enterprise is setting the software to automatically scan the network to build a digital twin of all devices, their state, and configuration. This collection is used
to create a visual topology of the network and provides the information needed by the mathematical model to calculate all
possible paths a packet could take.

The Network Query Engine (NQE) ingests the data collected by the Forward Enterprise platform, effectively turning the network into a database. Using plain language, engineers can search the network and extract detailed, actionable data for problem-solving and decision-making. According to one user, “NQE queries are solving problems that we couldn’t tackle before. We are much more
efficient now and can take on projects that were impossible before because we have better knowledge of our network. I can’t wait to explore new use cases.”

Based on internal calculations across four regular maintenance tasks, a minuscule subset of the overall project scope, NQE
reduced the time it would have taken to complete the tasks (e.g. understanding device utilization, finding mismatched timers, understanding port utilization) from over 700 person-hours in a year to just under 60 seconds in total.

The path search technology in Forward Enterprise reduced the amount of time spent on search and troubleshooting tasks by half, according to the company’s internal calculations. One of them engineers remarked, “I’m not sure if Forward has made me lazy or smart, but I don’t try to troubleshoot on my own anymore; I go straight to the platform for immediate answers.”

Increasing Overall Efficiency

Another main goal of the company was to increase the efficiency of the IT team by automating tasks. Forward Enterprise is helping here as well. As part of its annual maintenance, the company tests its emergency power by shutting down the electricity and checking that backup sources such as UPS and generators are working.

The IT department is tasked with verifying the state of all network devices before the test, while the main power source is cut, and after it’s restored. This manual process took a considerable number of people and a significant amount of time. In the event a device failed to come back online, or returned in a “hung” state, the only solution was physically deploying engineers on an expensive scavenger hunt throughout the location. Leveraging NQE nhelps them quickly identify devices that don’t come back online. Technicians are no longer going in search of devices that need their attention; they can go directly to an impacted device. The customer calls this a “game-changer” for their team.

Continuously Ensuring the Network is In Policy

Like most large enterprises, the customer struggled with ensuring network compliance in an environment that was constantly undergoing changes and upgrades. Given its size and the number
of engineers working within the environment, it would have been
impossible to ensure that all updates were in policy and would not cause performance or security issues. It would also be nearly
impossible to determine what changes caused an incident after the fact. When hundreds of thousands of customers depend on the network, this is an untenable situation.

Using the custom verification feature in Forward Enterprise, administrators configured regular checks of the network to search for misconfigurations, lack of connectivity, or lack of redundancy. Being able to run these checks allows the team to proactively identify and remediate non-compliant issues before they cause an
incident.

This capability helped the team prevent two potential outages which could have generated unwanted media attention. In both cases, the verification checks identified missing redundancy; had the main path failed without a backup in place, a brandnew attraction would have shut down and a park-wide evening performance would have been canceled.

Another feature the team finds extremely useful is behavioral diffs. When the network runs checks to verify compliance and device configuration and state, it saves a snapshot. The behavioral diffs function allows engineers to easily compare two snapshots and see exactly what changed in a given time frame. This makes it very easy
to find the causes of unexpected network behavior.

Looking Forward

There are so many potential use cases for the Forward Networks platform that the company held a hackathon to find ways to extract value in the Fall of 2021. Many of the ideas brought to light during that event are already in production, and they plan to continue co-hosting this event with the Forward Networks engineering leadership.

Another opportunity they are exploring is replacing an aging network verification system with custom NQE checks. This will free up engineering resources previously dedicated to maintaining this system and provide additional features they didn’t have the resources to develop.

Lastly, they are starting to take advantage of Forward’s hybrid cloud functionality, which extends all the features of Forward Enterprise to AWS, GCP, and Azure. This provides additional value by delivering hop-by-hop visibility from the point of origin through the clouds and to the Internet. The company anticipates cost savings and enhanced security policy enforcement from having the ability to view their entire hybrid, multi-cloud network in a single pane of glass with the ability to drill down to individual network devices and cloud object instances.

Results of embracing the Forward Networks Platform

Save 700 hours per year with NQE
Reduce time spent troubleshooting by 50%
Avoid P1 outages
Ensure accurate network inventory

Download and share this case study

Interested in learning more?

Explore these additional resources to see how the Forward platform can help you with your network needs.

Meet with our technical team

Have specific networking, security, and cloud concerns? Our team can tailor a demo of the Forward platform to address them and your questions.
Request a demo

How much would you save with Forward Enterprise?

Ticket resolution, audit-related changes, and labor-intensive change windows - they all add up. Forward Enterprise saves companies millions every year.
Caluclate your ROI

Sign up for our newsletter

downloadcrossmenu linkedin facebook pinterest youtube rss twitter instagram facebook-blank rss-blank linkedin-blank pinterest youtube twitter instagram