Did you hear about the change window that went exactly as planned? No? That’s because the odds of winning the PowerBall without buying a ticket are better than the odds of executing a change window on a global network without a glitch.
What about the story of the tier one network engineer that diagnosed and resolved an ACL in seconds? That one also seems as mythical as staying friends with your ex—but it’s not.
Instead of telling you the story, I want to show you how it’s done, which is why I recently hosted a workshop showcasing how we use search and intent verification within the Forward Networks Platform to tame ACLs (Access Control Lists).
I’ve spent untold hours trying to troubleshoot an ACL issue after a change window and that was on a network I’d been running for decades, for tier-one admin, or even a more advanced engineer working on a new (or newly blended) network, it’s like trying to find a needle in a haystack while wearing a blindfold and being chased by rabid badgers.
On the face of it, the process for resolving ACL issues is pretty straightforward:
- Determine where your ACLs are running (which interfaces)
- Locate the ACL creating the issue
- Analyze the ACL to find the problem and resolve the issue
Except—networks have evolved over decades and include tens of thousands of devices from dozens of vendors and cloud providers running billions of lines of config. The fact is network complexity is outpacing IT support capabilities. Today, nothing about running a global network is straightforward without a comprehensive understanding of the network’s behavior and detailed visualization of traffic paths.
Managing ACLs shouldn’t be that hard
At Forward Networks, we think that the hard stuff should be easy, so we’ve done something unique. We developed a mathematical model that creates a network digital twin with Google-like search capabilities. By collecting and analyzing device state and packet forwarding data over time, we provide more than network visualization – we put the humans back in control of the network by providing them synthesized, actionable insights around network behavior.
The Morning After the Change Window Before
The call comes in—a user can’t access an application – or worse, unauthorized users are accessing a secure app. What to do? The network team always gets the call first, but the firewall tribe and security squad were also making changes – so how do you know which change created the problem?
The Forward Networks Platform (which functions as SaaS or be loaded onto an on-site VM) collects snapshots of the network over time including state data (ARP tables, route tables, interface tables, and so on) to develop a behavioral model of the network, providing detailed information on how packets are forwarded, filtered, and mutated. The end result is not only detailed visualization of the network but also advanced behavior modeling. For the ACL workshop, I focused on two ways to solve the issue, search and intent verification.
Search Two Ways: Text and Behavioral Path
Wouldn’t it be great if your network was indexed the same way the Internet is, and you could search it as easily as using Google? Ima ‘bout to rock your world by doing it right in front of your eyes.
Maybe you only know the IP address of a device that’s misbehaving. Our text search bar lets you enter that IP address (or any other atomic network information) and instantly gives you everything you need to know about that device (including which ACL rules/policies are applied to it). Maybe you want to search by ACL names—you can do that as well, and the platform returns config information with the ACL-related lines highlighted. This is ridiculously helpful when firewall configs have tens of thousands of lines. Now, even Tier-one support engineers can diagnose the problem and route it to the correct team with the context they need to immediately resolve the issue—no more searching manuals or paging through thousands of lines of config.
By conducting a behavioral path search from the Internet to a specific application, you can see the exact path(s) traffic takes to the application in blue. The gray lines denote detailed information about what happens to the packets as they flow through the network and the functions that are applied to them which is explained in the path’s pane. The platform serves up the relevant information without the network admin having to know details about the firewall or its syntax. The search shown above tells us that there is a path, and helps us easily identify that there are issues are with the firewall config, saving tons of time (conversely, it would tell us if the network path is broken).
Behavioral searches can be saved as expected behaviors (intents) so that anytime the platform gathers information about the network, it will confirm that path is working as expected. In the workshop, I show how this function also can be used to verify if the “fix” applied by our friends in the tribe of firewall worked as expected (spoiler—it didn’t but network operations saves the day) without any risk to the production network, by using the predictive capabilities of the platform within the network digital twin.
NQE – Your ACL management BFF
In-App NQE (Network Query Engine), checks the data collected from the network and looks for states in the network that should (or should not) exist. For instance, an NQE Check can look for ACLs that are defined on a device but not applied to an interface. Custom checks can be written from inside the browser using syntax within the browser. There’s nothing to download; all of the reference information such as the data model and documentation is available within the browser window. This is a much better way to roll than my days of custom coding queries trying to pull information from the dozens of tabs I’ve opened to write code in the past.
Sound interesting? Watch the full ACL workshop (30 minutes of live-demo content). We host Forward Fix Live every month – On April 21, 2020 we’re going to dive deeper into one of our most popular features—NQE. There are two sessions, so no matter what time zone you are in! one for the East Coast and one
Only have a few minutes but you want to see more content by engineers for engineers? Check out our YouTube playlist Forward Fixes – no hype, just actionable information, in roughly five-minute chunks.