Query Your Network like a Database

by Fabrizio Maccioni

As companies are trying to become more agile to provide new services to their customers at a faster pace, more and more of them are embracing Network Automation in some shape or form.

Network Automation can be very powerful, but it comes with a price: it’s frighteningly dangerous! A mistake in the automation toolchain can bring down an entire organization! Well, you might argue that a mistake made by an engineer typing the wrong command in a single device SSH session can have a similar effect. That’s actually true, but usually the impact of a change made via automation can be an order of magnitude bigger than a manual change as it can potentially propagate the same error across each and every device in the entire network. Think about changing or deleting the wrong VLAN ID…scary, uh?

On the other hand, how much time would it take to make the same change manually, device by device? Automation is the only way possible to apply quick changes to a dynamic network environment.

Lately we hear more and more about Intent-Based Networking (IBN), from leading research and advisory companies, as well as incumbent networking vendors. The main idea behind IBN is to define how the network should behave at a high level without providing any detail about the implementation, like device vendor and related specific configurations. The idea is actually not entirely new. If you take the configuration management tools as an example, they have been using the same concept for applications, services, and networks for more than a decade now.

IBN is all about automation! A well-defined IBN solution should provide automation for both Network Configuration and Verification.

While there are already several very good network configuration automation solutions in the market, there are not as many options for feature-rich, scalable, accurate and open Network Verification solutions.

Why? The answer is very simple. It’s freaking hard to build one!

These are the usual steps involved in implementing any sort of network verification:

    • Collect the configuration and state from every device in the network
    • Parse all the data collected
    • Normalize the data coming from all the different sources
    • Build verification checks based on the normalized data

Whereas the collection phase is usually straightforward, it needs a well thought out architecture to make it scalable and fast.

Parsing and normalizing the data are by far the most challenging and time-consuming phases! Having to deal with a broad range of devices like switches, routers, load balancers and firewalls, across many different vendors, usually running different OS versions, on-premises and in the cloud, poses a very significant burden for engineers.

Usually, the data collected is plain text from the device configuration and several “show commands” outputs to gather the device state. As such, the data is completely unstructured making the parsing extremely complicated especially for the device state due to the lack of proper documentation and constant, unpredictable change from one software release to another.

What about device-level APIs? NETCONF? YANG? OpenConfig?

The very reason why the entire networking industry is moving toward standard-based network APIs, is exactly to provide well-defined structure data to make the device configuration and state parsing easier. The final goal is to enable faster and easier configuration and verification automation. That’s the future, no doubts about it, with YANG as a data modeling language and OpenConfig as the vendor-agnostic data model definition likely to be predominant in the mark for the foreseeable feature.

The reality is that, as of today, network device APIs are available on a limited number of devices, on bleeding edge software releases, very often covering only a subset of the features with very limited support for vendor and OS-agnostic data models like OpenConfig. This unfortunately forces the engineers to stick with CLI outputs only.

What if the data was already collected, parsed, normalized and made available in a similar fashion as OpenConfig? What if you could query your network like a Database? That would be a dream coming true for most of the engineers, right? This would allow the network teams to focus on the higher-level aspects of their use cases, which is actually making their networks more resilient, agile and robust without spending time writing collectors and parsers. That would be just amazing!!

Well, that’s exactly what the Network Query Engine (NQE) from Forward Networks does! Specifically, it provides an open platform for accessing structured data about the network as JSON data, in a fully-parsed form. Moreover, the information is normalized, it is presented uniformly across all the supported vendors in a scalable, easy, maintainable way. This means that the same sanity, verification and documentation checks can work across many different devices and vendors.

To make NQE easy to consume at scale, Forward Networks has made all the data queryable through a GraphQL API. GraphQL is a flexible data query language developed by Facebook in 2012 and released as an open-source project in 2015. Hundreds of organizations are already leveraging GraphQL.

Moreover, NQE is aligned with OpenConfig. It’s not literally the same data model as any of the various JSON representations of the OpenConfig YANG data models. Rather, Forward Networks NQE can be seen as an idiomatic representation of OpenConfig as a GraphQL data source. It sounds complicated but it’s not. In NQE the OpenConfig YANG data models have been modified to fit within the constraints of GraphQL and to take advantage of the powerful query features available in GraphQL. To give you an example, names are camel-cased, dashes are not permitted in GraphQL.

To get started with Forward’s NQE, read about it on this blog and then head over to the NQE GitHub repository. The repository shows you how to install the client library, provides a set of examples that you can use to bootstrap, and covers a variety of details about the API.

Happy Network Verification Automation with Forward Networks NQE!