Multicast IP Routing protocols have been increasingly popular over the last several years. 

They are used to efficiently distribute data to a group of destination hosts simultaneously in a one-to-many or many-to-many model

Typical examples of applications where the use of multicast is very common are audio/video streaming services (e.g content delivery networks, IPTV) and financial trading platforms (e.g Stock Exchanges). 

This blog is not intended to be a multicast tutorial but, but it aims to showcase how Forward can map and search multicast-enabled networks. Before going there, just a quick recap, at a very high level, of the multicast components: 

Putting all together, a Multicast Client signals that it’s interested in receiving a multicast stream identified by a Multicast IP address using a Group Management Protocol (e.g. IGMPv3). 

Multicast Source sends a multicast stream of data to a Multicast Group and the Multicast Routers forward the multicast streams to the Multicast Clients using a Multicast Routing Protocol (e.g. PIM-SM). 

If your background is more on applications, the concept is similar to a Pub/Sub (Publisher/Subscriber) messaging service where the Publishers in multicast are the Multicast Sources and the Subscribers are the Multicast Clients. 

Easy right? 

Well, the concept might be easy, and the basic configuration can be fairly straightforward but when something doesn’t work it’s a nightmare to troubleshoot a multicast network. 

This is where Forward Network can help you make, one more time, your life way easier! 

Forward Enterprise allows you to get full visibility and analysis of your multicast network by searching for multicast insights and performing multicast path analysis in the Forward Search application. 

Search for multicast group 

To search for multicast groups, all you have to do is navigate to the Search application and type the multicast group IP in the search bar. 

Forward Search automatically recognizes the IP address as a Multicast group. 

Let’s dive into the info provided by the Forward platform for the given multicast group: 

The Search Results pane provides a Multicast group category where you can find information about the Rendezvous Point name and IP address, the VRF where the group is configured in and direct links to the Rendezvous Point configuration and state for the given multicast group. 

The Multicast Group Card in the middle pane provides an at-a-glance view of all the group members as well as direct links to the configuration and state for the given multicast group for each member. 

The Topology map on the right is context-aware and highlights the devices relevant to the content you are navigating, like the Rendezvous Point, the devices in the VRF where the RP is configured, and specific members. 

You can also search multicast groups by subnet. 

In this case, all the matching multicast groups are shown in the Search Results along with any RP information. At this point you can select one multicast group to get more information as in the previous example. 


Multicast Path Analysis 

There are several IP Multicast protocols but the most commonly used are Protocol-Independent Multicast – Sparse Mode (or PIM-SM) at L3 and IGMPv3 at L2. 

With PIM-SM, multicast traffic flows on primarily two types of routes: (S,G) or s-coma-g and (*,G) or star-coma-g.
The (S,G) routes flow from a specific Multicast Source to a Multicast Group while traffic on a (*,G) route flows from any Multicast Source to the RP and from the RP to all the Multicast Clients of group G. 

Forward Enterprise provides PIM-SM Control Plane Modeling to compute complete (*, G) and (S, G) trees and show them in the Forward Enterprise topology along with the tree’s details. 

For instance, you can do a (*, G) path analysis to find out all the paths between the RP and the potential Receivers by issuing a query from the multicast group IP with Destination IP the multicast group IP: 

You can see all the different paths from the RP of the multicast group destined to the group members using the Paths selector and see hop by hop summary information as well as specific device state at Layer 2 and Layer 3. 

To perform a (S,G) path analysis, simply provide the multicast sender IP address and the multicast group IP address. 

In this case, the path analysis provides not only all the paths between the Source and the Receivers at a data plane level as in a common Path Analysis for unicast traffic, but it provides a Control Plane validation for each path as well. 

If both indicators in the Control plane validation section are green, it means that Forward Enterprise didn’t find any issue in the Control Plane, neither from the Sender to the RP nor from the Receivers to the RP, as you would expect. If either of the Control Plane validations are shown in red, it means Forward Enterprise has found some issues and it provides some insight that can help you find  whether connectivity exists from the sender to the RP and from the RP to the receiver. 

It also shows you what interface would be selected in case a receiver will show up based on the Reverse-Path Forwarding (RPF) for the given Sender and a brief explanation: 

As usual, more capabilities will be added in the future to make it even easier to work with your Multicast network, so stay tuned and, in the meantime, watch the demo video above and happy multicasting with Forward Networks!!! 

Here we are with yet another blog on the Forward Network Query Engine (or NQE for short).

If you have been reading our previous blogs on this topic, you already know how passionate I am about NQE.

In my first blog Query Your Network like a Database, I talked about how NQE helps to solve common challenges in network automation when it comes to retrieving network device configuration and state to verify the network posture, especially in large networks comprised of many different vendors, technologies, spread across on-prem and cloud deployments.

In a subsequent blog, In-App NQE Checks Examples Now Available on GitHub I described how In-App NQE Checks helps build network verification checks based on the NQE data entirely in the Forward Enterprise user interface, and I’ve introduced a GitHub repository with some examples.

If you haven’t read those blogs and you are not familiar with NQE, well, you might want to do some reading on the topic before coming back here 🙂 

Still (or back) here? Great!

In this blog, I’m going to talk about a big improvement we have made in our latest release, the NQE Library.

Many of our customers are enthusiastic about In-App NQE Checks. They say it’s very easy to find violations to their network intent using the intuitive language, the built-in documentation, data model schema based on OpenConfig, the provided examples, and so on.

As it frequently happens, the more customers use a product extensively, the more use cases come up.

One of the use cases that came up from several NQE users has been:

“In some scenarios, we are not looking for violations [yet] but network insights instead.
Can we do that with NQE?”

Now you can with the NQE Library!

At a very high level, we have decoupled the NQE queries from the NQE Checks to enable the new use case (find network insights) while preserving the original use case (find network violations).

In a nutshell, the NQE Library allows you to easily create and organize collections of NQE queries.

The NQE Library workflow consists of the following steps:

  1. Create a query
  2. Commit it
  3. Use it in Verify optional

As shown in the NQE Library page below:

Fig 1. NQE Library Workflow

Create a query

To simplify the creation of NQE queries we have built the NQE Integrated Development Environment (IDE).

Fig 2. NQE Integrated Development Environment (IDE)

It consists of 4 different panes:

All the panes are collapsable and resizable to allow you to manage the screen space more efficiently.

If you are familiar with the IDE we built for the In-App NQE checks you will notice that the biggest difference is the introduction of the Files pane to organize the queries.

The easiest way to get started is by using one of the examples in the Help pane.

For instance, the first check can be used to find every interface whose admin status is UP but operational status is not UP.

The query iterates through all interfaces on all devices of any vendor and returns the device name, the interface name, the admin state,  the operational state, and finally, the violation field is set to true if the admin state is UP but the operational state is down for the given interface.

Fig 3. Edit query

Simply copy the example of your choice by clicking on the Copy query icon, paste it in the Editor pane and optionally click on Prettify to properly align all the lines in the query to make it more readable.

The NQE Editor supports many useful features like auto-completionauto-save and, automatic error fix suggestions based on the Data Model among the others.

When you are done editing a query, select the Network and Snapshot you want to run the NQE query against and click on Calculate to see the query result.

Fig 4. Query results

Commit it

All the changes made to a query are automatically saved but they are visible only to you in the NQE Library application.

You need to commit the NQE query to make them available to everybody in your organization as well as to be used as NQE Verification Checks.

Fig 5. Commit a query

Use It In Verify

A quick refresh on NQE Verification Checks: they are formulated as queries for violations.

If the query finds no violations, then the check passes. If the query finds violations, the check is marked as failed, and the identified violations form the failure diagnosis.

By default, all the NQE queries published in the NQE library are disabled (inactive state) in the Verify NQE page.

To turn an NQE query in an NQE Verification Check, simply enable it by clicking on the toggle button on the left side of the query.

Fig 6. NQE Verification Checks in the Verify application

If the NQE Verification Checks fails, you can see the violations as well as the queries by clicking on the Failed status link.

Fig 7. NQE Checks result

In a networking world that is moving at a steady pace toward network automation and network-as-code, versioning of code, configuration, intent, etc. has become a prerequisite for adoption. The same concept applies to NQE queries. 

Every time an NQE query is modified and published, a new version of the query is made available in the Verify NQE application. You can select a specific version or always run the latest version available via the Version used option:

Fig 8. Query versioning

I always try to stay away from product roadmaps in customer-facing publications but…rest assured this is not the last time you are going to see a blog on NQE, so stay tuned!

In the meantime, check this demo video out and happy querying with the Forward NQE Library!

What do we do?

At Forward Networks we build digital twins for computer networks. Enterprises with large networks use our software to build a software copy of their network and use that for searching, verifying, and predicting behavior of their network. It is not a simulation. It is a mathematically accurate model of their network.

Why is it a hard problem?

Large enterprise networks contain thousands of devices (switches, routers, firewalls, load balancers, etc). Each of these devices can have complex behaviors. Now imagine a large graph with thousands of nodes where each node represents one of these devices and the links between nodes show how they are connected. You need to model exactly how traffic originating from edge devices is propagated through the network.

To do so, you need to understand the exact behavior of each device in handling different packets. A typical enterprise network not only includes different types of devices (routers, firewalls, etc), but they are built by different vendors (Cisco, Arista, Juniper, etc) and even for the same device type from the same vendor, you typically see many different firmware versions. To build a mathematically accurate model you need to model every corner case and a lot of these are not even documented by vendors.

At Forward we have built an automated testing infrastructure for inferring forwarding behavior of devices. We purchase or lease these devices; put them in our lab; inject various types of traffic to them and observe how these devices behave.

Where are we today?

I’m proud to show off publicly today, for the first time, that we can process networks with more than 45,000 devices on a single box (e.g. a single ec2 instance). Here is a screenshot of an example network with about 45k devices:

Some of our customers send us their obfuscated data to help us identify performance bottlenecks and further improve performance. It is a win-win scenario. Our software gets better over time and they get even faster processing time. The data is fully obfuscated in that every IP and MAC address is randomly changed to a different address and every name is also converted to a random name and these mappings are irreversible. These changes do not materially change the overall behavior of the model and the obfuscated data is still representative of the complexity and diversity of network behaviors of the original network. The network in the above example is built from those data.

This network includes more than 10^30 flows. Each flow shows how a group of similar packets traverses the network. For example one flow might show how email traffic originating from a specific host and destined to another host starts from a datacenter then goes through several backbone devices and finally arrives at the destination data center. 

Each of these flows can be complex. If we were to spend 1 microsecond to compute each of these flows, it would still take us more than 10^17 years to compute this. But with a lot of hard engineering work, algorithmic optimizations and performance optimizations we are able to process this network in under an hour and we are capable of processing this on a single box. You don’t need a massive cluster for such computation. The best part is that the majority of the computation scales linearly. So, if customers want faster processing speed or higher search and verification throughput they can use our cluster version and scale based on their requirements.

How long did it take us to get here?

Forward Networks was founded in July 2013. Our founders are Stanford PhD grads and as a result the very first test data that we got was a 16 device collection from part of the Stanford network. I joined Forward in Sep 2014 after spending a couple of years building and working with large distributed systems in Facebook and Microsoft. I started leading the effort to scale our computation to be able to finish the computation of that 16 device network in a reasonable amount of time and it took us about two years to get there (Mar 2015).

Then almost every year we were able to process a 10x larger network. Today, we have tested our software on a very complex network with 45k devices. We are currently working on further optimization and scaling efforts and our projection is to get to 100k devices in Dec 2020. The following graph shows our progress our last couple of months and the projection till Dec 2020 on logarithmic scale:

Lessons learned

It takes time to build complex enterprise software

As I mentioned above, we started with data of a very small network. As we made our software better, faster and more scalable, we were able to go to customers with larger networks to get the next larger dataset; find the next set of bottlenecks and work on those. We had to rewrite or significantly change the computation core of our software multiple times because as we got access to larger data we would see patterns that we hadn’t anticipated before.

Could we have reduced the time it took us to get here if we had access to large data on early days of our start up? Yes. Was it feasible? No. Why would a large enterprise spend the time to install our software, configure their security policies to allow our collector to connect to thousands of devices in their production network to pull their configs and send the data to a tiny startup that doesn’t have a proven product yet? It is only going to happen if it is a win-win situation. Every time we got access to the next larger dataset from a customer, we optimized our software based on that and went to other customers that had networks of that size where our software was already capable of processing all or majority of their network and when they shared this data with us. We would either find new data patterns that needed to be optimized or combine all the data we had received from customers to build larger datasets for scale testing and improvements. It is a cycle and it takes time and patience to build complex enterprise software.

Customers with large networks typically have much more strict security policies which means that they wouldn’t share their data with us. This is why we had to spend the time and build data obfuscation capabilities in our software to allow them to obfuscate their data and share the result with us which would reveal the performance bottlenecks without sharing their actual data. Some customers have such strict policies that even that is not possible and for those we have built tools that aggregate overall statistics which are typically useful for narrowing down the root cause of performance bottlenecks.

When selling enterprise software, customers typically don’t spend a large amount of money on a software platform if they’re not 100% sure that it would work for them. There is typically a trial or proof-of-concept period where they install the software and evaluate it in their environment. In our early years, we worked very hard with our first few trial customers to make the software work well for them. There were cases which didn’t end up in immediate purchase but their data gave us invaluable insight in improving our software.

On-prem software should work on minimal hardware

These days it is pretty easy to provision an instance in AWS, Azure or other cloud providers with 1TB or more RAM. But you would be amazed to know how many times we have had to wait for weeks or months for some customers to provision a single on-prem instance with 128GB or 256GB RAM. Large enterprises typically allow provisioning small instances pretty quickly. But as soon as your software needs a more powerful instance, there can be a lot of bureaucracy to get it done. And remember, during the initial interactions with customers, you want them to start using your software quickly to finish the proof-of-concept period. During this time, they are still evaluating your software and they haven’t yet seen the value in their environment. So, if someone in a large organization opens a ticket to the infra teams to provision a software he/she wants to try, it may not be among the highest priority tickets that would get resolved.

At Forward Networks, we have learned to be very careful with any new tool, framework or dependency we add to our system. In fact our resource requirements are so low that our developers run the entire stack on their laptops which is very critical for fast debugging and quick iterations.

We have also spent a lot of engineering time and effort on making this possible. Here are some of the high level approaches:

When you need to scale to 1000x or 10000x, you can’t simply use a cluster with 1000 nodes. Even if it is possible, there is no economic justification to that. You have to do the hard engineering work to get the same done with minimal resources. Majority of our customers run our software on a single box. But we also provide the cluster version for those customers that want to ensure high availability or have more concurrent users and want to have higher search or compute throughput. 

One of our customers was telling us that they had to provision and operate a few racks of servers for another software (in the same space as us but not exactly our competitor) and how they were pleased and amazed on what our software delivers with such low requirements. Of course not only this can speed up adoption of the software, it saves customers money and allows you as a software vendor to have better margins.

Open source tools are not always the answer

In the early years of our startup, we were using off-the-shelf platforms and tools like Elasticsearch and Apache Spark for various usages. Over time it became clear that while these platforms are generic enough to be applicable to a wide range of applications, they weren’t a great fit when you need to have major customizations that are critical to your application.

For example, initially we were computing all end to end network behaviors and were indexing and storing them in Elasticsearch. But later it became clear that it is computationally infeasible to pre-compute all such behaviors to be able to store them in Elasticsearch and even if it was possible, such an index would be enormous in size. We had to switch to a lazy computation approach where we would pre-compute just enough data that would be needed to perform quick searches and at search time we would do the rest of the computation that was specific to user query. 

Initially we were trying to write plugins or customizations for Elasticsearch to adapt it to such a lazy computation approach but soon it became clear that it just won’t work and we had to create our own homegrown distributed compute and search platform.

Moving fast without breaking things needs sophisticated testing

Every month we release one major release of our software. Currently, each of these releases includes about 900 changes (git commits); and this is just going to increase as we hire more engineers. At this rate of change, we have to have a lot of testing in place to make sure we don’t have regressions in our releases. 

Every git commit is required to be verified by Jenkins jobs that run thousands of unit and integration tests to ensure there are no regressions. Any test failure would prevent the change from getting merged. In addition to these tests, we also use Error Prone to detect common bugs and Checkstyle to enforce a consistent coding style.

We also have many periodic tests that every few hours run more expensive tests against latest merged changes. These tests typically take a few hours to complete and hence it is not feasible to run them on individual changes. Instead when they detect issues, we use git bisect to identify the root causes. Not only these periodic tests check for correctness, they also ensure there are no performance regressions. These tests upload their performance results to SignalFx and we receive alerts on Slack and email if there are significant changes.

Are we done?

While we believe we have already built a product that is a significant step forward on how networks are managed and operated, our journey is 1% complete. Our vision is to become the essential platform for the whole network experience and we have just started in that direction. If this is something that interests you please join us. We are hiring for key positions across several departments. Note that having prior networking experience  is not a requirement for most of our software engineering positions.

If you operate a large-scale complex network, please request a demo to see how our software can de-risk your network operations and return massive business value.

The idea of building the Forward Dashboard has been floating around for a while now, but we kept postponing it to give priority to other features, like In-App Network Query Engine Checks, that built on the functionality and the customization of the Forward Platform. We’ve also done lots of work recently to improve usability and scalability. 

But why a dashboard? Who needs it? 

Senior IT leaders, like the Network Manager from an operations and engineering team, need metrics to establish targets and set goals that aim at improving the overall network reliability, correctness, and hygiene, on a regular basis.  Unfortunately, most of the dashboards available in the market provide so much information in a single pane that most of the users find overwhelming and, in the end, not very useful. 

Forward Dashboard has been designed to provide key network insights with visualizations that are easily consumable and exportable. All attributes that are perfectly in line with our mission: make the life of Network and Security teams easier by “making easy what is hard”, that is, managing very large, complex, heterogeneous, multi-vendor networks. 

 So…let me introduce you to the Forward Dashboard! 

In the initial release, Forward Dashboard provides insights on the network complexity, on the health of the network in terms of network verification, and on the improvements made by the team to make the network healthier. 

Network Complexity 

The Network Complexity pane includes insights on  Device modelsOS versions,  Device by type and several key metrics like Number of links,  Number of configuration lines,  Number of VLANs and  Number of VRFs with an indication of how they changed in the last 30 days. 

Network Verification 

The Network Verification pane provides insights on  Passive and Failing Active checks,  Devices with and without failures , and Violations by check type

It also displays historical data on intent and policy checks, that allows operators to see whether the network is improving its correctness over time (and also as a consequence of using Forward!).  

More network insides are going to be added in the future, so stay tuned and, in the meantime, check this demo video out to get a better understanding of Forward Dashboard.  

Network Query Engine (NQE) is my favorite Forward Enterprise feature and is one of the reasons why I decided to join Forward Networks. After working on Network Programmability and Automation for years, the idea of a single platform able to provide network device data, fully parsed, normalized, structured in OpenConfig-like format, across switches, routers, firewalls, and load balancers from different vendors, on-prem and in the cloud was (and still is!!) mind-blowing.

Based on my experience, vendors devote massive effort to provide native and OpenConfig YANG data models over common device APIs (e.g. NETCONF and RESTCONF) for the main network OSes. I doubt we’ll see any device APIs for legacy platforms from any networking vendor…ever. This inconsistency in the data formats and protocols creates a complexity that, more often than not, leads network automation projects to inevitable failure.

So, how can customers have all the benefits of structured data and device APIs and eliminate the inconsistency issues? Unless I’ve missed a recent announcement (and I’m pretty sure I’ve not), there is no networking vendor able to provide such a platform (or capability or functionality). The only option available is to rely on a solution like NQE.

With NQE customers can easily develop scripts applications to perform sanity checks or to display information across the entire fleet of devices in their network in a matter of minutes.

It’s an awesome solution for customers with development skills and a good knowledge of JSON format and YANG data models. Those skills are pretty common in DevOps or NetDevOps teams, but what if the customer doesn’t have those skills or they want to avoid building and maintaining those scripts/applications?

The answer is pretty easy, and it’s called… In-App NQE Checks!

In-App NQE Checks augments NQE by enabling customers to create custom verification checks, using the NQE data model, directly in the Forward Enterprise browser-based interface. No development skills needed, no scripts to build and maintain, just a simple and intuitive 3 step process:

  1. Explore the Data Model
  2. Create the Check
  3. Execute the Check and Save

The NQE Data Model can be explored using the built-in data model explorer that helps users find the relevant fields needed for the check, as shown in the picture below:

Fig 1. NQE Data Model explorer

The Check queries for violations uses 3 simple constructs, foreachwhere and select (sounds familiar, right?) and the violations will form the failure diagnosis.

If no violations are found, the check will pass.

The picture below shows a check that queries for interfaces with administrative status UP but with operational status other than UP– a fairly common scenario operators often need to monitor to validate the links’ health.

Fig 2. In-App NQE Check Example

This video shows the process needed to create and save the check described above.

Have you noticed how long is the query? 

Fig 3. In-App Check example

It’s a 9-line script for easy readability, but it’s essentially only 4 lines!

You can find some simple query examples like the one in the video as well as the Documentation in the tabs close to the data model.

Fig 4. Documentation and Examples

Moving forward, we’ll add more examples in the GUI but what we’re most excited about is building a knowledge base of useful checks shared with our customers and partners.

We’ve created a new dedicated In App NQE Checks examples repository on GitHub. It currently includes an initial list of examples and it’s waiting for your contribution ☺

Fig 5. In-App NQE Checks Examples repository

In summary, we did it again! We took something already amazing like NQE and we brought it to the next level with In-App NQE Checks. Now it’s your time to contribute!

Further reading:

In-App NQE Checks @ Tech Field Day 21: https://www.youtube.com/watch?v=MvrdQft7N5w

NQE repositories in GitHub:

NQE presentation @ ONF Sep 2019: https://www.youtube.com/watch?v=MvrdQft7N5w

If you are already a Network Query Engine (NQE) ninja, feel free to jump to the “NQE with Postman” section, otherwise, keep on reading.

Network Query Engine (NQE)

If you are familiar with the Forward Platform, you might already know that the platform discovers switches, routers, load balancers, and firewalls from all the most common vendors and collects their configuration and state. It then parses and normalizes all the data before running a mathematical model to reason about the network behavior in terms of routing and security policies.

The data parsing and normalization are key in allowing an efficient and very scalable platform, but before NQE all this data was available for internal use only.

With NQE we have essentially opened up the Forward Platform to provide parsed and normalized data to customers and partners for use cases like custom dashboards and custom network checks.

The beauty is that we do it for every supported platform, vendor and software version, even for very old legacy platforms that have been around for 30 years!

The exposed data structures are aligned with OpenConfig, the de-facto standard for vendor-neutral network device configuration and state data models (written in YANG) and it is available through a GraphQL API.

What is GraphQL?

GraphQL is a flexible data query language for APIs developed by Facebook in 2012 and released as an open-source project in 2015. GraphQL is an alternative to REST (Representational State Transfer). It offers several benefits compared to REST (see REST vs GraphQL article) like enabling users to specify exactly what data they get back in their response – nothing more, and nothing less, and it allows querying for multiple fields in a single request.

Hundreds of organizations, like Forward Networks, are already leveraging GraphQL!

NQE with Postman

Postman recently announced the inbuilt support for GraphQL to enable all the GraphQL users to leverage the most popular testing and development tool for HTTP APIs.

Unfortunately, it doesn't support GraphQL Introspection [yet?], a key feature that allows to populate the schema inspector, provides autocomplete and enables to easily build the schema documentation.

The GraphQL schema can be imported manually instead, providing autocomplete (no documentation, sorry). GraphQL Schema Definition Language (SDL) is the only format supported at the moment.

The NQE schema in SDL format can be exported from the Forward platform and manually imported in Postman. See the NQE Github repository for how-to instructions.

After the NQE schema is manually imported, you can easily build the Postman requests by selecting GraphQL, selecting the imported NQE schema, writing the query, send it, and… voilà!

   

Figure 1: NQE query with Postman

For more information on NQE with Postman, check this video below.

For more info on NQE, check the following articles and the NQE repo on GitHub:

Enjoy your NQE queries with Postman!!

Every time I explain the Forward platform and what a Snapshot is, I know the next question I will be asked. I would start with something like "A Snapshot is a collection of the network devices’ running configuration and state at a specific point in time. The Forward Platform uses all the data gathered to run its mathematical model and calculate every possible traffic path in the network.” And here it comes the same question, every, single, time: “How long does it take?” I smile and I say “Well, it depends :)"

It’s not that I want to hide bad news or that I’m afraid the answer could close any further conversation, but it really depends on different factors, and most of them actually are not related to the Forward platform but to the managed networks instead, like the number of devices, the size of the forwarding tables, the ability of authentication servers to keep up with the parallel collections Forward instantiates just to name a few. Some of these factors can impact one or more stages that go into building the mathematical model of the network. At the end of the day, what customers care about is the total time between when a new collection starts and when all the data has been processed and available on the Forward User Interface (or the Forward APIs). In large network environments, where thousands of devices are contained in a single view, this time can be significant.

To simplify the concept, the total time to build the mathematical model of the network can be divided into two main components:

We spend a lot of time and effort to constantly improve the Forward Platform to support bigger networks and at the same time to reduce the time needed to build the model. Many of these improvements are driven by our customer’s use cases. For instance, some customers are using the Forward platforms in change management windows where it’s critical to verify that the network is behaving as expected as quickly as possible after a change has been made. Often only a small subset of devices is affected by the change, the collection time is usually way longer than the processing time and, last but not least, the biggest bottleneck is the authentication server's scalability and response time.

Today I’m happy to announce the release of Partial Collection, a new feature designed with the change management windows use case in mind.

Partial Collection allows users to dramatically reduce the collection time by restricting and triggering a new collection from a subset of devices and then merging the new data with the most recent Snapshot.

The merged data is then fully processed again by the Forward platform.

Some customers have already reported huge improvements by leveraging Partial Collection. For instance, a customer with 5000 devices, very large routing tables and, more importantly, a TACACS server that can support only 16 parallel connections has been able to reduce the collection time by 10 times, allowing them to verify the network in very short change management windows!

We have also added new useful information to the snapshot menu, the collection Span. It's the period of time between the oldest and newest device collection times in the Snapshot. In this way, customers can understand how old the oldest collection is, as well as how far apart the collection stretched for the given network.

For more information on Partial Collection check the Snapshot documentation page in the Getting Started section.

Stay tuned for more enhancement on Collection and Processing Time!

Top cross