Building network analysis tools
The resilience of the world’s infrastructure relies on understanding connections. From trans-continental gas pipelines to small IT networks, a single failure can have a significant impact down the chain.
To manage that risk, huge sums are spent on network monitoring and network analysis tools. These tools collate terabytes of data, detailing every device and connection in a network. In this post, I’ll demonstrate why network analysis (often called graph visualization), powered by a network visualization tool like KeyLines, is the ideal way to make sense of that data.
The example I’m sharing here relates to an IT network topology, but the network analysis techniques and approaches work equally well for any kind of infrastructure management. If you’d prefer, you can see a video of the application running as part of this webinar, co-hosted with our partners ArangoDB.
Why visualize an IT network as a graph?
There are two key advantages to exploring infrastructure data as a graph.
Firstly, it gives analysts a faster way to discover and understand the network topology. Often network diagrams are painstakingly compiled using diagramming tools like Visio. A complete picture is slow to create and impossible to maintain. An automated approach, using interactive network analysis tools built with KeyLines, gives detailed and up-to-the-minute views of the network topology without the need to trawl logs and documents.
Secondly, graph visualization can help reveals potential network threats. Having a complete and accurate view of the network topology makes it easier to find weaknesses or bottlenecks, showing the point that can be exploited or on which there is dangerous over-reliance.
Let’s look at an example.
Note: I’m using falsified data here, but it borrows heavily from real IT network datasets we have visualized in KeyLines.
The architecture of network analysis tools
In this example, I opted to use an ArangoDB back-end for my visualization. It gives me the scalability and performance I need to work with the large datasets involved. It also integrates easily with KeyLines.
Between Arango and KeyLines, I built a simple microservice using the Foxx framework, giving a neat end-point that reduces lag and delivers data to KeyLines pre-parsed into the required JSON format.
Let’s load our starting point, in this case, a network bridge:
So far, so good. We can see the bridge, labeled with its device name and highlighted with a double ‘halo’. But a single-node network isn’t very insightful – let’s expand out a level:
This shows us the 10 switches connected to the bridge. I’ve used two different types of link (solid and dashed) to show, for example, connection type or status, and glyphs on the nodes, which show device uptime:
Already we can see several nodes with poor uptime – something we can drill into further by expanding out another level:
This view shows us there’s a lot more red glyphs – more devices with uptime issues. Let’s expand out again to see the final level in our dataset: hosts.
This gives us a busy chart, but thanks to the KeyLines standard layout we can start to pick out patterns.
There are some clusters, representing large numbers of hosts connected to an individual router. These could be bottlenecks or devices on which there’s a disproportionate dependency. A failure of, or attack on, one of these would cause problems.
An advantage of using graph-based network analysis tools is the ability they give to run social network analysis measures. These help us uncover a network’s most important nodes.
Here we’ve calculated the significance of nodes based on the number of incoming links from any distance. It’s another way to perform impact analysis and uncover nodes that are heavily relied upon.
Finally, we can apply a powerful bit of functionality that really helps simplify the data so users can drill into the right areas. KeyLines allows me to combine nodes based on common properties or connections. Here, we can group by level and type:
The result is a drilled down version of the original view – a topology of the topology which simplifies the chart. Additional information can be added back in thanks to KeyLines’ combos functionality, which reveals additional data on demand: