The resilience of the world’s infrastructure relies on understanding connections. From trans-continental gas pipelines to a small IT networks, a single failure can have a significant impact down the chain.
To manage that risk, huge sums are spent on network monitoring tools. These tools collate terabytes of data, detailing every device and connection in a network. In this post, I’ll demonstrate why graph visualization, powered by a tool like KeyLines, is the ideal way to make sense of that data.
The example I’m sharing here relates to an IT network topology, but the techniques and approach work equally well for any kind of infrastructure management. If you’d prefer, you can see a video of the application running as part of this webinar, co-hosted with our partners ArangoDB.
There are two key advantages to exploring infrastructure data as a graph.
Firstly, it gives analysts a faster way to discover and understand the network topology. Often network diagrams are painstakingly compiled using diagramming tools like Visio. A complete picture is slow to create, and impossible to maintain. An automated approach, using in an interactive KeyLines front-end, gives detailed and up-to-the-minute views of the network topology without the need to trawl logs and documents.
Secondly, a graph visualization can help reveals potential network threats. Having a complete and accurate view of the network topology makes it easier to find weaknesses or bottlenecks, showing point that can be exploited or on which there is dangerous over-reliance.
Let’s look at an example.
Note: I’m using falsified data here, but it borrows heavily from real IT network datasets we have visualized in KeyLines.
In this example, I opted to use an ArangoDB back-end for my visualization. It gives me the scalability and performance I need to work with the large datasets involved. It also integrates easily with KeyLines.
Between Arango and KeyLines, I built a simple microservice using the Foxx framework, giving a neat end-point that reduces lag and delivers data to KeyLines pre-parsed into the required JSON format.
Let’s load our starting point, in this case a network bridge:
So far, so good. We can see the bridge, labelled with its device name and highlighted with a double ‘halo’. But a single-node network isn’t very insightful – let’s expand out a level:
This shows us the 10 switches connected to the bridge. I’ve used two different types of link (solid and dashed) to show for example, connection type or status, and glyphs on the nodes, which show device uptime:
Already we can see several nodes with poor uptime – something we can drill into further by expanding out another level:
This view shows us there’s a lot more red glyphs – more devices with uptime issues. Let’s expand out again to see the final level in our dataset: hosts.
This gives us a busy chart, but thanks to the KeyLines standard layout we can start to pick out patterns.
There’s some clusters, representing large numbers of hosts connected to an individual router. These could be bottlenecks, or devices on which there’s a disproportionate dependency. A failure of, or attack on, one of these would cause problems.
An advantage of visually analysing this data as a graph is the ability it gives to run social network analysis measures. These help us uncover a network’s most important nodes.
Here we’ve calculated the significance of nodes based on the number of incoming links from any distance. It’s another way to perform impact analysis and uncover nodes that are heavily relied upon.
Finally, we can apply a powerful bit of functionality that really helps simplify the data so users can drill into the right areas. KeyLines allows me to combine nodes based on common properties or connections. Here, we can group by level and type:
The result is a drilled down version of the original view – a topology of the topology which simplifies the chart. Additional information can be added back in thanks to KeyLines’ open combos functionality, which reveals additional data on demand: