Advanced graph visualization with D3

In previous posts of this series I showed how to build a basic network visualization with a force-directed layout and some customized labels, arrows, and glyphs.

This time, let’s continue our journey with some more advanced features.

Graph Function: Neighbours

Normally after running their layout, a user wants to make a selection and start diving into their graph. A common way to do this is by selecting and highlighting a node and its neighbors, using the mouse click event.

Let’s see how this works in D3, then KeyLines.

Select and highlight neighbours with D3

D3, unhelpfully, doesn’t have a concept of neighbors. Instead, we need to build it.

The first step is to create a rectangle in the background, to capture clicks on the canvas and call the restore selection function:

const canvas = svg.append("svg:rect")
    .attr({
        "height": height,
        "width": width,
        "fill": "none"
    });
...

canvas.on("click", () => deHighlight());

We also bind the highlight function to the node click:

const node = svg.selectAll(".node")
    ...
    .on("click", d => highlightNeighbours(d));

Then we create a dictionary to look up neighbors of a specific node:

const neighbours = {};

force.nodes().forEach((node) => {
    neighbours[node.index] = neighbours[node.index] || [];
});

force.links().forEach((link) => {
    neighbours[link.source.index].push(link.target.index);
    neighbours[link.target.index].push(link.source.index);
});

Next we want to visually highlight the neighbours of the nodes we click. This again is a bit fiddly in D3. We add a new “background” class to non-neighbouring item, using D3 .classed method, and control the class behaviour with css: .background { opacity: 0.1; }. This will ‘ghost’ items into the background.

In the same function, we also want to display details (node and edge labels) of the highlighted items:

function highlightNeighbours(d) {

    deHighlight();

    node.classed("background", (n) => {
        return neighbours[d.index].indexOf(n.index) != -1 ? false : true;
    });

    link.classed("background", (n) => {
        return d.index == n.source.index || d.index == n.target.index ? false : true;
    });

    nodeLabels.filter(n => neighbours[d.index].indexOf(n.index) != -1);
    // we can't use display:none with labels because we need to load them in the DOM in order to calculate the background rectangle dimensions with the getBBox function. 
    // So we used visibility:hidden instead.
      .style("visibility", "visible");

    textBackground.filter(n => neighbours[d.index].indexOf(n.index) != -1)
      .style("display", "inline-block");

    linkLabels.style("display", function(n) {
        return d.index == n.source.index || d.index == n.target.index ? "inline-block" : "none";
    });

    // select self and change its properties
    d3.select(this).classed("background", false);
    d3.selectAll(this.childNodes).style({
        "display": "inline-block",
        "visibility": "visible"
    });
};

It might also be useful to add a counter glyph to the node, showing the node degree:

...
glyphLabels.text(() => neighbours[d.index].length);
...

Then finally, we also need a function to remove highlight settings:

function deHighlight() {
    node.classed("background", false);
    link.classed("background", false);
    linkLabels.style("display", "none");;
    nodeLabels.style("visibility", "hidden");
    nodeGlyph.style("display", "none");
    glyphLabels.style("display", "none");
    textBackground.style("display", "none");
}

After all of that coding, you might – with some luck – get a result like this:

Select and highlight neighbours with KeyLines

Happily, a comparable effect can be achieved in KeyLines using some handy functions:

chart.foreground(),
chart.selection(),
chart.getItem()

combined in the following order:

chart.on('selection-change', highlightNeighbours);

function highlightNeighbours(id) {

    // this function looks up into the neighbours dictionary
    function areNeighboursOf(item) {
        return neighbours[item.id];
    }

    const neighbours = {};
    deHighlight();

    // select timebar lines
    tlSelect(chart.selection());

    // exclude click on canvas
    if (id != null) {

        const item = chart.getItem(id);

        // capture click on nodes
        if (item && item.type === 'node') {
            
            // get neighbouring items (nodes and links)
            const result = chart.graph().neighbours(id);

            // store property changes in a variable for later use and add neighbours to the dictionary
            const changes = result.nodes.concat(result.links).map((id) => {
                neighbours[id] = true;
                const neighbour = chart.getItem(id);
                return {
                    id: neighbor.id,
                    t: neighbour.d.label
                }
            });

            // add self to the neighbours dictionary and add the glyph
            neighbours[id] = true;
            changes.push({
                id: id,
                t: item.d.label,
                g: [{
                    p: 'ne',
                    c: '#de5835',
                    w: true,
                    e: 1.3,
                    t: result.nodes.length
                }]
            });

            // chart.setProperties() allows us to change multiple properties at a time, targeting item ids
            // note that in deHighlight() we use it with RegEx option to target all ids!
            chart.setProperties(changes);

            // put neighbours in the foreground
            chart.foreground(areNeighboursOf, { type: 'all' });

            // select neighbouring items
            chart.selection(result.nodes.concat(item.id).concat(result.links));
        }
    }
};

function deHighlight() {
    chart.setProperties({ id: '.', t: null, g: null }, true);
    chart.foreground(() => true, { type: 'all' });
};

KeyLines associates a click with an item id. This is much more convenient than D3’s approach that associates it with the target visual object, as it means we can construct simple conditionals to catch canvas clicks and node clicks (or others).

The chart.graph().neighbours function returns neighbour nodes. This function accepts some very convenient, graph specific, options, for example:

• direction defines if neighbours should be found – inbound links, outbound links, or both
• hops defines how many levels of neighbours to find

In the code above we also used KeyLines custom data property (d), which we have previously fed with multiple fields in the parsing function:

const nodes = obj.nodes.map(function(node) {
    return {
        ...
        d: {
            label: node.Name,
            address: node.Address,
            city: node.City,
            events: [node.Event1, node.Event2]
	    ...
        },
        ...
    };
});

By contrasting the two approaches here, we can see that the KeyLines API makes some common graph visualization tasks much simpler for the developer to implement.

Advanced graph functions

In the previous section, we explored the structure of our network by searching for a node’s neighbors and calculating its degree. Degree is a useful centrality measure but can be too simplistic. Sometimes more advanced options – e.g. betweenness or PageRank, give a more useful view.

Let’s discuss how we implement these in the two technologies.

Advanced graph functions in D3

With D3 you are left to your programming and math skills to create a graph function – and graphs tend to involve very demanding calculations. Optimization is also a big hurdle, not to mention the task of binding your custom centrality scores to a visual property or updating the graph in real-time in response to user behaviors.

Advanced graph functions in KeyLines

Happily, KeyLines has you covered.

There are plenty of native graph functions (learn more about social network analysis) that have been designed and carefully optimized to work across virtually all use cases.

There is another benefit in using KeyLines: it has been designed only for graph visualization.

Every function has been written to work alongside the rest of the graph API, and graph theory is interwoven into the code – making it easy to layer functionality and tell a great network story.

Network Analysis is only one of the features of KeyLines. Other functionality, like automatic layouts, combining nodes (combos), network filtering, help users to work through dense networks and focus on the data they need to understand.

Another key task your users will probably want to do is understand temporal or geographic trends in their graphs. Let’s look at these two in more detail.

Dynamic networks

Networks are hardly ever static. They change through time, so your graph visualization needs to be able to show that. The most intuitive way for this is using a time bar component.

Dynamic networks with KeyLines

One of KeyLines’ most popular pieces of functionality is the time bar:

It is incredibly responsive, easy to use and simple to incorporate into your graph visualization app. First, we change a bit our KeyLines.create() function to create the two different components (chart and time bar):

[chart, timebar] = await KeyLines.create([{
    container: 'kl',
    type: 'chart'
}, {
    container: 'tl',
    type: 'timebar'
}]);

We need to set up our time data in the appropriate way, to include timestamps – either in Epoch or JavaScript format. We do this by assigning a date object to the dt property, and value (if needed) to the v property:

const links = obj.links.map(function(link) {
    return {
        ...
        dt: [new Date(link.Event1), new Date(link.Event2), new Date(link.Event3)],
        v: [link.Value1, link.Value2, link.Value3]
        ...
    };
});

And then load our data into the graph and time bar:

timebar.load(chartData);
timebar.zoom('fit', { animate: false });

We need to bind to a filtering function so only items present in the user-defined time range are visible. We’ll also apply a standard layout in adaptive mode, so the chart adapts smoothly to updated items:

timebar.on('change', tlChange);

const tlChange = async function() {
    await chart.filter(timebar.inRange, {
        animate: false,
        type: 'link'
    });
    await chart.layout('standard', {
        mode: 'adaptive',
        animate: true,
        time: 400
    });
};

Finally, we can add some interaction to display a selection line for the selected nodes and its neighbours using timebar.selection():

...
tlSelect(chart.selection());
...

function tlSelect(selection) {
    const selectionObject = [{
        id: selection,
        index: 0,
        c: '#79b300'
    }];
    timebar.selection(selectionObject);
}

With just these few lines of code, we can produce a truly impressive dynamic graph visualization (watch the screencast in the next section).

Dynamic network with D3

D3 has some helpful functions to manage time based data. D3.time.scale, for instance, helps calculating responsive time axis and provide ticks based on time intervals. It is easy then to position elements at the correct time-point, and with D3 .filter method you can  shows items in a specific time range.

Apart from that, you’re on your own. Creating a time bar, especially one that synchronizes with a graph component, is a hugely complex task. Far too complex for this blog post.

Geospatial networks

Another often crucial dimension to data is geography. If nodes have geolocations, users will want to see their network’s geo trends.

Geospatial networks with KeyLines

The good news is that with a ridiculously small amount of code we can enable KeyLines’ map mode.

We implement chart.map().show() and chart.map().hide() functions, binding them to two different html buttons, and push coordinates in the pos property. If you remember, our data was about fake US companies, and we had latitude and longitude already stored for each one of them.

...
pos: {
    lat: node.Latitude,
    lng: node.Longitude
},
...

document.getElementById('mapOn').addEventListener('click', () => chart.map().show());
document.getElementById('mapOff').addEventListener('click', () => chart.map().hide());

Since KeyLines uses the Leaflet library to draw maps, we also add a link to Leaflet assets in our html file.

<script src="map/leaflet.js" type="text/javascript"></script>

This way, we can easily get our graph, timeline and map working together. The result is again impressive and some further customization is possible using map options and functions.

Geospatial networks with D3

D3 has native support for map projections and geometric shapes. You can also implement a 3rd party map drawing library such as Leaflet, as KeyLines does. But again, no built-in function will take care of switching between network and map modes, and you need to code this by yourself.

If you can get all of this working alongside a time bar too, well… get in touch. We’re always looking for outstanding JavaScript developers.

A quick note on performance

When talking about performance in web-based graph visualization, besides code optimization, discussions often focus on rendering engines: SVG, Canvas, or WebGL?

KeyLines, our JavaScript graph visualization toolkit, uses HTML5 Canvas as a default renderer, this can handle many more elements than SVG, the default of D3. KeyLines also provides a WebGL renderer which can leverage the GPU calculating power to boost performance even more.

It is possible to use Canvas with D3, but you can’t use D3 built-in drawing functions and you need to code your own completely from scratch. Switching between rendering modes in KeyLines is as easy as changing an option.

The result: KeyLines is by far the better option for visualizing very large graphs, or even moderately large graphs.

Conclusion – manageable complexity

At the end of this post series I can confirm my previous conclusions:

• As a graph-specific library, KeyLines provides a better graph visualization experience for users and developers alike.
• The KeyLines API is beautifully designed, and developing with it is simpler and faster.
• D3 provides great flexibility, but the trade-off is time, resource and lack of boundaries. Put simply, it’s easier to build a terrible graph visualization in D3, and easy to build something powerful and beautiful with Keylines.

Building graph visualizations is always fun, regardless of the tool you choose and whatever your use case (read about our most popular use cases). I would encourage you to try both and see how you get on.

Try KeyLines for yourself

If you want to try KeyLines for yourself and see how easy and fast it is to visualize your connected data, you can request a free trial.