Visualizing Graphs with ArangoDB and KeyLines

20th March, 2017 Estimated reading time 8–14 minutes

graph database landscape
Some of the players in the growing graph database market

The growing interest in graphs shows no signs of slowing as organizations seek to realise the value buried in their connected data. In the last 12 months alone, IBM Graph, Microsoft Graph Engine, and Google & IBM-backed Janus Graph joined existing players like Oracle, DataStax Enterprise, Titan and Neo4j, in the enterprise graph database market.

But there’s another genre of datastore that’s optimised for connected data: the multi-model database.

This blog post focuses on ArangoDB – a native multi-model database that combines the strengths of graph, document and key-value stores – and how it can power your KeyLines application.

What is a multi-model database?

Database technologies are optimized for specific types of data:

  • Structured data —> relational database
  • Unstructured data —> document store
  • Highly connected data —> graph database, etc.

This approach, known as polyglot persistence, ensures good performance but presents challenges. Deploying to multiple database technologies in a single project is complex and has a greater maintenance overhead.

One way to avoid these problems is to use a multi-model databases like ArangoDB. ArangoDB combines the advantages of a document, key-value and graph database with good performance and scalability.

ArangoDB can work with KeyLines to power a robust graph visualization application. Let’s see how.

[Disclaimer: This blog post uses v3.1 of ArangoDB. We don’t plan to maintain the code here, so can’t guarantee this will work for all future versions of the database.]

Step 1: Set up ArangoDB

Download and install the latest ArangoDB version. There are full instructions in their excellent getting started guide.

Next, start the ArangoDB server. I’m using a Mac, so at the command line I’ll enter:

/usr/local/sbin/arangod

You can run commands in the shell interface called ‘arangosh’. It’s best to create a user or change the root password before you continue. See ArangoDB’s instructions on managing users.

Go to http://localhost:8529 to see ArangoDB’s neat web interface. In this example, we’ll build a network of movies and actors to explore in a KeyLines application.

graph data model

So, using the Arango interface, create:

  • a database called ‘movies’
  • two document collections called ‘movie’ and ‘actor’
  • an edge collection called ‘links’

Next, create a graph called ‘movies-graph’ with this setup:

visualizing Arangodb - setup 1

We’ve defined the graph with a single edge collection and 2 collections for vertices. (Note: vertex is a graph database term for node, and an edge is a link).

Now we’ve set up our collections and graph, we can import the data.

ArangoDB comes with a command line tool ‘arangoimp’, which makes it easy to import data from different formats. The movies data is in JSON files, so we’ll run these commands to import it into our database:

	arangoimp --server.database movies --file movies.json --collection movie
	arangoimp --server.database movies --file actors.json --collection actor
	arangoimp --server.database movies --file links.json --collection links

You may see some WARNING messages in the console about ‘unique constraint violated’. These may look scary, but are completely safe to ignore. Due to the way we created the json files, this is just Arango’s way of telling us that it will not create a vertex more than once and that behaviour is completely reasonable for us.

Don’t forget to add the options ––server.password [password] or ––server.username [username] if you have different credentials. See ArangoDB’s documentation on importing data.

Step 2: Write some queries

To make querying ArangoDB easier, the team have created the Arango Query Language (AQL). For example, to find Keanu Reeves in the database, run:

FOR a IN actor FILTER a.name == 'Keanu Reeves' RETURN a

These results are a starting point that we can build on. To find movies that Keanu Reeves acted in, run:

LET start = (FOR a IN actor FILTER a.name == 'Keanu Reeves' RETURN a)[0]
FOR movies IN 1 OUTBOUND start GRAPH 'movies-graph' RETURN movies

A great feature of ArangoDB is that we can use JavaScript syntax to decide how query results should be returned. It means we can present our JSON result in a KeyLines-ready format, without any client-side processing.

For example, to request a specific actor and their direct connections, run:

LET start = (FOR a IN actor FILTER a.name == 'Keanu Reeves' RETURN a)[0]
  RETURN APPEND([{
  id: start._id,
  type: 'node',
  t: start._key,
  d: start
  }], FLATTEN(FOR v, e IN 1 ANY start GRAPH 'movies-graph'
    RETURN [{
      id: v._id,
      type: 'node',
      t: v._key,
      d: v
    },
    {
      id: e._id,
      type: 'link',
      t: e._key,
      id1: e._from,
      id2: e._to,
      d: e
    }]))

The ArangoDB website has some great AQL resources to help you return custom results.

Step 3: Write a Foxx microservice

Foxx is a JavaScript framework that runs inside the ArangoDB container. It lets you write standard V8 JavaScript, and create REST endpoints that can return ArangoDB queries. The syntax is similar to the Express framework for Node.js.

Here’s a simple example of a Foxx service:

‘use strict’;
const createRouter = require('@arangodb/foxx/router');
const router = createRouter();
router.get((req, res) => {
  res.write(‘Hello World!’);
});

This sets up a router for any request that hits our endpoint and responds with ‘Hello World!’. If we load this Foxx service onto our database (more on how to do this later), then we can go to a URL similar to http://localhost:8529/_db/my_foxx_service/ to see a browser-based response.

Let’s build a more useful Foxx microservice. Create a new folder called ‘keylines-service’, for our Foxx microservice code.

Now create a manifest.json file that describes the service:

{
  "name": "keylines-service",
  "version": "1.0.0",
  "engines": {
    "arangodb": "^3.0.0"
  },
  "main": "main.js",
  "description": "Send queries to an ArangoDB graph and get the response in the KeyLines JSON format",
  "keywords": ["keylines"]
}

Next create the main.js file that contains our Foxx code:

'use strict';
const createRouter = require('@arangodb/foxx/router');
const router = createRouter(); 
const joi = require('joi');    
const { db, aql, errors } = require('@arangodb');

router.get('/:name', function (req, res) {
  const { name } = req.pathParams;
  const query = aql`
    LET start = (FOR a IN actor FILTER a.name == ${name} RETURN a)[0]
    RETURN APPEND([{
      id: start._id,
      type: 'node',
      t: start._key,
      d: start
    }], FLATTEN(FOR v, e IN 1 ANY start GRAPH 'movies-graph'
      RETURN [{
        id: v._id,
        type: 'node',
        t: v._key,
        d: v
      },
      {
        id: e._id,
        type: 'link',
        t: e._key,
        id1: e._from,
        id2: e._to,
        d: e
      }]))
  `;
  res.send({type: 'LinkChart', items: db._query(query).toArray()[0]});
})
  .pathParam('name', joi.string().required(), 'The name of the actor to query.')
  .response(joi.object().required().keys({
    items: joi.array().items(joi.object().required().keys({
        id: joi.string().required(),    
        type: joi.string().required()   
      })),
    type: joi.string().required()   
  }))
  .summary('Query arango and get results in KeyLines JSON format')
  .description('Get neighbours for a node with a given ID. The query result is returned in KeyLines JSON format.')

module.context.use(router);

That’s all the code we need for our Foxx microservice. To install this in ArangoDB, go to the parent directory of the service, then run this command to create a zip that contains the service:

zip -r keylines-service.zip keylines-service

Go to the web interface and click ‘Services’, then ‘Add Service’. Choose an appropriate mount-point (the URL from which we can access the service – we called it ‘/keylines’ in this example) and upload the zip file.

visualizing Arangodb - setup 2

Finally, go to http://localhost:8529/_db/movies/keylines/Keanu%20Reeves to see the JSON result of our service.

Building simple KeyLines demo

Now that ArangoDB returns data in KeyLines format, you can easily set up a KeyLines demo using the microservice as the backend. The fastest way to get started is to:

  • Start your KeyLines trial
  • Follow the simple step-by-step Quick Start guide
  • To your ‘chart-styles.js’ file (or the corresponding folder of whichever demo you started with) add this:
var chart;
function chartLoaded(err, loadedChart) {
  chart = loadedChart;

  $.getJSON(‘http://localhost:8529/_db/movies/keylines/Keanu%20Reeves’,     function (data) {
chart.load(data).then(chart.layout);
});
}

$(function () 
  KeyLines.paths({ assets: 'assets/' })
  KeyLines.create({id: 'kl', type: ‘chart’}, chartLoaded);
});

The JavaScript file doesn’t need to process data. We could easily extend this demo to incorporate other KeyLines features, or provide a search box and allow users to search for any actor (or movie with some minor adjustments to our microservice). The KeyLines SDK offers a huge range of visualization and analysis functionality.

Try it yourself

If you’re inspired to try visualizing your own ArangoDB instance, or hook your KeyLines application up to the Arango database, why not get started now?

An example of a graph visualization application, powered by KeyLines & Arangodb
An example of a graph visualization application, powered by KeyLines & ArangoDB

| |

Try KeyLines