Hooking KeyLines up to the Neo4j Bolt Protocol

The Bolt Protocol is a popular way to connect your KeyLines application to a Neo4j database. Compared to the original REST protocol, Bolt is faster and more secure, making it even easier to build high performance graph visualization tools.

In this blog post we’ll take a closer look at the Bolt Protocol. If you’re a more experienced Neo4j user, you can also follow the code snippets and step-by-step instructions to visualize your Neo4j data in KeyLines faster than ever.

What is Bolt?

When it was first released in 2016, the Neo4j team described Bolt as a “connection-oriented protocol to access the graph”. It’s another way for Neo4j to communicate with other technologies. It encodes plain text Cypher queries to a binary format, then sends them directly to the database via Websockets or a standard TCP connection.

The Bolt approach has three advantages over the REST API:

• Speed – binary encoding means less data gets sent over the wire, and access is faster.
• Performance – Websockets execute high volumes of queries faster, giving higher throughput and lower latency.
• Security – Bolt has TLS security built and enabled by default to protect the data connection. The Bolt team has created official drivers in multiple languages – including JavaScript, which can run on the client or server-side.

Let’s try using a KeyLines/Neo4j Bolt combination to visualize graph data.

Step 1: Setting up the environment

For the purposes of this blog post, we’ll use the official Neo4j JavaScript driver, and Neo4j v4.x, loaded with dummy data from the movie database. A similar example of this can be found in our Neo4j KeyLines tutorial, where we use similar data.

Use these commands to start a new project and install the neo4j-driver:

npm init --yes
npm i neo4j-driver -S

Download your copy of KeyLines and unzip it into the same directory.

Step 2: Connect KeyLines to Neo4j

There are several ways to connect to KeyLines with the JavaScript bolt library, but we’ll focus on the simplest option: running in NodeJS or directly in the browser (client-side).

• To setup and authenticate the Bolt connection for a NodeJS setup, require (‘neo4j-driver’) in Node.
• For the browser setup, include the browser version of the Neo4j driver, which is in node_modules.

Step 3: Set up the driver

The next step is to set up the driver with some credentials.

First, we’ll reassign the Neo4j variable at the top of the JavaScript file (whether client-side or server-side):

const neo4j = require('neo4j-driver');

Then we’ll set up the driver with our credentials:

const configs = {
    boltPath: 'bolt://localhost:[port]',
    username: 'neo4j',
    password: '',
};
 
const driver = neo4j.driver(
    configs.boltPath,
    neo4j.auth.basic(configs.username, configs.password),
    { disableLosslessIntegers: true },
);

Now we can create sessions to run queries:

function queryNeo (query) {
  const session = driver.session();
  return session
    .run(query)
    .catch(e => console.log(e))
    .then(results => {
      session.close();
      // do something with results here
      return results;
    });
}

queryNeo(
  ‘MATCH (movie:Movie{title: "The Lord of the Rings: The Fellowship of the Ring"})<-[rel]-(actor:Actor) RETURN *’,
);

Neo4j recommends closing the sessions between requests, unless you want to run a batch of queries in parallel, to reduce simultaneous connections overloading the server.

Step 4: Process the Bolt response

One disadvantage of the Bolt protocol for graph visualizers is that the JSON response you get is flatter than the JSON we’re used to from the REST protocol.

With the REST API, we can request that results are returned in a graph-like structure, which is easy to convert into a KeyLines format. This isn’t the case with the Bolt protocol, because we get a response like this:

{
  "records": [
    {
      "keys": [
        "actor",
        "movie",
        "rel"
      ],
      "length": 3,
      "_fields": [
        {
          "identity": {
            "low": 819,
            "high": 0
          },
          "labels": [
            "Person",
            "Actor"
          ],
          "properties": {
            "name": "Andy Serkis",
            "lastModified": "1299996916000",
            "id": "1333",
            "biography": "",
            "version": {
              "low": 134,
              "high": 0
            },
            "profileImageUrl": "http://cf1.imgobject.com/profiles/02e/4ceeb6645e73d654f900002e/andy-serkis-profile.jpg"
          }
        },
        {
          "identity": {
            "low": 802,
            "high": 0
          },
          "labels": [
            "Movie"
          ],
          "properties": {
            "studio": "New Line Cinema",
            "releaseDate": "1008716400000",
            "imdbId": "tt0120737",
            "runtime": {
              "low": 178,
              "high": 0
            },
            "description": "Young hobbit Frodo Baggins, after inheriting a mysterious ring from his uncle Bilbo, must leave his home in order to keep it from falling into the hands of its evil creator. Along the way, a fellowship is formed to protect the ringbearer and make sure that the ring arrives at its final destination: Mt. Doom, the only place where it can be destroyed.",
            "language": "en",
            "title": "The Lord of the Rings: The Fellowship of the Ring",
            "version": {
              "low": 410,
              "high": 0
            },
            "trailer": "http://www.youtube.com/watch?v=Pki6jbSbXIY",
            "imageUrl": "http://cf1.imgobject.com/posters/8b8/4cc48d685e73d677850018b8/the-lord-of-the-rings-the-fellowship-of-the-ring-mid.jpg",
            "genre": "Action",
            "tagline": "One ring to rule them all",
            "lastModified": "1299912557000",
            "id": "120",
            "homepage": "http://www.lordoftherings.net/"
          }
        },
        {
          "identity": {
            "low": 1005,
            "high": 0
          },
          "start": {
            "low": 819,
            "high": 0
          },
          "end": {
            "low": 802,
            "high": 0
          },
          "type": "ACTS_IN",
          "properties": {
            "name": "Gollum"
          }
        }
      ],
      "_fieldLookup": {
        "actor": 0,
        "movie": 1,
        "rel": 2
      }
    },
…

But all is not lost. We’ll convert this into a JSON format KeyLines can use.

Step 5: Convert the response into KeyLines JSON

There are three stages to convert the Bolt format into KeyLines JSON:

• Loop through each field to look for KeyLines nodes with the right IDs that haven’t been stored already, then convert them.
• Convert the ‘start’ and ‘end’ properties of relationships, and match them to an element in the fields array (nodes).
• Convert other numeric properties if possible, e.g. time stamps for the time bar.

To convert the number formats, we loop through every property to see if it’s a Neo4j Integer object:

neo4j.isInt(object);

Next, if we’re sure that this number is 32-bit, we use this code to convert the object into a standard integer:

function convertInt (neoInt) {
  return neo4j.integer.inSafeRange(neoInt) ?
neo4j.integer.toNumber(neoInt) : neoInt;
}

If you want more functionality (e.g. to test if it’s safe to convert an integer before you do the conversion), consider using long.js.

To convert your Neo4j response into a KeyLines-friendly format, we’ve created a downloadable JavaScript file for you. Download ‘klbolt.js.zip’

The file works in a number of scenarios – including in a NodeJS setup or just in the browser. To get klbolt.js running in a NodeJS environment, add this code to your server.js file:

// in a server.js file:

const klbolt = require('klbolt.js');
// official neo4j bolt js driver
const neo4j = require('neo4j-driver').v1;

// allow cross-origin access
app.use(function (req, res, next) {
    res.header("Access-Control-Allow-Origin", "http://localhost:3000");
    next();
});

const driver = neo4j.driver(
    configs.boltPath,
    neo4j.auth.basic(configs.username, configs.password),
    { disableLosslessIntegers: true },
);

driver.onError = function (error) {
    logger.error("Error starting neo4j bolt driver. (server: " + boltPath + ')', error);
};

async function runNeoBoltQuery(query, params) {
    const session = driver.session();
    return session.run(query, params)
        .catch(function (error) {
            logger.error('Error running neo4j query', error);
            session.close();
        })
        .then(function (results) {
            let data = null;
            session.close();
            try {
                const parser = new klbolt.Parser(neo4j);
                data = parser.parse(results);
            } catch (e) {
                throw e;
            }
            return data;
        });
}

// // And an express handler:
app.get('/neo4j', async function (req, res) {
    runNeoBoltQuery(req.body.query, req.body.param)
        .then((data) => {
           // return the formatted data for KeyLines
           res.send(data)
         });
});

Ready to give it a try?

It has never been easier to integrate Neo4j databases with KeyLines. We have other useful Neo4j blog posts to help you make the most of your Neo4j visualizations.

You’ll also find more in-depth Neo4j integration tutorials and demos on our KeyLines SDK site.

FREE: Start your KeyLines trial today

Visualize your data! Request full access to our KeyLines SDK, demos and live-coding playground.

TRY KEYLINES