Server-side rendering of visualisations using NodeJS

The team in Berlin is currently working on automated reporting for our users. We identified a use case wherein users want to check summary information of Brandwatch queries as a periodic report, without needing to log into Brandwatch Analytics to check a dashboard.

The Automated Reports product fulfils this use case. Users can log into Brandwatch and set up a report to be delivered on a schedule, and then wait for an email that contains a link to a static report as well as the same report as a PDF attachment.

Another time we'll examine how this entire solution was built, but in this post I'm going to talk about what turned out to be one of the most significant challenges for our team - generating charts on the server in NodeJS.

JavaScript charting libraries

A lot has been written about client-side JavaScript charting libraries, so I'll not dwell on those here. It is, however, worth noting that we decided to work with d3 and some abstractions built on top of it (vega and plottable respectively) in order to provide charts for this project.

Two schools of thought

When it comes to charting in NodeJS, there are two primary methods that people turn to:

  1. Headless browser-based solutions
  2. JSDOM-based solutions.

Let's examine each of these in turn.

Headless browsers

PhantomJS is by far the most popular headless browser. The project describes itself as:

a headless WebKit scriptable with a JavaScript API. It has fast and native support for various web standards: DOM handling, CSS selector, JSON, Canvas, and SVG.

As such it's a powerful tool for this use-case. With decent Canvas and SVG support out of the box, it's a simple task to spin up Phantom and use any of the aforementioned charting libraries in order to render an SVG. The SVG can then be taken and placed into the report template.

Sounds great, but there's significant drawbacks Spinning up a headless browser instance to render a single chart seems pretty heavyweight, especially given that each report that we create is comprised of multiple charts. We could either rearchitect so that we render a whole report in Phantom in one go, or consider the alternative of using JSDOM.

Keeping the creation of charts modular was very important to us, so we decided to give JSDOM a go.

JSDOM

JSDOM is an incredible project. It's essentially an implementation of the DOM in pure JavaScript, specifically designed for use with NodeJS. JSDOM is typically used for simulating the DOM for use in web crawler code and associated projects, but it seemed like a fairly good fit for our needs too, given that it provides a virtual window and document that we can use.

It turns out that d3 is already usable within NodeJS. If we take a peek at d3's source on Github, we see the following in package.json:

"main": "index.js",
"browserify": "d3.js",

the main property defines an entry point for d3 when required in a nodeJS application, so let's dive in there and have a look at how it works.

var document = require("jsdom").jsdom(),  
    globals = {};

// Stash old globals.
if ("d3" in global) globals.d3 = global.d3;  
if ("window" in global) globals.window = global.window;  
if ("document" in global) globals.document = global.document;

// Set temporary globals to pretend we’re in a browser.
global.window = document.parentWindow;  
global.document = document;

module.exports = require("./d3");

// Restore old globals.
if ("d3" in globals) global.d3 = globals.d3; else delete global.d3;  
if ("window" in globals) global.window = globals.window; else delete global.window;  
if ("document" in globals) global.document = globals.document; else delete global.document;  

It's pretty straightforward. It uses JSDOM to create a virtual document, creates temporary globals for window and document based on that, requires in the d3 client-side code (which assumes these globals exist) and then tidies up after itself. Nice!

We'll go into the details of working with d3 on the server in the next part of this series of posts.