Managing Visualization State02 Mar 2015
How best to manage the state of interactive browser-based visualizations.
To illustrate the notion of interaction state we borrow an example from the site that demonstrates a technique called brushing. Brushed visualizations can be used as a powerful visual query tool, and make it possible to achieve direct manipulation of query filters on related data. For an inspirational example see Kai Chang’s Nutrient Explorer.
In a simpler example below from Mike Bostock on bl.ocks.org, brushing is used to define a subset of one dimension of the data. The selected values are highlighted by, in this case, applying or removing a CSS class that paints a dot red.
But now we go beyond the intentionally simplified rendering in Bostock’s example and ask what other potential degrees of freedom are possible in this one simple image. It is easy to imagine the underlying domain values changing, and when these values are returned we may find they exceed the current range, perhaps calling for the presentation of a scrollbar. A pinch gesture could perhaps expand or contract the range, or maybe we use a sliding window approach and pan horizontally using a swipe gesture. The long press of a mouse button could call up a magnifying glass, or the user could resize their browser window and the new ratio of width to height might trigger the presentation of a vertical axis and new vertical brush handles.
Each of these potential features would require associated state, the maintenance of which could be made local to the feature (as is the case above) or more usefully it could be held in a more central data structure to make it possible to inform other aspects of the page like the lists of food groups or specific dishes we see in Chang’s Nutrition Explorer.
But these examples still only cover events local to the visualization. Imagine a system design where one user’s brush arrangement represents just one submission in a larger collaborative real-time system, and before updating the actual display the user’s individual input is averaged with those of other users connected to a server. Maybe our collaborative visualization is connected to a mix of virtual and physical devices (see earlier posts on RESTful API Prototyping and MIDI API), raising again the question of which collaborative data repositories or devices are considered sources of ultimate truth. In such cases the need for thoughtfully managing state becomes paramount.
When considering applications in these wider usage contexts it becomes useful to add an abstraction layer to hold onto state, and when we reach that point it’s almost certain that our application could benefit from a client-side application framework. If understanding the code behind custom visualizations is already a challenge, blending it with frameworks like Backbone, Angular, or React can be all the more daunting. Here are a few key points to consider:
- Store interaction state centrally, but replicate only by necessity
- Maintain a strict separation between state and domain objects
- Strive for code that statelessly renders your views and visualizations
- Be selective in what gets rendered after a change to model or state
- Need for modes of interaction beyond simple clicks and drags
- Desire to leverage existing application and graphics libraries
- Potential need for multi-device collaborative input
- Bewildering abundance of choice in available development tools
Example (to come) - D3 meets React and Angular
In an example to come soon, we will combine D3 for visualization with React for stateless rendering and Angular for application coordination. For a preview of the concepts please follow this link to a video of a whirlwind coding session from Joe Maddalone of Egghead.io where he deftly pulls together exactly these technologies.
Even though each of these libraries is excellent for its own reasons, I must include a caveat that this particular combination of libraries/frameworks does not, on its own, guarantee satisfactory results. We might agree that pizza and gelato make a pleasant meal, but they probably don’t make a pleasing mouthful. We’ll need to ensure each of our libraries serves its own distinct role in the meal of MVC. We’ll let AngularJS handle the model and controller in MVC, and we’ll let React collaborate with D3 to produce the view. As for application state, we’ll start by handling this with React for now, but we may end up writing a custom module later.
For a little more detail on the issues, we want to avoid letting both AngularJS and React manipulate the DOM because they take such a different conceptual approaches to rendering. AngularJS seeks to augment the standard retained mode view, whereas React seeks to ensure consistency and performance by employing an immediate mode approach that pre-renders and then adds or updates DOM elements (for a quick definition of retained and immediate modes see this MSDN page on graphics APIs, and for a conceptual rationale of the value of immediate mode in React see this video with Facebook’s Pete Hunt).
But web browsers do render documents after all, not applications, and so ultimately both approaches are stuck with a stateful DOM. The React library lets us write our code as if we are re-rendering the DOM on every state change, but it does have code to deal with the state of certain DOM elements as we would expect (e.g. it doesn’t lose track of cursor position while updating textboxes, or scroll position while adding to lists). But in general it does what we were after by putting our own application state firmly under our control. In my experience there is also a performance benefit over AngularJS directives, and conceptually React components are much easier to understand.
For a little more background on our example, roughly three years ago the author of D3 wrote a blog post calling for new conventions for coding reusable charts. This eventually inspired another developer to suggest using React components to answer Bostock’s request, asking in turn: D3 and React - the future of charting components? And in terms of providing an MVC architecture to consume external resources, we’ve already introduced Joe Maddalone’s integration video at the top of this example where he shows how to get an AngularJS directive to delegate rendering to a ReactJS component, which in turn uses D3 to make a chart. In the next post we will put all these ideas together and then discuss where we’ve ended up.