Often designers of visual analytic tools (or authoring tools like Tableau) assume that latency is low, and much effort has been put into making computation more efficient. However it's not always possible to have guaranteed "interactive latency", of less than 500ms.
In practice, these interactive speeds are quite challenging to deliver reliably, as the 95th percentile network latency exceeds 300ms for WiFi networks even if data processing time is ignored. This requirement poses a challenge for traditional visualization tools when applied in Cloud and Big Data environments
Assuming short latencies causes visualizations to be not only unpleasant to use but often lead to wrong results. Visualizations break under long latency in different and often opaque ways.
Hopefully, by the end of this post you will be convinced that one ought to design with latency in mind, that there is a simple, clean model to capture these asynchronous behaviors, and that there are a rich design space and many benefits!
To start getting a sense of what it means to design with latency in mind. Let's first take a look at a simple, blocking design that's often used in practice, for instance, in Tableau. The blocking interface prevents you from performing another interaction if the previous interaction has not loaded.
To give you a better sense of what is going on with regards to the interaction requests and responses, we illustrate an "interaction timeline" to the right, where on the top row you will see the "accepted" interactions in blue, and "rejected" interactions in red cross. Your interactions may be rejected because the interface is busy processing your last response. And on the bottom row, the received responses of your interaction.
To make the set up more realistic, assign yourself a task of finding the maximum value of the year 2010 across the months, or see if it ever crosses 80, or maybe even try to grasp the basic trend across the months in the year 2010.
This blocking design could be annoying if you no longer want to see the result that is being requested. The following design allows you to intervene, and see only the most recent result. You will see in the lower rows that your previously requested response is rejected, in red cross.
So this "non-blocking" interface is perhaps a bit better as it allows you to intervene whenever you please. However, it still is subpar because you can only see one result at a time. What if I just want to see all of the results as fast as possible?
Our initial hypothesis is that perhaps people can still make sense of results out of order, if the task is simple enough, such as seeing if a month's value crossed a line. Perhaps you can give it a try.
We found, through an experiment on Mechanical Turk, that people were very reluctant to experience the results arriving randomly---they just waited for the previous result to arrive. Which got us thinking... Can there be anything that helps? We know that websites load asynchronously all the time (think Facebook, Twitter, Gmail), and often we can make progress using the application without waiting for the whole thing to have loaded. Are there any ideas we can borrow from there?
You can play with different settings here:
For multiple results to be meaningful, we added some annotation to the charts, which brings the following design you see, go ahead and play with the visualization.
Hopefully, you have discovered that you can interact in parallel and make sense of the results. We hypothesize that this effect is because the history of all the interactions creates a stable interface that will not change randomly due to latency, as is seen in previous cases.
If the key here is to use history to transform transient interactions into something stable, we probably do not need to show all of history---see below for an example with a limited buffer. In fact, there are generic, well-established visualization mechanisms that support these goals--- below is a quote from Tufte that grounds our proposed design.
Spatial parallelism takes advantage of our notable capacity to compare and reason about multiple images that appear simultaneously within our eyespan. We are able to canvass, sort, identify, reconnoiter, select, contrast, review -- ways of seeing all quickened and sharpened by the direct spatial adjacency of parallel elements.
Parallel images can also be distributed temporally, with one like image following another, parallel in time.
We also started thinking, if this is just visualizing history, time is essentially just another dimension to be visualized, we are not limited to using small multiples. We can overlay the charts, as follows.
Furthermore, if we just need to show correspondence between the interaction and the result, we can change how we encode interaction history --- doesn't have to be shades of the same color.
Play around with the parameters of this design, which we are calling "chronicles" (as in chronicling your interactions). If you change the buffer size to 1, it reduces to the original designs we talked about earlier.
We speculate that different corners in the design space will have different tradeoffs and should be adapted to different kinds of visualizations and tasks. However, a more pressing question on your mind at this point is probably...
Asynchronous designs could be applied to other scenarios that don't seem "parallelizable" immediately. We hope the following examples could illustrate ways to generalize asynchronous designs with chronicles---just visualize short-term history and the correspondence between interaction and the corresponding results!
See the following example of zooming on a scatter plot. Everytime you interact, the corresponding interaction shows up immediately, with a small legend that is your actual interaction, so you know that your interaction is acknowledged and remind you of what the result is actually for.
Here is an example of cross-filter using chronicles. Use the top row, light green colored visualizations to interact (the generated charts cannot be interacted with). The small black bar on the bottom indicates the specification of the brush that was used to filter the value. As you interact, new data will be appended, in reverse order. Cross-filter is a fairly complex interaction, and you can still apply Chronicles. The key here is to ensure that the controls are data independent, which means that you are free to specify your interactions regardless of the loading situation. It also helps to stabilize your navigation as well.
In fact, having a notebook style persistence of visualizations results can be pretty handy for comparisons. For the future, we plan to support "clipping" interesting interaction results, so that they will not be buffered out.
This technique could apply pretty generally to all UIs---most user interface can be thought of as a visualization of some information (see Bret Victor's magic ink). For instance, sometimes entering a value in a form may bring up a different popup window that takes forever to load; it would be great to have Chronicles.
We don't think that this is the only solution for what we call slow interactions, there are other techniques like progressively showing results as they update. It would look something like the following:
In fact, you can compose the two techniques together. Chronicles is probably not going to help if the latency is longer than 30 seconds, which can happen with larger data/more complex computation, and progressive visualizations could also potentially take a while, especially if the user wants to have some threshold of the error bound. Below is an example composing the views.
Lastly, chronicles designs are none trivial to implement and require a "time-centric" way to treat the application. We will talk about that in another article.