{"id":1414,"date":"2019-02-15T10:32:09","date_gmt":"2019-02-15T10:32:09","guid":{"rendered":"http:\/\/kusuaks7\/?p=1019"},"modified":"2023-08-09T08:48:16","modified_gmt":"2023-08-09T08:48:16","slug":"third-wave-data-visualization","status":"publish","type":"post","link":"https:\/\/www.experfy.com\/blog\/bigdata-cloud\/third-wave-data-visualization\/","title":{"rendered":"Third Wave Data Visualization"},"content":{"rendered":"

Ready to learn Data Science? Browse courses like Effective Data Visualization<\/a> developed by industry thought leaders and Experfy in Harvard Innovation Lab.<\/em><\/strong><\/p>\n

\n

This is a write up of my keynote from the\u00a0<\/em>2018 Tapestry Conference<\/em><\/a>\u00a0(above). You can see the slides (from which I\u2019ve derived many of the images below) with speaker notes\u00a0<\/em>here<\/em><\/a>.<\/em><\/p>\n

Imagine what it was like to do data visualization 30 years ago. It\u2019s 1988 and you\u2019re using Excel 2.0 for simple charts like pie charts and line charts, or maybe something like SPSS for more complicated exploration and Arc\/Info for geospatial data visualization. Some chart types that have become rather ubiquitous, like the treemap, haven\u2019t even been invented yet. But in 1988, Edward Tufte\u2019s\u00a0The Visual Display of Quantitative Information<\/em>\u00a0was already five years old.<\/p>\n

<\/canvas><\/figure>\n

Footage of analysts operating Excel 2.0 to make pie charts c.\u00a01988<\/p>\n

Now, imagine what it was like to make data visualization 15 years ago. There was no D3, no Tableau, no ggplot or even Prefuse\/Flare. If you wanted to do network visualization you might use the newly published Cytoscape though it was focused on bioinformatics. Geospatial options were more advanced, with ArcGIS providing more and more cartographic functionality in its many red toolboxes.<\/p>\n

<\/canvas><\/figure>\n

The difference in the traditions backing geospatial data visualization and network data visualization: in 2003 it was already version 8.0 of ArcGIS whereas that same year saw the release of the first modern network data visualization tool: Cytoscape.<\/p>\n

I know how much the data visualization has changed because I spent the last ten years or so making data visualization products in one form or another. Big public-facing work like\u00a0ORBIS<\/a>\u00a0and\u00a0Kindred Britain<\/a>\u00a0as well as less public data visualization for fun or to support research, analysis and exploration. Along with practical work making data visualization, I wrote a\u00a0book<\/a>\u00a0about the most powerful data visualization library in the world today: D3. And then I wrote that book again. For the last year I\u2019ve been building out my own charting library,\u00a0Semiotic<\/a>. I\u2019ve also been running an\u00a0annual survey of data visualization professionals<\/a>\u00a0for the last two years. And along the way I\u2019ve written about data visualization here on Medium.<\/p>\n

That\u2019s not why I was invited to give a keynote at Tapestry. I was invited because I make incendiary comments on social media and in interviews and occasionally in talks. I have long said that we should be more comfortable with critique in data visualization but without context the remarks I make might seem arbitrary and mean-spirited.<\/p>\n

So I was happy to have this opportunity to give context and express my concern that there has been a convergence of tools and modes but no corresponding reorganization of thought and practice. It seems like we\u2019re still talking about and evaluating data visualization as if it was 1988 or 2003 when the number of people doing data visualization, the capabilities of their tools and the expectations of their audiences has dramatically increased.<\/p>\n

<\/canvas><\/figure>\n

We continue to split the data visualization community into old categories like analysts using BI tools to create reports, developers using code to make custom data visualization, journalists creating data-driven stories or data scientists leveraging exploratory data analysis. Within or between those groups we have data artists, professional scientists, business analysts and toolmakers of the kind we might see\u00a0immortalized in the cartoons of Susie Lu<\/a>. These categories of practice map directly to particular tools and modes that have, as of late, begun to transform.<\/p>\n

1st Wave:\u00a0Clarity<\/h3>\n

And so I\u2019d like to offer up that there was, in the modern sense of data visualization, a 1st wave centered on Edward Tufte that emphasized clarity, simplicity and direct 1-to-1 mapping of data points avoiding as much transformation as possible. From this era we see the rise of spartan color schemes \u2014 often focused on neutral or desaturated colors with one standout color \u2014 the importance of labels and natural language titles, and a sort of idealization of the perfect chart that is immediately readable, accessible and actionable. A sort of chart-as-sentence with clear structures and rules like you might see in The Elements of Style.<\/p>\n

<\/canvas><\/figure>\n

Wave 1:\u00a0Clarity<\/p>\n

2nd Wave:\u00a0Systems<\/h3>\n

The 2nd wave focused on systematizing the encoding of information necessary for the development of tooling to produce data visualization. It is centered on the most influential work on this subject:\u00a0The Grammar of Graphics\u00a0<\/strong>by Leland Wilkinson. Instead of the chart as a linguistic analogy, it is the chart as an extrusion from a valid specification. The Grammar of Graphics aspired to enable people to create an ensemble of graphics and if we take a look at any data visualization library, my own included, and see that philosophy in action.<\/p>\n

These tools and libraries advertise massive amounts of examples and gone is the careful use of color and text, replaced with a love letter to geometry. The entire \u201cfor example\u201d approach is so ingrained in these tools that\u00a0Mike Bostock wrote an entire piece on it<\/a>.<\/p>\n

<\/canvas><\/figure>\n

Wave 2:\u00a0Systems<\/p>\n

D3, like all the tooling from this period, is directly inspired by or heavily influenced by The Grammar of Graphics. Semiotic, Victory, Highcharts, Plotly: All of these come from or are themselves highly influenced by D3. The two g\u2019s in Grammar of Graphics is where\u00a0ggplot<\/code>\u00a0gets its name. Leland Wilkinson, the author, was a Principal Research Scientist at Tableau. The grammar of graphics focuses in a razor-sharp way on encoding data via channels onto geometry. This is a system for encoding graphics from data, where the data attributes correspond and dynamically affect the length, angle, color or position (or any other graphical character) based on the data and changes in the data.<\/p>\n

Wave 2 was about taking these theoretical systems and producing the tools necessary for any data visualization practitioner to create any graphical expression based on data. That\u2019s great for engineers and software architects trying to build libraries to allow for data visualization, but not necessarily for practitioners creating data visualization products. That\u2019s why we\u2019ve seen such a\u00a0proliferation of tools and libraries for data visualization<\/a>\u00a0but a concurrent rise in hideous graphics posing as charts.<\/p>\n

<\/canvas><\/figure>\n

The search for a perfect specification for encoding data attributes through graphical channels is a means to an end. But the power and success of the systems built during this period became an ends to themselves, which runs afoul of a basic precept of The Grammar of Graphics:<\/p>\n

This system is capable of producing some hideous graphics.\u00a0\u2026 This system cannot produce a meaningless graphic, however.<\/p><\/blockquote>\n

By this, Wilkinson means that there\u2019s a logical relationship between the graphics and the data but just because graphics have some kind of \u201crecipe\u201d does not mean it\u2019s somehow \u201cnot meaningless\u201d. Data visualization is a form of communication and if you produce hideous charts that means they are illegible and that means they are meaningless.<\/p>\n

Data visualization is a form of communication and if you produce hideous charts that means they are illegible and that means they are meaningless.<\/p><\/blockquote>\n

2nd Wave Data visualization, with its lack of emphasis on design in favor of systems produced a forest of horrible dashboards and reports. Every engineering department in every company has at least one ugly realtime chart written with some framework built on D3. Every executive has a dozen tableau dashboards filled with tabs and views of the same 10-color desaturated line charts. We see screenshots from what is ostensibly exploratory data analysis dropped into documents or integrated into dashboards without any sense of optimization. All of this happens in spite of the growing feature parity across technologies.<\/p>\n

3rd Wave: Convergence<\/h3>\n

We are in desperate need of reevaluating and renaming our modes. 5 or 10 years ago, what kind of data visualization you made \u2014 whether it was a dashboard, a notebook, a report or a bespoke communications piece \u2014 was very different depending on the language, library or tool you were using. That\u2019s no longer the case.<\/p>\n

We need to imagine new approaches which acknowledge that convergence isn\u2019t just happening in the capabilities of tools but also in the expectation of users who no longer are willing to accept that they need to exit out of one mode to optimize for another. That entails shifting our emphasis away from individual charts to the construction, evaluation and delivery of the products where those charts appear.<\/p>\n

<\/canvas><\/figure>\n

The\u00a0nteract notebook platform<\/a>\u00a0advertises itself not only to traditional notebook users (researchers and data scientists) but to a broad audience interested in more than\u00a0just<\/p>\n

Notebooks are becoming more dashboard-like, dashboards are becoming more storytelling-like and in general there\u2019s a growing cross-pollination and convergence amongst media\/modes. You can make beautiful graphics with R, you can have hierarchical charts in Tableau, you can easily deploy email reports from your custom dashboard.<\/p>\n

<\/canvas><\/figure>\n

At Netflix, we\u2019re experimenting with analytical notebooks designed not for exploratory data analysis but explanatory data visualization and the collaborative and communication needs required in that mode. Storytelling techniques common to data journalism are top-of-mind to stakeholders who have grown sophisticated in their tastes and expect cued animation and personalized frames of reference.<\/p>\n

There are more and more of these trends that we need to better understand:<\/p>\n