Power to the people: Charting

Its great to have a team of people to do analysis for you. I love being one of the people who can do the analysis that people want to see (and the analysis they don't think they want to see!) It usually works something like this:

They ask for some analysis
I do the analysis and pass it to them
Upon seeing it, they decide they want something a little different
Go to 1

Anticipating this, I make sure I'm pretty clear what the user wants and I try to provide analysis in a flexible way so that it also can be used to answer questions around the edge of the specific question that they asked. This works well, but its becoming clearer that there is a much better way to structure it: give the user the tools to interrogate the data themselves. I'm not talking about giving them a database, or even a spreadsheet. I'm talking about giving them the analysis you would otherwise provide, but making it such that they can easily change it and play with it.

My earlier post on mapping started to make this point. A GIS map in PowerPoint is a great output from analysis, but a Google Map where the user can add and remove layers and easily alter the data that is shown does all that and MUCH, MUCH more.

I just came across another example to do with charts.

A question that any number of people have been asking around election time for as long as I can remember is 'who votes when?'. What type of person votes early / late / on their lunch break? People have all sorts of theories, that I've always found to be not based on data. So I got some data to answer the question. Here's a chart I did showing Democratic votes, Republican votes and Independent votes by time of day in a recent election.

Here is the turnout by party the voter is registered with.

Not much of interest here. Some difference, but not much of a difference and certainly not enough to use to change turnout operations.

Here is the turnout by age:

Much more interesting now. Big differences by age of voter.

I did a few of these and PowerPointed them up and sent them around. But I knew what would happen. People would want something else. What about party registration and age combined? What about urban and rural differences by age? What about income and party registration crossed?

So I squashed down the several million record database in to a table that would fit in to Excel's 65,536 record limit (while preserving the key data that the user cared about) and stuck on a pivot table and a pivot table chart that allowed the user to change the chart and answer all of the questions above by simply pointing and clicking. This allowed the user to interrogate the several mission records with no database skills required.

I'd love to hear about other examples or tools that can do this. That can provide output-ready analysis for non-technical users, but that at the same time give power to the user to alter and tweak the analysis. Examples that give Power to the people!

One example of one of the tables that I created, that fit nicely in to excel was:

SELECT Count(VoterID) AS NumberOfVoters, Time, Gender, Income, Party, AgeGroup, Race ...
GROUP BY Time, Gender, Income, Party, AgeGroup, Race ...

Pete Tong's International Music Summit last week gave me the perfect excuse to use one of my favourite visualisations. Here we use EMI's consumer insight to look at how passionate 17 countries are about dance music.

To get a good picture of what's happening you need to think about not just what percent of people are passionate, but also what that means in terms of the number of people who are passionate. So I like to chart them both and then join them with lines.

This allows you to pretty quickly draw out some fascinating observations. For example: just look at how low the passion is for dance music in the US and yet it's such a big country that it has by far the most people passionate about dance music.

Whereas the UK scores pretty high on both metrics. I called it the 'biggest developed market'.

My full report is here and EMI have made all of this data available by country by demographic group by genre. Take the data and have fun with it! Please share what you come up with!

I gave a talk at the Big Data Insight Group in London recently and they've just posted my talk online.

I talk about how we've helped EMI Music make use of data and about how we're doing so in zeebox.

One of the themes throughout my talk is the importance of people. Both in terms of how we use data to help people make decisions and about how we need to understand the people we're trying to help, in order to give them what we need. Technology enables this, but without the right people and without understanding people, technology is as good as useless.

I also talk about how important skills and judgement are. And that, although it's sometimes seen as the things that drives decisions, it's usually or perhaps always used alongside skills and judgement.

I think that admitting to the role of skills and judgement isn't being 'anti-data'. I think that being honest about this enables and empowers us to better use data in the right ways. And it certainly helps people to feel comfortable with data, also!

With the right people in place and data playing the right role in an organisation, the opportunity for data to help an organisation is massive. The way that EMI Music has embraced data across the organisation alongside skills and judgement shows that this is the case.

We all know there are decisions where you need data to help you make them and there are decisions where data just isn't that important. This morning XKCD did a wonderful job of illustrating it. http://xkcd.com/1036/

Buying a lamp is a creative decision. Turn your eye away from the reviews and go with your heart :)

The same is true of many decisions data folks are asked to help with every day in organisations. We shouldn't be afraid to champion this strategy there, either!

We sat down recently to talk data and insight. Here is what we talked about, plus a little video of me talking about insight at both zeebox and EMI.

http://www.thebigdatainsightgroup.com/site/article/david-boyle-emi-zeebox-data-driven-includes-video

I don't like the term 'scientist' as it makes the role sound unaccessible and elite. Google's Hal Varian said "the sexy job in the next ten years will be statisticians" ... but I don't like that term either. I'd replace 'statisticians' with 'working with data' or something ... and then I believe it! I think data people have a tendency to overplay the role of the 'statistics' and magic of it and underplay the importance of the 'bringing it to life' and 'helping people understand / make use of it' parts of working with data.

I thought about this because of this cool article in The Guardian about data scientists.

As it points out, "science" is defined as "systematic study of natural or physical phenomena". I guess that's us all. Perhaps I shouldn't shy away from that phrase.

The journalist describes the role well, as "someone who can bridge the raw data and the analysis - and make it accessible. It's a democratising role; by bringing the data to the people, you make the world just a little bit better." Perfect, eh?

One last quote: "the four qualities of a great data scientist are creativity, tenacity, curiosity, and deep technical skills." That list sounds pretty good to me, also. So perhaps I should rename this the 'data scientist' blog and be done :)

So much data, so easily displayed in such a small but easy to understand format. I need say no more. I'm in love with the new sparlklines just made available in Google Spreadsheets: http://support.google.com/docs/bin/answer.py?hl=en&answer=2371371

It's this simple:

Google Spreadsheets is rapidly becoming my go to choice for building business dashboards. Bye, bye cost. Bye, bye developers (would be VERY sad not to work with them, of course). Bye, bye Microsoft!

I spoke on a panel last night on the subject 'data as the new black gold'. There are three challenges I think this metaphor poses to the data world.

First, that of crude oil. Data is everywhere in organisations, but too often left in it's crude form: gloopy and unusable. The oil industry had to work this out before it could be mainstream. It had to refine oil to a form that works for consumers day-to-day and it had to make it available to consumers in ways that fitted in to their daily life. It's trivial to stop by a petrol station and pick up some oil in a format you can instantly make use of. Data doesn't yet work the same way: it's rare to find an organisation that appropriately refines it and then makes it available to it's people in a way they can access and make use of as part of their day-to-day work.

Second, I think we need to demand higher 'miles per gallon' from our data. Often we gather fantastic raw data, capable of being a really powerful part of decision making ... but then business leaders don't ask interesting questions of it. They don't demand smart analysis and challenge the data to offer insight. It's like demanding that cars offer higher miles per gallon from the oil they are burning.

Finally, I think we need to embrace hybrid technology. In cars that's about oil being only part of the story for how the car gets powered. In data it's about saying that data is only part of the story for how organisations get powered. We need to be honest and bold about the role of skills & judgement alongside data in powering organisations. Too many people believe / pretend that data alone can power organisations to greatness. Everything I've seen tells me that data is necessary but not sufficient: smart people to use the data alongside their expertise is ALWAYS required. The data world should be honest about this and build data and systems around that truth. I've always found that has a much greater impact :)

I've used a lot of word clouds recently. But I think of them as charts really, since they are still pretty faithful to the underlying data. The size of the word is proportional to the number of times that word is in the data set. Simple.

But reading a cool data visualization book I came across this. Really it not based on 'data', but it's interesting his words and their location on the page conveys such a lot of information. Perhaps some good, well placed words can replace the need to chart actual data?

http://creativeroots.org/2011/03/italy-infographic-map/

Simple, easy to read, but really powerful. Nice little sparklines spotted in the papers from the 20 week scan my wife just had. Cool little chart like this should be everywhere!

And by the way, it's a boy!

Power to the people: Charting

View comments

Info Clarity

It's all about the people

Add a comment

Knowing when to use data and what to use it for

Add a comment

A conversation with the Big Data Insight Group

Add a comment

What's a data scientist?

Add a comment

The life of an analyst: Fun with charts

Add a comment

New sparklines in Google Spreadsheets are AWESOME

View comments

Data as the new 'black gold'

Add a comment

Sometimes words can replace data

Add a comment

Little sparklines in hospital notes

Add a comment