Author Archive

Designing Data – Part 5: Launch and Iterate

May 24th, 2012 by Matt

We’re asked about our design all the time – usually in an incredibly kind way full of high fives and “how’d you do that?!”s but sometimes in a “ugh, did you even think about talking to a customer??” kind of way. So, we decided to give you a week-long deep dive into our design process in this “Designing Data” series. Today's the last post in this series...though iterating never ends, so maybe there will  be more down the line. Stay tuned!

We make changes constantly. Seriously. Every day.

Sometimes they’re small and only our most dedicated users notice them; sometimes they entirely change the way people interact with our data. We don’t look at these changes as regressions, dead ends, or a waste of time; we look at this as the right way to design products. We vet everything intensely, but we can only do so much in the confines of our office with a few dozen users in pre-launch testing. And the only way to know if we’ve succeeded is to put it all out there into the real world (even if it sometimes results in a few harsh Tweets headed our way). So, when a new design is launched, we can't just pick up and move onto the next thing. We listen. We read every email coming into customer service, we consider every tweet, we grab everyone who wants to talk to us about what we’ve done and we listen. We figure out where things rocked and where things sucked and then this whole cycle starts again. If you've been following this design blog series, you know that in our initial design we tried to answer the question our users were always asking, “what's special happening on my site right now?”  with the new dashboard Overview. We stripped out a whole bunch of detail and put the focus on Notable Pages (an algorithm that detects what pages have unusual traffic) and Peer Stats (information from similar sites.) With the combination of those two stats our dashboard was always able to alert our users when something surprising, unique, or interesting was happening. But when it got out there into the wild...it turned out that people wanted to see more. They didn’t realize it at first, but it wasn’t just “is something special happening?” that they wanted answered rather it was “is everything normal?” While the questions sound similar, there is a world of difference between the two. We had built a dashboard that always caught and called out the big stuff, but it had trouble distinguishing between a normal day and a really bad one. We thought users would turn to the other views by clicking down their left-hand nav to understand these more nuanced things, but we were quickly told they wanted to put these new things on a second screen to always have a pulse of their site, even if it was just to tell them that everything was going smoothly. So we went back to into research mode. We asked what it meant when something was running smoothly, we watched how people used a dashboard when they weren’t paying attention, and we found out what screen they put our dashboard on. We learned a lot of little things (like that people liked to put our dashboards on big screens, but there wasn’t enough contrast in the colors to display information well enough at that scale). We got some bigger ones, too, that weren’t solved by changing colors. It turned out it wasn’t just the surging pages they wanted to see... they also wanted to know if anything unusual was happening with their more always-trafficked content.  So we brought back Top Pages and showed them with the Notable ones as context. Now the dashboard doesn’t just show you when something special is happening - it also lets you know when you’re doing alright and everything's just fine. But that only solved things on the individual page level when the information about the people sending it was every bit as important. So we changed the visualization of Traffic Sources to show two new things: the top referring domains and search terms and how your performance today compares with your average performance. Now, no matter if your day is normal, great, or terrible, you can tell right away. And that was the point all along. So welcome to our new Overview page. We think it answers the three questions that were most important...
  1. How many people are on my site?
  2. What are they looking at?
  3. Where are they coming from?
  ...but maybe you don’t. That’s awesome. We want to know. This process won’t stop until we do...

Designing Data – Part 4: Design and Prototype

May 24th, 2012 by Matt

We’re asked about our design all the time – usually in an incredibly kind way full of high fives and “how’d you do that?!”s but sometimes in a “ugh, did you even think about talking to a customer??” kind of way. So, we decided to give you a week-long deep dive into our design process in this “Designing Data” series. Yesterday we looked at the data, and today we're talking about designing against it.

Once we know that we’re solving real problems and using good data, we get to the part that everyone sees, touches, and (hopefully!) falls in love with: The design.

While we love delighting users and get pretty jazzed when people ohh or ahh over what they see on screen, we make sure to have a few people on our product design team who are solely responsible for making sure that the design uses and highlights the data we're displaying in the best possible way. So that it's not just pretty, but usable. Although this is the stage where pixels are pushed, lines of code are also written. Usually, we need to prototype a live design to make sure we can touch it, vet it, and sometimes get user feedback on it, regardless of how rough it is. Whether it’s quickly building a navigation system to make sure it functions instinctively or prototyping a visualization with real data pumping through it to see what it looks like -- a functioning, built design lets you see things that you never would in the static world of Photoshop. For this very reason, every designer on our team is also a developer. When pulling together the original Chartbeat dashboard Overview, we went through a bunch of prototypes that were pushed out to small groups of our users. Well, they didn't love everything, that's for sure. That sucked. We'd spent hours/days/weeks designing and prototyping....and it just didn’t work. But we don't try to make it fit or fall on our sword and go with it anyway. We made that first version of the Overview better. From the beginning, our goal for this view was to provide a high level, at-a-glance view of the site so it was obvious if something interesting or unusual was happening. Some of our early concepts were knocked down by users mainly because we weren't answering the questions they were asking, the needs they had at that instant level. Honestly, many of these concepts were visually much more appealing than the design we launched with but, as always, the goal isn’t only to make things pretty; the goal is finding the delicate balance between beauty, usefulness, and delight. To us, that's the core of good design. That's is what makes people who don’t have time for raw data able to understand, use, act on, and enjoy our products. So what were those pretty (but not so useful) designs that didn’t make it for Traffic Sources? The radar chart is a beautiful visualization and has the advantage of being able to quickly compare shapes, but it falls apart in the eyes of many users who are trying to get precise values of individual traffic sources. A radar chart is intended to show balance, when in reality most sites don’t have (and shouldn’t have) an equally balanced set of traffic. So we canned that guy and moved on to the dot chart. The dot chart was definitely cool, but  it didn't work for us a different reason. We liked the idea of being able to show a shape, or thumbprint, of your site’s traffic and added a connecting line between the values. The line didn't have a value - it was just meant to aid the eye in spotting the dots easily. But users were interpreting it is a relationship between each traffic source, which it certainly wasn't meant to be. We took a hit on that one for sure. The last thing we ever ever want is to launch a misleading visualization! Then came the circle chart. This guy had a lot of promise. Circle charts can be beautiful and usable, but its main weakness really shows when trying to compare values on different radii. The eye gets a little tripped up. Not to mention that it's tricky to grasp at a quick glance - the whole point of the Overview. So we inevitably trashed this guy, too. How did we land on our "final" design? Check back tomorrow to read our last piece of Designing Data: Launch and iterate.

Designing Data – Part 1: The Mindset

May 21st, 2012 by Matt

We're asked about our design all the time - usually in an incredibly kind way full of high fives and “how’d you do that?!”s but sometimes in a “ugh, did you even think about talking to a customer??” kind of way. So, we decided to give you a week-long deep dive into our design process in this “Designing Data” series. Stay tuned every day this week for a next step in our design process - and let us know what you think about every one of them.

Design is pretty important to us.

We have one job: Taking a crazy amount of data and turning it into something meaningful, so people on the front-line - people who don’t always have time for numbers - know exactly what to do and when to do it. Without good design that’s just not possible. If it takes them 10 minutes to figure out what the numbers stacked in front of them mean, let alone what they should do with them, that data might as well not exist. But it's not just about pretty. Design is way more than the visual. Design is a process of understanding. Where do people want to be and how can we make it stupid easy for them to get there? When we say design we mean everything: from the look of things when your traffic spikes to why you want to know that in the first place. Design is a holistic process that's about people and solving their problems - and making it beautiful while we're at it. So we start, unsurprisingly, by talking to these people, watching these people. We don't lock ourselves in a room and run through scenario after scenario with fictional, ideal users and refrain from putting anything out there until it’s completely finished. We actually get out there and find out what kind of problems people are facing and solve those first. But we don’t stop there. After we’ve got a solid idea and confirmed that it works with a handful of our users we do something that’s a little bit riskier... We iterate in public. We put out stuff that isn’t done, and sometimes we’re not even totally confident in it. Because we - and our early testers - don’t have all the answers. But we’re getting ahead of ourselves. Tomorrow, we’ll get into the nitty-gritty of the user research portion of the design process and explain all of this launch-into-the-wild stuff later this week.