Archive for 2009

Looking back, looking forward

Monday, December 21st, 2009

2009 has been kind.

Professionally it’s been unsurpassed, despite the recession. Clearleft have grown to double figures, moved into a studio with decent wallspace, produced some great work, run two successful conferences and were humbled to be voted Agency Of The Year in the .net awards.

As the office winds down, colleagues jet off overseas and lunches linger into the afternoon, thoughts turn to gifts and time off. Since I opt out of the commercial trappings of the season, I’ve chosen this year to make my annual donation to WWF and Reprieve, two fantastic clients I’ve worked with this year. I’ll be spending a unique Christmas on a military base. In lieu of ubiquitous WiFi, it’ll be an opportunity to spend time with family, read, write and get my breath back.

2010 will be a year of abundance – and the first casualty, sadly, will be my carbon footprint. I have three speaking gigs booked so far (South by Southwest, the IA Summit and UX London) and as a punter I’m hoping to grab a seat at Paris’s Content Strategy Forum, Berlin’s UXCampEurope and New York’s Design for Conversion. But of course 2010 is likely to be dominated by the book. Emails are a-flying and chapters are a-forming. More on that soon.

Thanks for sharing this year with me and here’s to the next one! Merry Christmas.

{PS. It’s also the done thing to list your favourite albums of the decade. In no order, I’ll throw out Michigan, Tarot Sport, Change, Turn On The Bright Lights and Leaves Turn Inside You.}

Posted in Uncategorized | 2 Comments »

I blame the designer

Tuesday, December 8th, 2009

[In which Cennydd has a downright sense of humour failure over a silly web comic.]

Here’s an excerpt of a comic that recently did the rounds in the web design community.

You know what? I’m tired of this attitude.

Clients From Hell is admittedly pretty funny. Sometimes clients say stupid things; but hey, so do designers. I’ve said lots of them myself. But this sort of thing is different. It’s not an amusingly misguided email. Rather, it epitomises a harmful arrogance and entitlement that pervades the design community. It carries a bitter subtext that clients are idiots with no design skill, and it’s a designer’s duty to disempower them by any means possible.

And I’m tired of it. Of course clients aren’t skilled designers; that’s why they had the foresight to hire us. But you know what? They know business. They’re as passionate, committed and talented as anyone. Many of them put their livelihoods on the line to make the web happen. And let’s be blunt: they also pay our salaries.

If a web design project goes to hell this way, I usually blame the designer. He wasn’t skillful enough to make the situation work. He didn’t provide the force of argument required, couldn’t handle the politics, or couldn’t convince the client of the value of good design. On the rare occasion when the relationship with a client goes entirely rotten, the designer should end the relationship gracefully rather than passive-aggressively working to rule.

Unconvinced? I suggest you read Scott McCloud’s excellent post about criticism and the equally insightful comment from Mike L:

“The most common misconception about criticism is that one has to be on a similar skill level as the creator in order to have a valid opinion. I read stuff from many different artists from many different disciplines who cannot abide ramblings of people that couldn’t compete with them in some way. If said person is not an artist, their opinion doesn’t matter. But isn’t art, all art about communication? And who is the artist generally trying to communicate with? … My #1 critic is someone who cannot draw at all. He tells me things I can’t see because I overthink them as an artist.”

(Oh, and here’s what ‘pop’ means.)

Posted in design, web | 16 Comments »

Oxymoron

Tuesday, November 24th, 2009

This is the ELSE Mobile. It’s a touch screen phone. They’re all the rage, I hear.

I’ve not used the ELSE Mobile, but I know from their website that I needn’t bother. I know because they claim this handset demonstrates a:

“user-experience-centric philosophy designed to enhance man-machine capabilities through pre-integration services.”

With this lone sentence, ELSE instantly destroy any pretence of user-centred design. No user-centred company would let their copywriters produce such unmitigated nonsense. I barely need to mention the splash screen, the breaking of the Back button, the grammatical errors (“Most device are…”) and the autoplaying music on the Flash monstrosity they call a website.

This, dear reader, is the opposite of user experience design.

[Thanks to Lewis for the link.]

Posted in mobile, user experience | 8 Comments »

Latest Clearleftie happenings

Tuesday, November 24th, 2009

Lots to report, much of which I neglected to mention thanks to my brush with our porcine friends.

UX London 2010

First, we’ve announced our programme for UX London 2010, which features (amongst others) Jesse James Garrett, Scott McCloud, Whitney Hess and Bill Moggridge.

Once again we were delighted with how much of our Christmas wishlist came true. The north wall of the office has been awash with post-its of names and topics for several weeks now, and there’s a certain Machiavellian joy in seeing it come together into a coherent programme. I’m particularly happy to see some names I pressed especially strongly for.

Bands always say their difficult second album will surpass their first, but I think it’s true this time. Not a prog-rock bass solo in sight. It’s happening 19–21 May 2010, and tickets are on sale on 1 December.

Spring internship

We’re taking on a User Experience intern early next year. It’s the first time this is a dedicated UX position – our previous interns have come from across the whole web spectrum. It’s a paid position lasting ten weeks, and would suit anyone with a talent and love for good user experience design. More details are on the Clearleft site – drop us a line or talk to me if you’re interested.

The book

Finally, my big news is that I’m writing a book with James. Several daunting but hopefully inspiring months lie ahead. More details will follow when we confirm them.

Posted in conferences, personal, user experience | No Comments »

Statistical significance & other A/B test pitfalls

Monday, November 16th, 2009

Last week I tossed a coin a hundred times. 49 heads. Then I changed into a red t-shirt and tossed the same coin another hundred times. 51 heads. From this, I conclude that wearing a red shirt gives a 4.1% increase in conversion in throwing heads.

A ridiculous experiment (yes, I really did it) with a ridiculous conclusion, yet I sometimes see similarly unreliable analysis in A/B testing.

It’s logical and laudable that designers should seek data in our quest for verifiability and return on investment. But data must be handled with care, and mathematical rigour isn’t a common part of a designer’s repertoire.

Here’s an example from ABTests.com, a worthwhile project that I feel slightly bad to pick on.

The two versions are subtly different:

• Version A: Upload button bold, Convert button bold, Convert button has a right arrow
• Version B: All buttons regular weight, no right arrow on Convert button

Although minor changes can cause major surprises, I wouldn’t expect these small differences to improve the form’s usability. With the caveat that I don’t know the users or product, I’d even speculate that Version B could perform worse since it reduces the priority of the calls to action and removes the signifier of progression.

The designer claims that version B showed a 30.4% conversion improvement in an A/B test. Here’s why this isn’t quite accurate.

The role of chance

Any A/B test is a trial, so called because we’re observing evidence gained by trying something out. I can never truly know that there’s a 50% chance of a coin landing as a head or a tail – I can only run trials and observe the evidence. Similarly, we can never truly know that a design leads to higher conversion – we can only run trials and observe the evidence. If that empirical evidence is strong enough, we conclude that the design is an improvement. If not, we don’t.

To be valid, trials need to be sufficiently large. By tossing my coin 100 or 1000 times I reduce the influence of chance, but even then I’ll still get slightly different results with each trial. Similarly, a design may have 27.5% conversion on Monday, 31.3% on Tuesday and 26.0% on Wednesday. This random variation should always be the first cause considered of any change in observed results.

The null hypothesis

Statisticians use something called a null hypothesis to account for this possibility. The null hypothesis for the A/B test above might be something like this:

The difference in conversion between Version A and Version B is caused by random variation.

It’s then the job of the trial to disprove the null hypothesis. If it does, we can adopt the alternative explanation:

The difference in conversion between Version A and Version B is caused by the design differences between the two.

To determine whether we can reject the null hypothesis, we use certain mathematical equations to calculate the likelihood that the observed variation could be caused by chance. These equations are beyond the scope of this post but include Student’s t test, χ-squared and ANOVA (Wikipedia links given for the eager). Here’s a site that does the calculations for you, assuming a standard A/B conversion test with a clear Yes or No outcome.

Statistical significance

If the arithmetic shows that the likelihood of the result being random is very small (usually below 5%), we reject the null hypothesis. In effect we’re saying “it’s very unlikely that this result is down to chance. Instead, it’s probably caused by the change we introduced” – in which case we say the results are statistically significant. Note that we still can’t guarantee that this is the right interpretation – significance is about proof only beyond reasonable doubt.

Running the calculations on the above data shows that the results aren’t statistically significant: the evidence isn’t strong enough to reject the null hypothesis that the difference in conversion is simply down to luck. The main problem is the small sample size (128 and 108 users respectively), so I would advise the designer, Johann, to repeat the test with more users. Assuming the observed conversions seen didn’t change (a big assumption) a sample size of approximately 200 users per variant should be sufficient for significance. He could then either reject the null hypothesis or the results would remain inconclusive, in which case there’s no evidence the design has made a difference. In Johann’s defence, he recently posted that he takes the point about significance, and I’m looking forward to seeing more conclusive data for this intriguing test.

Percentage confusion

Significance isn’t the only slippery problem A/B tests face. For starters, quoting conversion improvements is always fraught with difficulty. Since conversion is usually measured in percentages (in this example, 31.3% and 40.7%) there are two ways to quote improvements. We can say that conversions increased by:

• 9.4% – the difference between the two
• 30.4% – the amount that 40.7% is bigger than 31.3%*

Any percentage improvement quoted in isolation should be challenged: which of these two calculations has been used? It’s dangerously easy to assume the wrong figure without sufficient context.

The A/B death spiral

A/B tests also suffer from a common quantitative problem, in that they tell us what but not why. I’ve written about this previously in What if the design gods forsake us. It’s wise to back up numerical tests with qualitative evaluation (eg. a guerrilla usability test) so we can make informed decisions if data suggests we need to rethink a design.

Even with backup, sometimes A/B tests are simply the wrong tool for the job. They can provide powerful insight in some cases, but in the wrong place they can be a blind alley or, worse, a weapon of disempowerment. Logical positivism and design don’t mix – not everything we do can be empirically verified – yet some businesses fall back on A/B testing in lieu of genuine design thinking. I call this the “A/B death spiral”, and it plays out something like this:

Designer: Here’s a new design for this screen. You’ll see it has a new navigation style, tweaked colour palette and I’ve moved the main interactions to a tabbed area.

Product owner: Wow, those are pretty big changes for such a high-risk screen. I tell you what: let’s test them individually to see which of these changes works and which doesn’t…

As the proverb suggests, sometimes you can’t jump a twenty foot chasm in two ten foot leaps. Cherry-picking only those design elements that are “proven” by an A/B test can be a route to fragmented, incoherent design. It may earn marginally more money in the short term, but it becomes hard to avoid a descent into poor UX and the long-term harm this causes.

Being faithful to data

Given the potential hazards, I’m concerned about the naïveté with which some designers approach quantitative testing. The world of statistics rewards an honest search for the truth, not dilettantism, and I’d advise any designer moving in statistical circles to pick up some basic stats theory, or at least partner with someone knowledgeable.

A flawed A/B test, be it statistically insignificant, misapplied or misquoted, is nothing more than anecdotal evidence. It’s the same crime as making a website red on the feedback of one user. Yet an impatient designer, seeing the example I quoted above, could quickly jump to a false conclusion: “I should remove arrows from continue buttons: it’s 30.4% better.” Perhaps this designer deserves what he gets. It’s likely he’s only really interested in shortcuts to good UX, and linkbait lists of “Twelve ways to make your site more usable.” Since he understands neither the mathematics nor the context of this trial (timescales, userbase, surrounding task) he will inevitably grab the wrong end of the stick. Nonetheless, he is out there.

Don’t let yourself be that designer.

Photo: snellgrove
* subject to rounding.

Posted in design, statistics, user experience | 41 Comments »

Why designers should care about HTML5

Wednesday, October 14th, 2009

Concurrently published on the quite excellent HTML5Doctor site.

After a while on the fringes of our collective consciousness, HTML5 is finally getting the attention it deserves. The development community (as typified by the SuperFriends) has come together to debate practical elements of the spec, argue over the inclusion of controversial elements, and assess the timeframe over which we can unleash HTML5 in the wild.

However those of us more accustomed to the world of Post-Its, sketches, and .psds – the designers – haven’t been so vocal. Perhaps we’ve been distracted by the bright lights of CSS3 and those surface thrills we’ve longed for. (Rounded corners! Gradients! Transparency!) Or, alternatively, we’ve been in the thrall of @font-face and looking forward to the coming age of passable web typography.

Understandable. But it’s time designers got excited about HTML5 too.

Partly, it’s just good practice. Whatever your flavour of design – visual, web, interaction, user experience – knowing the native technology makes you better at your job. Just as composers should understand the capabilities of the orchestra’s instruments, designers need to understand the language of the web.

But there’s more to HTML5 than simply keeping our skills sharp. It could make a big difference to the way we design for the web.

Semantic elements

Information architects (and, by extension, user experience designers) should be excited by the new HTML5 elements – <nav>, <header>, <aside> and so on. While they won’t immediately revolutionise today’s web, they’re an investment for the future. Doing useful stuff with information is the central theme of IA, and therefore its practitioners should be at the forefront of the new experiences that machine-readable semantics will offer. HTML5 allows us to mark text up in a more meaningful way than a sea of <div>s, meaning we’ll soon see applications appearing at a sub-page level. We’ve started to scratch the surface – think about the Operator toolbar or customisable UIs à la iGoogle – but we’ll need detailed design thinking to work out how to bring the benefits of semantic richness to the end user.

APIs and other extensions

While it’s clear that some of the HTML5 APIs are far from perfect right now, when they’re refined they will offer us intriguing new opportunities and challenges.

Designers of location-based services should of course find the geolocation API invaluable. [I stand corrected. See isgeolocationpartofhtml5.com.] The contentEditable attribute gives us further power to make the web truly read/write without resorting to JavaScript and custom interfaces. New input types (eg type=”search”) can provide extra visual cues about input function, although of course this depends on the solutions chosen by the browser manufacturers.

Until now, it’s been easy to consider our domain as bounded by the viewport and the web server. But HTML5 is another step toward seamlessness: the merging of desktop, offline and online. For instance, the drag and drop API could see the line between online and desktop experience blur further. Local storage could allow for a web-like experience in areas of poor connectivity. This convergence is clearly a good thing, but we must also design how to expose those hidden seams at the user’s request. Users should stay in control of how their locations are published and what data is synchronised to their machine.

<video>, <audio>, <canvas>

There is of course something of a reported schism between the standards world and the Flash world. Some see the advent of these new media elements (particularly <canvas>) as heralding the death of Adobe’s poster child.

I don’t think this is either likely or desirable. Neither technology is perfect. Flash is, of course, proprietary and thus subject to the whims of a third party that stands between browser and user. <canvas> has known accessibility problems. But the two can live in harmony, if we play to their respective strengths. Some current Flash applications might be better suited to <canvas>, particularly those based around dynamic visualisation: graphs, animations, infographics. Some applications will benefit from the powerful capabilities of Flash: games, heavily interactive widgets.

This aside, there’s clearly a user experience benefit in not having to rely on an external plugin to play rich media elements, and it will be interesting to see the uptake of the <video> and <audio> elements. Although it will initially be down to browser makers to define the interface elements involved, we will need to figure out how to integrate them into everyday web experiences. The good news is that they can be styled in the same way as any other HTML element. If your visual aesthetic relies on slanted images with box shadows, it’s trivial to apply this to video too.

That said, we can’t ignore the elephant in the room: the thorny codec issue. I’m sure we’d all agree that the sooner it’s resolved, the better.

What can designers do?

Only the most patient and detail-oriented designer will relish the idea of reading the spec in full and arguing the finer points on the WHATWG list. That’s just not the way designers roll.

But as a community it’s important that we start talking about HTML5. If you’re new to HTML, now’s a great time to learn. HTML5Doctor and A preview of HTML5 give useful guidance on the differences between HTML5 and its predecessors. Above all, designers should get chatting with their developer friends: there can’t be many left who no longer have an opinion on this technology. How do they see their practices changing? What can we do today to prepare our sites for the advent of HTML5? How can we build on its strong points to make the web a better place?

We don’t yet know what we’ll accomplish with HTML5, but then it’s not often that the vocabulary of the web changes this deeply. However, one thing is clear: if we prepare now, we have a great chance to bring innovation to our users’ online lives.

Posted in development, web | 1 Comment »

Q&A: getting into user experience

Tuesday, October 13th, 2009

For the past few years I’ve given an annual talk at UCL to students of the HCI with Ergonomics M.Sc. It’s always a pleasure to share my questionable world view with impressionable minds, and I look forward to the sessions in much the same way as one secretly enjoys a visit from a drunken uncle.

In an effort to make this year’s session a little more interactive, I pulled out an old Knowledge Management set piece:

1. Distribute post-its
2. Ask everyone to write one question they wish they knew the answer to (preferably about the topic at hand).
3. Stick the post-its on the walls. (It’s surprising how much people group them, despite your invitation to use any of the three free walls)
4. Ask everyone to read each post-it.
5. If they too want to find out the answer to a question, tell them to mark the post-it with a question mark. If they think they have an answer, mark it with a tick.

It’s not that surprising to find that a room of similarly qualified students share similar concerns. What’s more interesting is that many of them can also help to answer each other’s questions.

The purpose of this exercise is of course to show that networking and collaborating is valuable, and not just a case of awkward conversation and limp handshakes. However, having made this slightly facile point, I realised that most of the posted questions were damn smart and deserved to be shared more broadly. So here are a few that were particularly interesting, and some proposed answers from myself. I’ll throw a few more up later this week.

Please contribute in the comments if you have any opinions, particularly if they differ from my own.

Is the graphic design of a site more important than usability when initially attracting users to the site?
I say yes. Research shows users form an opinion on the credibility of a site within milliseconds of visiting it. To form a valid opinion on usability takes use, which may not happen if those impressions are negative. However, the line between the two is of course blurred, and a site can successfully convey usability through layout, visual design and information hierarchy. There are plenty of other factors that have an impact too: load times, content and proposition spring to mind.

How many hours do you work a week?
Define “work”. I’m paid for 37 hours, and most of that is spent on billable client work. But add in commuting, writing articles and conference talks, mentoring, and reading about my field and it would exceed 60. Yes, I’m aware that’s a little unhealthy. Good thing I enjoy it.

What’s the most useless skill you think we’ll learn from this course?
Probably rifling through academic papers to find an authoritative source that proves or disproves a detailed HCI argument. Truth is, not many people in industry will care. It’s more important to judge the the problem at hand and make the right design decisions based on context. HCI theory can give a strong advantage here, but you’ll need to state your case with something more real: usually how your client will make more money by following your advice.

How much do you get paid?
Not telling. But here are some approximate London figures: £25,000 is fair for a graduate-level position, rising to £35–40,000 with a couple of years of experience. Senior people should be looking at £60,000 and up (seven years and above, probably managerial responsibility). Freelance rates typically range between £275-£400/day.

What are the best design tools in HCI?
Thinking, conversation, sketching, software. In that order.

Can you be a good UX designer and a good programmer at the same time?
You can be good at both, yes. But who wants to be just good? Deep specialists tend to better than jacks-of-all-trades, and only extremely rare superheroes can be world class at both. I do, however, strongly recommend that all designers learn to code to a reasonable standard, and that all developers learn the fundamentals of design. Speaking each other’s language is the easiest way to ensure good designer-developer relationships, and one of the easiest ways to become substantially better at your job in a short time.

Do you need to draw well / be arty to be a user experience designer?
Some drawing talent helps, but sketching well is a skill that can be learned and that comes with practice. Its main value is when communicating with clients – a well-crafted sketch can simply convey more information than a poor one. However, it’s more important to develop a designer’s mindset. As Jason Santa Maria says, “sketchbooks are not about being a good artist, they’re about being a good thinker.”

To finish, two questions I don’t feel fully equipped to answer. How would you answer them?

• What’s the most interesting HCI related job out there?
• How do I get into the user’s head?

Posted in design, user experience | 5 Comments »

The behaviour you design for

Thursday, October 8th, 2009

I’m working on a site that’s grown from no deal to big deal. Earlier design oversights have created user coping strategies so ingrained that I mustn’t disrupt them with my new design work.

Another reminder that you get the behaviour you design for.

Photo: Tim Bradshaw

Posted in design, user experience | 2 Comments »

• « Older Entries