User testing ought to include extra dimensions on the User attribute

Disclaimer: I'm not a designer, but I wear the hat from time to time. I think by now in 2013 a majority of interface designers have arrived at the conclusion that user testing is really important to help shape an interface and designers who don't do user testing are met with "wat?" as the response. This user testing can take many forms: from hallway usability testing with an actual web mockup, or paper and sticky notes, or a whiteboard, to live A/B testing, and beyond. Designers try to find the best interface for everyone, and that's what gets deployed.

There's a problem, though: some users are different in important ways. Is the user young and presumably knowledgeable about modern UI idioms (like pinch-to-zoom), or is the user old and this is their first time with a touch device? Is the user English-speaking or alternate-language-speaking? Is the user introverted or extroverted? Is the user an active user or infrequently-active user? Is the user a "technical/power user" or a "non-technical user"? Don't you think it's worth exploring alternate designs for each of those alternate categories? You should be exploring alternate designs anyway, but if you have some theory driving some of your choices, you can apply what theories say about different user groups.

For instance, generally "flat" designs work better for technical users who are willing to spend some time learning and remembering where everything is on the screen, and "nested" designs work better for non-technical users where they are presented with only a very limited set of choices. Sometimes you might want to support just one or the other: for instance, who would design an airplane cockpit for someone who's not a pilot? And who would design an installation wizard with all sorts of power user features and a single screen when almost every user just wants to install the thing and go through the steps as quickly as possible, with the only choice they want to have being the "Next" button?

But what about web interfaces that are used by many technical and non-technical users alike? Operating systems? Myself and a lot of technical people I know can't think of a single design for a product that was better now, in 2013, than it was in 2012, and in some cases in 2011, 2010, and so on. In contrast a lot of non-technical users seem to love the changes, or at least they accept them and don't complain with a wish for old days. (A few examples of designs I think have changed for the worse: Gmail, Youtube, Google Docs/Drive, and Ubuntu.)

If you're designing a turn-based combat RPG, you might have the main characters on the left side of the screen and the enemies on the right side of the screen for your English port, but for your Japanese port maybe it would be better to put the main characters on the right side of the screen and the enemies on the left side. Similar swaps for other elements of the game interface. Why? Because English speakers read left-to-right, Japanese read right-to-left. Maybe you'll find no appreciable difference in user responses from your tests, but if you bother to think about localization at all then you ought to also consider how your interface can change apart from "render this text instead of that text".

Isn't this more work? Yes! But it can lead to happier users if you have separate interface that suit their individual needs better. (It can also lead to bigger paychecks.)

With a modular interface it may not even be too much more work. Gmail Labs is one of the greatest things about the service: power users who are interested in alternate interfaces can test them out, and the designer can see what works or doesn't work purely from the number of people who switch a component and switch back. Unfortunately Labs have been given less and less love over time and the Gmail design team has actively forced horrible redesigns on users that there isn't a Lab plugin to undo.

How do you find out what additional subcategories a user falls into? Well, with Labs, Gmail never tells you upfront that they exist, you have to find them in the mail settings. Most users are can find the settings button on their own are probably at least "mid-technical". So you the designer could just add features that users who might fall into X group would likely find on their own (such as a Japanese Flag in a sea of English text that is intuitively understood to mean "go to Japanese version of page"), or you could use some machine learning classification algorithms based on actions users do and make best-guesses, or you could just ask... I wish support forms had a checkbox asking "Are you a technical user?" so that I could check it and get put through to someone who isn't going to say something like "Have you tried plugging it in?"

Major products already have mobile and non-mobile versions. They fundamentally get the idea that different classes of users have different needs. But what's missing is the granularity: instead of just one or two designs, why not more, if you can show that more will be helpful?

---

Posted on 2013-11-06 by Jach

Tags: design, philosophy, rant

Permalink: https://www.thejach.com/view/id/291

Trackback URL: https://www.thejach.com/view/2013/11/user_testing_ought_to_include_extra_dimensions_on_the_user_attribute

Back to the top