WebSort : easy online card sorting [a review]
WebSort by UXPunk is an online tool designed for conducting and analysing remote card sorts. Its target market is website architects (or webmasters). Cardsorting aims to help solve the problem of organising a site’s content to meet audience needs and expectations.
Now that ambition is an obvious target for this novice bereted librarian, so I went to find out more.
The basic idea is simple, as Donna Spencer put it:
Get people to put stuff into groups that make sense for them.
Online card sorting is useful, as Neil Turner concludes, when you need lots of participant sorts or when you can’t easily set-up, or need to supplement, face-to-face card sorting sessions. Online card sorting tools offer the added advantage of automated statistical analyses and rapid production of statistical reports. Not, as Courtney Clark points out, that such quantitative data could suffice to gain insight into audience mental models .
How Websort Works
As outlined at the very inviting website:
- You enter the items you want sorted, refine the study parameters and invite participants;
- Participants drag and drop items into groups that make sense to them – and in open sorts define their own group labels;
- WebSort compiles the results into a variety of statistical reports.
WebSort‘s features & options
|Features I used||Features I did not use|
|Labels, Descriptions (‘tool-tips’), and Images||Paste in items from text file or spreadsheet|
|Customize the instructions||Display your own logo|
|Let participants define the categories (an ‘open’ sort)||Predefine the categories (a ‘closed’ sort) (can allow partipants to add)|
|Randomise item presentation||Sort in English, French, German, Portuguese, Spanish, Japanese, and Swedish|
|Set limit to number of participants and closing date||Use tags to group different kinds of participants or Bypass participant login|
|Drag and drop interface||Track the progress via RSS|
|End with a thank-you message or redirect to any URL||Connect with Wufoo or other survey engines|
|Collect participant comments||Pass username and tag variables in and out of the study|
|Viewed several different result presentations online||Download Excel-compatible files|
|See how long participants spent on the study||Share the results link with colleagues|
|View category summaries||Merge categories|
Easy and enjoyable to use
Getting straight to the point: it was easy to set up and operate (provided Flash is installed). Of course, I cannot report on performance of features I did not use, however I was aware of each of them through the interface. Except how to track progress – I have since noticed the “track progress through RSS” link; but did not at first.
I dragged and dropped and labelled intuitively. I particularly enjoyed being able to move groups around the canvas as I would cards on a table – this helped me track the way I was thinking about the groups.
Visual information not captured
Unfortunately, the mental connections I was making by inter-group placement would not appear in the quantitative results. I wonder whether WebSort might find a way to capture the screen layout when participants click “I’m done”?
Also, when I did the study I did not see images I had added to some cards. I was able to watch one participant do my study and she could see the images. However all cards were sized as if they had images (that is larger) so fewer displayed in the panel. This did not appear to bother her, and she appreciated the images.
“Leave a comment”
Few of my participants left comments. The link “Leave a comment” appears to be an effort to capture some of the thinking that many reviewers mention being missed in online sorts compared to face-to-face. Opportunities for researcher-participant interaction such as clarifying cards or body-language indicators of points of struggle that could be investigated.
I believe when I tried a colleague’s study in Optimal Sort the invitation to comment was more obvious. The fact that I did make a comment to her might have been more because of my interest in the methodology. Could this “other” thinking be better captured? Does it need to be?
Success, and the challenge…
is in the preparation…
My imaginary trial run highlighted to me that careful preparation is required. Particularly: Careful selection of card contents, and participants.
If you’re going to give it a trial run, use these lessons learned:
- Use images for all items or not at all. – maybe?
- Consider who you plan to ask to be your (up to 10) trial participants, and choose relevant content for them. I tested on an obscure hobby of mine, and the struggle to understand the items interfered with thinking about how they would go together.
If you’re ready to plan an online sort:
- Read both the support/FAQ page and the WebSort category of UXPunk’s blog. Some potentially vital advice is only in the blog. For example, blogged advice on careful selection of card contents could prevent some causes of wanting to change a study part way through.
- As the FAQ admits, a card sort is only one of several possible techniques for gaining understanding about your site’s users. The blog links to creative use cases which identify how card sorting works in with other research methods.
Without such preparation it is impossible to give a fair evaluation of WebSort’s usefulness.
– and in interpretation
What you want to be able to interpret really should have fed into how you prepare, and whether a card sort, or an online one, is really your solution. The relative value of quantitative and qualitative information is not unique to card sorting, nor the online versions.
Requires Flash: ?accessibility
As Flash is already installed in my browser I did not, at first, give a thought to the code platform. When temporarily disabled I discovered that it is not operable via my Xoom – it would not let me enter an email address.
While a mobile phone does not offer ‘table’ space for a sort, tablets may. Perhaps androids do not have sufficient distribution for it to be worth trying to make WebSort android-friendly. I suspect however, that the first online card-sorting tool to adjust for use on i-pads will be very popular, given a growing distribution of i-pads among mobile workers. Karen Jorgenson mentioned such a situation in the OptimalSort support forum. Optimal Sort is an alternative online card sorting tool – and the two teams are friends, how sweet :-).
Not for the visually impaired
Although content apparently can sometimes be developed for accessibility in Flash, I do not think that drag and drop functionality would be included. I pondered how do paper versions of card sorting cater to blind participants? Suppressing, for the moment, my curiosity over the feasibility of printing cards in braille; I wonder if card sorting is sufficiently vital a research technique to require such effort. If so, and if anyone has or does any additional research on card sorting with the visually impaired, the team at WebSort would like to hear from you.
Not something I can evaluate really. For “all in” costs, assuming all other considerations (ie whether or not you need face-to-face information; the number of participants’ views you want) are equal:
- For a WebSort: add to one of the three price options any incentives that might be needed.
- For an in-person sort: add to the very cheap cost of paper/cards, time to physically prepare the cards, arranging participant schedules, transporting, feeding, compensating them.
Quick response to support requests
My queries: about typical usage; variations; about Flash accessibility; and permission to share interface screenshots received prompt replies.
Great free tool for small teams
I can see this being triply handy for teams of 10 or less.
- it is free for up to 10;
- they could sort separately to ascertain different perspectives first, and
- screenshot their layouts to capture the additional inter-group relationship information to then discuss their models.
Beyond that I do not know
Sure, it is easy to use (on a PC with Flash, as long as you use a mouse and can see clearly). However I would need a real situation in which to test WebSort‘s relative (against other methods) value for designing user-friendly information architecture. I’d welcome hearing from anyone who has gained insights from well-prepared studies in real contexts with WebSort (or other online card sorting) that they had not already been seeing before the study.