WebSort : easy online card sorting [a review]

WebSort by UXPunk is an online tool designed for conducting and analysing remote card sorts. Its target market is website architects (or webmasters). Cardsorting aims to help solve the problem of organising a site’s content to meet audience needs and expectations.

Now that ambition is an obvious target for this novice bereted librarian, so I went to find out more.



The basic idea is simple, as Donna Spencer put it:

Get people to put stuff into groups that make sense for them.[1]

Online card sorting is useful, as Neil Turner concludes, when you need lots of participant sorts or when you can’t easily set-up, or need to supplement, face-to-face card sorting sessions[2]. Online card sorting tools offer the added advantage of automated statistical analyses and rapid production of statistical reports. Not, as Courtney Clark points out, that such quantitative data could suffice to gain insight into audience mental models [3].

How Websort Works

As outlined at the very inviting website:

Websort's excellent landing page

Every uncluttered item of text and image on landing page contributes to rapid impression of purpose, next steps and desirable information.

  1. You enter the items you want sorted, refine the study parameters and invite participants;
  2. Participants drag and drop items into groups that make sense to them – and in open sorts define their own group labels;
  3. WebSort compiles the results into a variety of statistical reports.

WebSort‘s features & options

Features I used Features I did not use
Labels, Descriptions (‘tool-tips’), and Images Paste in items from text file or spreadsheet
Customize the instructions Display your own logo
Let participants define the categories (an ‘open’ sort) Predefine the categories (a ‘closed’ sort) (can allow partipants to add)
Randomise item presentation Sort in English, French, German, Portuguese, Spanish, Japanese, and Swedish
Set limit to number of participants and closing date Use tags to group different kinds of participants or Bypass participant login
Drag and drop interface Track the progress via RSS
End with a thank-you message or redirect to any URL Connect with Wufoo or other survey engines
Collect participant comments Pass username and tag variables in and out of the study
Viewed several different result presentations online Download Excel-compatible files
See how long participants spent on the study Share the results link with colleagues
View category summaries Merge categories

Easy and enjoyable to use

Getting straight to the point: it was easy to set up and operate (provided Flash is installed).  Of course, I cannot report on performance of features I did not use, however I was aware of each of them through the interface. Except how to track progress – I have since noticed the “track progress through RSS” link; but did not at first.

I dragged and dropped and labelled intuitively. I particularly enjoyed being able to move groups around the canvas as I would cards on a table – this helped me track the way I was thinking about the groups.

I enjoyed placing the stacks where I wanted, placing some close to each other or overlapping demonstrating their mutual relationship. (Screenshot of interface used with permission).

However:

Visual information not captured

Unfortunately, the mental connections I was making by inter-group placement would not appear in the quantitative results.  I wonder whether WebSort might find a way to capture the screen layout when participants click “I’m done”?

Also, when I did the study I did not see images I had added to some cards. I was able to watch one participant do my study and she could see the images. However all cards were sized as if they had images (that is larger) so fewer displayed in the panel.  This did not appear to bother her, and she appreciated the images.

“Leave a comment”

Few of my participants left comments.  The link “Leave a comment” appears to be an effort to capture some of the thinking that many reviewers mention being missed in online sorts compared to face-to-face. Opportunities for researcher-participant interaction such as clarifying cards or body-language indicators of points of struggle that could be investigated.

I believe when I tried a colleague’s study in Optimal Sort the invitation to comment was more obvious. The fact that I did make a comment to her might have been more because of my interest in the  methodology. Could this “other” thinking be better captured? Does it need to be?

Success, and the challenge…

is in the preparation…

My imaginary trial run highlighted to me that careful preparation is required. Particularly: Careful selection of card contents, and participants.

If you’re going to give it a trial run, use these lessons learned:

  • Use images for all items or not at all. – maybe?
  • Consider who you plan to ask to be your (up to 10) trial participants, and choose relevant content for them. I tested on an obscure hobby of mine, and the struggle to understand the items interfered with thinking about how they would go together.

If you’re ready to plan an online sort:

  • Read both the support/FAQ page and the WebSort category of UXPunk’s blog. Some potentially vital advice is only in the blog. For example, blogged advice on careful selection of card contents could prevent some causes of wanting to change a study part way through.
  • As the FAQ admits, a card sort is only one of several possible techniques for gaining understanding about your site’s users. The blog links to creative use cases which identify how card sorting works in with other research methods.

Without such preparation it is impossible to give a fair evaluation of WebSort’s usefulness.

– and in interpretation

What you want to be able to interpret really should have fed into how you prepare, and whether a card sort, or an online one, is really your solution.  The relative value of quantitative and qualitative information is not unique to card sorting, nor the online versions.

Requires Flash: ?accessibility

Tablet un-friendly

As Flash is already installed in my browser I did not, at first, give a thought to the code platform. When temporarily disabled I discovered that it is not operable via my Xoom – it would not let me enter an email address.

The interface, cards at left, empty canvas at right, can be seen enticingly underneath the email entry box.

It was not possible to enter an address into that box. (Interface screenshot shared with permission)

While a mobile phone does not offer ‘table’ space for a sort, tablets may. Perhaps androids do not have sufficient distribution for it to be worth trying to make WebSort android-friendly. I suspect however, that the first online card-sorting tool to adjust for use on i-pads will be very popular, given a growing distribution of i-pads among mobile workers. Karen Jorgenson mentioned such a situation in the OptimalSort support forum. Optimal Sort is an alternative online card sorting tool – and the two teams are friends, how sweet :-).

Not for the visually impaired

Although content apparently can sometimes be developed for accessibility in Flash, I do not think that drag and drop functionality would be included. I pondered how do paper versions of card sorting cater to blind participants? Suppressing, for the moment, my curiosity over the feasibility of printing cards in braille; I wonder if card sorting is sufficiently vital a research technique to require such effort.  If so, and if anyone has or does any additional research on card sorting with the visually impaired, the team at WebSort would like to hear from you.

Price

Not something I can evaluate really.  For “all in” costs, assuming all other considerations (ie whether or not you need face-to-face information; the number of participants’ views you want) are equal:

  • For a WebSort: add to one of the three price options any incentives that might be needed.
  • For an in-person sort: add to the very cheap cost of paper/cards, time to physically prepare the cards, arranging participant schedules, transporting, feeding, compensating them.

Quick response to support requests

My queries: about typical usage; variations; about Flash accessibility; and permission to share interface screenshots received prompt replies.

Partial conclusions

Great free tool for small teams

I can see this being triply handy for teams of 10 or less.

  1. it is free for up to 10;
  2. they could sort separately to ascertain different perspectives first, and
  3. screenshot their layouts to capture the additional inter-group relationship information to then discuss their models.

Beyond that I do not know

Sure, it is easy to use (on a PC with Flash, as long as you use a mouse and can see clearly). However I would need a real situation in which to test WebSort‘s relative (against other methods) value for designing user-friendly information architecture.  I’d welcome hearing from anyone who has gained insights from well-prepared studies in real contexts with WebSort (or other online card sorting) that they had not already been seeing before the study.

References

Advertisements

Tags: , , , , , , ,

About Mica Meerbach

Labelled variously: Mica, Michaelina, Mum, dag. Librarian (AALIA) in Victoria, Australia.

3 responses to “WebSort : easy online card sorting [a review]”

  1. Stacy Wilson says :

    Use it with caution. After using WebSort for years, we’ve run into a fatal flaw. Because of their project pricing, when you hit 100 completions, they automatically shut it down. And, it’s next to impossible to reach a live person there (via Twitter, their website, email, or voicemail) to turn it back on in a reasonable time. Poor service when you run into a problem during a sort has convinced us to turn to a different tool. We’re opting for one that provides unlimited participants.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: