Finding and fixing usability problems early

When the cook tastes the soup, that’s formative. When the guests taste the soup, that’s summative. Robert Stakes

Formative evaluation is a type of usability evaluation that is carried out early in the design process to detect and eliminate usability problems during design, when changes are much cheaper and easier to implement than in later stages of software product development.

Designers are the primary consumers of usability study findings – their involvement in the study planning and participation in the analysis are critical to the evaluation’s success. Misunderstanding between the designer (or, more broadly, design decision makers) and the user is a common cause of usability problems.

Usability evaluation session

Before the session starts, you select a range of test participants who represent your expected audience. You should recruit participants from the target user population using your personas as a guide.

Next, define precisely what one is going to check and measure as well as develop tasks for the users to perform. Consider focusing on tasks that:

  • are critical
  • are done frequently
  • have serious consequences if done incorrectly
  • product owner, designers, or developers have doubts about

During the session try to get a complete sense of the workflow, specifically where and when people click. Takes notes on the behaviours observed:

  • Do users find some tasks especially difficult?
  • Is there sufficient online help?
  • Are parts of the interface error-prone?
  • What gripes do users have?
  • What mistakes do users make?
  • Where are users likely to get stuck?
  • Will users need a wizard to guide them through certain complex tasks?
  • What changes would users like to see introduced?

Debrief users after the session and collect their feedback using a Questionnaire for User Interface Satisfaction. With a prototyping tool or paper you can also let users mock up ideas they think would solve the problem(s) identified.

Let’s do it!

Early usability evaluation provides valuable insights into the user’s mind and allows designers to learn what users intend when interacting with the product. It is not an alternative to interactive design or a source of new great ideas that can make a compelling product, but it makes it possible to assess the effectiveness of design ideas.

As Jakob Nielsen, (named “the world’s leading expert on Web usability” by the U.S. News & World Report), writes in one of his articles: “It’s a rough estimate, but I would say that the benefits from early usability data are at least ten times greater than those from late usability data. Late usability studies often add about 100% to the final design’s desired metrics, but early usability can add 1,000% or more.”