| Expert - or heuristic - reviews are an
opportunity for the design team to obtain the perspectives of designers who have been
outside the development process. There is probably no single more cost-effective way to
improve the quality of a product. Sample Heuristic Review Report (PDF  )
Heuristic Review
Guidelines
Why Expert Reviews are Valuable
An outside perspective can be extremely valuable. When a software product is released,
one of the most important questions is "will users find it intuitive"? Although
the design team has struggled with this question throughout the entire project, they
become so enmeshed in the process that they can no longer view it with naive eyes. They
have been involved in many debates and compromises. In addition, the design team has
confronted the realities of schedules, budgets, working with marketing and technical
colleagues who may have different priorities and dealing with management expectations.
If we could predict the future, we could really reduce risk. Since we can't, the best
we can do is to try to identify elements in the design which may be barriers to the users
and impair the product's acceptance. The goal is to make the groups of successful users as
large as possible and to have users feel that the software is a real asset - a tool that
enhances their productivity.
Two tools that we have to help in this process are heuristic design reviews and
usability tests. These two techniques complement each other. Usability tests show us how
new users relate to the product on initial exposure. Unfortunately, usability tests are
also expensive and time-consuming. And their yield can be relatively low compared to the
effort expended. Usability testing is most useful at the beginning of a project, when we
want to see if users "get" the basic design concept, and at the end, to identify
any low-effort, high-yield refinements which are still possible.
A heuristic review is a consultative process. While there are a number of different
techniques which can be used, the core process is to ask experienced designers, to work
through the software, adopting as far as possible the viewpoint of a novice user. This is
perhaps analogous to a teacher asking another educator to look at some instructional
material to assess how well students will work with it.
Cognetics' approach is to conduct a heuristic review with 2-3 designers. This is a
large enough group to provide a diversity of opinion yet small enough to be efficient.
Before coming together, each designer looks at the software alone. The first look at a
product is very valuable. It is important for the designer to make notes of initial
reactions, misconceptions and blunders before he or she becomes too familiar with the
software. Having walked through the software for initial reactions, the designer re-visits
the software. He or she now looks at each screen looking for violations of the Cognetics
heuristic guidelines. They note issues such as:
- What problems, if any, did I have here? Why was I confused and how might that confusion
have been reduced or avoided.
- What elements might confuse novice users, even though I was not confused?
- What concepts and skills must the user have to understand the material presented at this
point and the options available? Are there ways in which the presentation and options
might be simplified.
- Are there demands on the user's recall (both conceptual and short-term memory) which
could be reduced.
At the conclusion of this process, the designer should have developed a comprehensive
set of notes, have reasonably well-developed ideas about design issues and possible
solutions.
Now the design team comes together and performs a second walk-though of the software.
As they go through it together, they raise and discuss the issues they noted. Some issues
may be dismissed while new ones may emerge. Solutions are also suggested. At the
conclusion of this process (usually ended by exhaustion) the combined set of notes is
organized and a report created.
The results of the review are usually presented in a work session with the software
design team. It is important that heuristic reviews not be seen as attacks on the design
but as collegial input. It is not our role to make judgements about the design (like a
movie critic) but to provide input that can help a software team achieve its goals.
The Expert Review Process
Expert reviews are often done as the beginning of a redesign project, as a way of
correcting design or usability flaws before a formal usability test, or as part of a
design engagement to solve a problem which cannot be resolved from within the current
team.
The details of the schedule will vary, but typically they are structured in three
steps:
- Cognetics reviews the software
The Cognetics designers receive, install and review the software on their own, to
approximate the new users’ experience, and record their comments and observations.
For complex software or task domains, an expert will need to be available to the review
team to answer questions about the software and its typical uses.
- Cognetics assembles the notes from the review observations
Review observations are recorded and compiled with a brief summary describing the
areas where the interface needs the most improvement.
- Cognetics works with the product team to solve usability problems found in the
review
The Cognetics designers then meet with the product team to discuss the observations and
work on design solutions to solve the problem. This session lasts from one to two days. A
final day is spent, when necessary, working with the development or product lead to
assemble the notes from the working session.
The goal of this review structure is to shift the emphasis from discovering problems to
solving them, with the active involvement of the product development team. The review
becomes part of the design process, rather than a "test" of the team's
capabilities. |
