Modes can be dangerous in a user interface, especially if the UI does not make the modes and their states clearly visible. This is often heard advice about UI design. I collected snippets of this advice/heuristics here.

If you have a different point of view or know scientific articles or textbooks that further discuss this, I would love to hear about it!


See also: Visual Feedback and How Modes Kill (thanks Sasha Maximova)

http://www.usabilityfirst.com/glossary/modal/:

modal

a type of interface where the user moves between different states, or modes. In each mode, input from the user is interpreted in a different way. When a user is in a given mode, it’s a good idea to provide an extremely salient indicator of which mode is currently active, e.g. by providing a large icon or shifting the color of the screen.

Modes are usually good to avoid because users are restricted in their available actions and they may be confused that the meaning of their actions changes as they change contexts. However, modes can sometimes be helpful to control and guide input (providing security and preventing accidental errors), and modes may be appropriate when they are short and actively maintained by a user (e.g. the fact that keys refer to capital letters only when the Shift key is held down) and when they follow well-understood conventions, such as painting tools.

Examples of modes include:

  • Choosing a painting tool in a paint program. When the Line tool is selected, for instance, all mouse clicks create lines. When a brush tool is selected, the user is in a different mode because all mouse clicks are then interpreted as painting.
  • Old text processors, such as ‘vi’ in UNIX, often used a command mode and an edit mode. In a command mode, keystrokes issued commands. In an edit mode, keystrokes were entered into a text document.
  • Moving between pages on a website. Each page is a mode which limits the kinds of actions a user can take (where they can go).
  • Modal dialog boxes are ones that force a user to respond to them before the user can go back to using other aspects of the application.

http://catb.org/~esr/writings/taouu/html/ch01s03.html:

Rule of Modelessness: The interface’s response to user actions should be consistent and never depend on hidden state.

This rule could be taken as a consequence of the Rule of Transparency, but violations are so common and such an important cause of interface problems that we have chosen to foreground it by making it a separate rule.

An interface with “modes” responds to the same input gesture in different ways depending either on hidden state not visible to the user, or on where you are within the interface. Since the early days of the Macintosh, which eliminated mode dependencies almost completely, it has been widely understood that modes are a bad idea.

The classic bad example in Unix-land is vi(1), a commonly-used editor in which typed characters are either inserted before the cursor or interpreted as editor commands, depending on whether or not you are in insert mode. This is a well-known cause of confusion to novices, and even experts at this editor occasionally trash portions of their documents through having lost track of what mode they are in or inadvertently switched modes. Modern versions of vi(1), partly mitigate this design flaw by providing a visible mode indicator. We know this is not a complete solution, because even expert users can overlook the state of the mode indicator and find themselves in trouble.

We’ll see in the Wetware chapter that modes interfere with the process of habituation by which users become expert with interfaces; indeed, they turn habituation into a trap. A computing system ideally designed for human use would have one single set of gestures and commands that are unformly applicable and have consistent meanings across its entire scope.

http://catb.org/~esr/writings/taouu/html/ch04s04.html

Raskin makes the empirical point that human beings cannot be prevented from forming habits about routine tasks, and elegantly defines good interfaces as those which encourage benevolent habituation. We can go a bit further and observe that “benevolent habituation” – the ability to do the right thing by unconscious or half-conscious reflex – is precisely what we normally mean by expertise.

Thus, our Rule of Modelessness. Users will form gestural habits, and it is a bad thing when gestural habits they form in one part of the system become inappropriate or even destructive in another part. When this leads to problems, the fault lies not with the user’s habits but with system designers’ failure to maintain consistency.

Also some of Nielsen’s heuristics seem relevant:

http://www.useit.com/papers/heuristic/heuristic_list.html

Visibility of system status

The system should always keep users informed about what is going on, through appropriate feedback within reasonable time.

Error prevention

Even better than good error messages is a careful design which prevents a problem from occurring in the first place. Either eliminate error-prone conditions or check for them and present users with a confirmation option before they commit to the action.

Consistency and standards

Users should not have to wonder whether different words, situations, or actions mean the same thing. Follow platform conventions.

A Dictionary of Computing, http://www.oxfordreference.com/views/ENTRY.html?subview=Main&entry=t11.e6022:

dialog management The methods and procedures governing the way that information is exchanged between the user and the computer system. It involves the consistent formatting of prompts, check boxes, lists of options, sliders, messages, and other constructs. There are modal dialogs, where the user can do nothing else until the action required to satisfy the dialog has been taken; this can be as simple as acknowledging a message. Conversely, a modeless dialog is one that may be returned to after attending to other matters. See also dialog box. (Ed John Daintith and Edmund Wright. Oxford University Press, 2008. Oxford Reference Online. Oxford University Press. Tampere University. Looked up on 14 November 2011)