Basic Principles of Multimodal Systems
Multimodal System research is focused on two main concepts: System Inputs and System Outputs.
System Inputs
The input to a system is any way in which the user can provide a command or set of instructions for the system to act on. Traditional input devices have keyboards, normal buttons, and pointing devices like mice. People with impairments may need alternatives to these. Glasgow has put a lot of thought into the best ways to create alternative input interfaces for a system. A popular choice of interface may prove to be speech technology, which the University of Edinburgh is developing for MATCH but may not suit every user. Glasgow has been concentrating on an unusual and relatively new alternative, known as Haptic Technologies.
Haptic input devices use movement but in different ways from pressing buttons. As an example, motion-sensitive controls can be operated by tilting a device in a certain direction and obtaining input from this. An example is the Remote Control device shown here.
System Outputs
The output of a system gives the user a response to an action or command. A Multimodal System relies on using a combination of different output methods depending on the user's circumstances. The University of Glasgow is focusing on a number of alternative techniques.
A convenient idea is the use of text and graphics as a tool to provide output. An example would be the use of text on a display device such as a TV or computer monitor, prompting the user with reminders. A similar idea is the use of graphics such as icons for the same purpose.
Earcons are good alternative for people with vision impairment. An Earcon uses sounds rather than graphics to convey a message. Unique sounds can be developed to correspond to a particular task or meaning. The user would be instructed to select these or set the earcons up themselves. These sounds would then be played when appropriate.
Haptics can also be used for creating outputs as well as inputs. Devices can be worn by the user which create vibrations relating to a reminder or signal. This has proved popular in computer games, where the handset vibrates to reflect what happens on the screen. This may also prove useful in a home care scenario. Another example is a mouse that uses a number of moving metal pins on the top of the device casing. When the user moves the cursor over an object (e.g. a window edge or an icon), the device moves the pins to portray the outline of the object so that the user can feel what the object is.
