respect logoWizard of Oz Prototyping

Primary Reference Sources

Maudsley, Greenberg & Mander (1993) Prototyping an intelligent agent through Wizard of Oz. Interchi í93 Conference Proceedings, pp 277-284.

Summary description

This approach involves a user interacting with a computer system which is actually operated by a hidden developer - referred to as the "wizardí" The wizard processes input from a user and simulates system output. During this process the user is led to believe that they are interacting directly with the system. This form of prototyping is beneficial early on in the design cycle and provides a means of studying a userís expectations and requirements. The approach is particularly suited to exploring design possibilities in systems which are demanding to implement.

Typical Application Areas

The approach is highly applicable to "intelligent interfaces" which feature agents, advisors and/or natural language processing.

Benefits

This approach allows usability requirements and issues to be explored at an early stage in the design process, particularly for systems which go beyond readily available technology.

The member of the design team who plays the wizard can gain valuable insights from the close involvement in the userís activity.

Limitations

The person playing the role of the wizard must appreciate the functionality of the proposed system in order to provide a convincing representation.

This approach requires a higher commitment of resources than other approaches to prototyping such as those that rely on simple paper-based materials.

It may be difficult to carry this out effectively in situations where there is a large graphic element in the interface.

Cost of use

Two computer systems would be required, one each for the user and the wizard. Two staff are required to conduct the evaluation - one to play the wizard, another to instruct the user and record the session. The wizard should be an experienced member of the design team so that system responses are logical and not beyond the realms of possibility. The time overhead largely depends upon the task domain and the number of users exposed to the prototype.

Costs of Acquisition

Compulsory training is not specified. However, many reports on the approach stem from research papers where the details of the technique are sketchy. The approach is however, likely to require experience and / or training for the practitioner to reach competency.

Suitability for requirements engineering in Telematics:

Some existing expertise is available from RESPECT partners. Industry could adopt the principles, it is already applied within Apple. While the approach may lack the general applicability of other prototyping approaches it is particularly suited to multimedia and telematics applications.

How to get it

See primary reference source for more information.

Detailed description of approach

The general procedure for implementing this approach is outlined in the following.
  1. Firstly, allow enough time to fabricate the wizard of Oz prototype, design some tasks, recruit users, conduct the evaluation of the prototype and report the results.
  2. Allocate the role of wizard and the role of facilitator to the relevant staff.
  3. Assemble the necessary equipment and inter-connecting software.
  4. Select appropriate users to test the prototype, try to cover the range of users within the target population.
  5. Prepare realistic task scenarios for the evaluation.
  6. Pilot the evaluation procedure and ensure the wizard is well practised in playing the role of the computer.
  7. Ensure recording facilities are available and functioning.
  8. Conduct each session. The facilitator instructs the user to work through the allocated tasks interacting and responding to the system as appropriate.
  9. Conduct post-session interviews with the users, drawing upon pre-set questions and issues raised during the use of the prototype.
  10. Debrief the users and thank them for their co-operation.
  11. Analyse information obtained, summarise observations and user evaluations. Consider the themes and severity of the problems identified.
  12. Summarise design implications and recommendations for improvements and feed back to design team. Video recordings can support this.
  13. Where necessary refine the prototype and repeat the above process
The evaluation of wizard of Oz prototypes yields information regarding user expectations and requirements for novel technologies. The data gained from this approach may be used to refine the initial prototype, and inform subsequent evaluations.


[Go to Methods Table (by Lifecycle)] [Go to Methods Table (by Special Needs)] [top of page]