Wizard
of Oz Prototyping
Primary Reference Sources
Maudsley, Greenberg & Mander (1993) Prototyping an intelligent agent
through Wizard of Oz. Interchi ’93 Conference Proceedings, pp 277-284.
Summary description
This approach involves a user interacting with a computer system which
is actually operated by a hidden developer - referred to as the "wizard’"
The wizard processes input from a user and simulates system output. During
this process the user is led to believe that they are interacting directly
with the system. This form of prototyping is beneficial early on in the
design cycle and provides a means of studying a user’s expectations and
requirements. The approach is particularly suited to exploring design possibilities
in systems which are demanding to implement.
Typical Application Areas
The approach is highly applicable to "intelligent interfaces" which feature
agents, advisors and/or natural language processing.
Benefits
This approach allows usability requirements and issues to be explored at
an early stage in the design process, particularly for systems which go
beyond readily available technology.
The member of the design team who plays the wizard can gain valuable
insights from the close involvement in the user’s activity.
Limitations
The person playing the role of the wizard must appreciate the functionality
of the proposed system in order to provide a convincing representation.
This approach requires a higher commitment of resources than other approaches
to prototyping such as those that rely on simple paper-based materials.
It may be difficult to carry this out effectively in situations where
there is a large graphic element in the interface.
Cost of use
Two computer systems would be required, one each for the user and the wizard.
Two staff are required to conduct the evaluation - one to play the wizard,
another to instruct the user and record the session. The wizard should
be an experienced member of the design team so that system responses are
logical and not beyond the realms of possibility. The time overhead largely
depends upon the task domain and the number of users exposed to the prototype.
Costs of Acquisition
Compulsory training is not specified. However, many reports on the approach
stem from research papers where the details of the technique are sketchy.
The approach is however, likely to require experience and / or training
for the practitioner to reach competency.
Suitability for requirements engineering in Telematics:
Some existing expertise is available from RESPECT
partners. Industry could adopt the principles, it is already applied
within Apple. While the approach may lack the general applicability of
other prototyping approaches it is particularly suited to multimedia and
telematics applications.
How to get it
See primary reference source for more information.
Detailed description of approach
The general procedure for implementing this approach is outlined in the
following.
-
Firstly, allow enough time to fabricate the wizard of Oz prototype, design
some tasks, recruit users, conduct the evaluation of the prototype and
report the results.
-
Allocate the role of wizard and the role of facilitator to the relevant
staff.
-
Assemble the necessary equipment and inter-connecting software.
-
Select appropriate users to test the prototype, try to cover the range
of users within the target population.
-
Prepare realistic task scenarios for the evaluation.
-
Pilot the evaluation procedure and ensure the wizard is well practised
in playing the role of the computer.
-
Ensure recording facilities are available and functioning.
-
Conduct each session. The facilitator instructs the user to work through
the allocated tasks interacting and responding to the system as appropriate.
-
Conduct post-session interviews with the users, drawing upon pre-set questions
and issues raised during the use of the prototype.
-
Debrief the users and thank them for their co-operation.
-
Analyse information obtained, summarise observations and user evaluations.
Consider the themes and severity of the problems identified.
-
Summarise design implications and recommendations for improvements and
feed back to design team. Video recordings can support this.
-
Where necessary refine the prototype and repeat the above process
The evaluation of wizard of Oz prototypes yields information regarding
user expectations and requirements for novel technologies. The data gained
from this approach may be used to refine the initial prototype, and inform
subsequent evaluations.