|
Frequently Asked Questions about User Validation:
|
Do we need users for user validation ?(a1)
Who are real users ? (a2)
How do you select appropriate user groups for validation ? (a3)
What is the role of end users in usability studies ? (a4)
How do you validate a product which is aimed at generic users ? (a5)
What is a user champion group ? (a6)
Can one use the same group of real users for user testing of different services ? (a7)
Do we need users for user validation ?
Theoretically, it may be possible to validate an application with reference to guidelines, standards, and lists of desirable criteria that the application must exhibit, because standards should be based on user needs and requirements. For instance, practitioners may believe that following the quality characteristics of ISO 9241 standards, parts 12 onwards, gives a reliable assessment of usability.
BASELINE strongly rejects this claim, and in this it is in line with the Framework IV so-called 'ten commandments of telematics' of the Telematics Applications Programme. Standards can only be used to test if minimal requirements are fulfilled. Real users may have additional requirements, and users requirements evolve and change faster than standards. Therefore user validation must take place with the active participation of actual users at all stages of the development of an application. The user-centered aspects of the 'ten commandments' can be simply summarised in the following 'golden rule': Plan to validate with end users early in the development process and frequently thereafter.
Real users are the potential users of the application you are developping. They are not their managers. They are not computer experts who may decide which application to select for the real users. They are not persons from the purchase department who decide which application to buy with respect to financial constraints. Real users are those persons who perform their work tasks with the support of the application.
How do you select appropriate user groups for validation ?
Analyse user needs and requirements. Identify groups of users with homogeneous characteristics within a group and heterogeneous characteristics between groups. Validation work with a homogenous user group gives more tangible results than validation work with a similar size group which is composed of people from diverse backgrounds and of diverse skills and needs.
A common problem which may arise is that several different user groups with differing characteristics emerge from the analysis, and user validation may not be performed with all these groups. The key to the solution is to prioritise the groups in terms of their relevance to the project main objectives.
What is the role of end users in usability studies ?
Having real end users has become a sine qua non, certainly in the Telematics Applications Programme. The appropriate function for users in the development process is to act as a 'reality check' that the development is proceeding in the correct direction. However, it is wise not to accept everything the users say as dogma!
How do you validate a product which is aimed at generic users ?
If you are really focussing on generic users, start by asking why any group of users would wish to use your product, and what these potential users would have in common that other potential users do not. Groups of users may be defined which vary by:
A discussion about the intended context of use of a product will usually reveal a mass of assumptions about many characteristics of the intended users. Once these are understood it becomes much easier to specify user groups and to select users who are required to validate the product.
What is a user champion group ?
A user champion group consists of users who are directly involved in a development project. The user champion group members play the role of a real user. They actively contribute to the project their needs and requirements, test prototypes, system components, and intermediate versions, and feedback their opinions and finding to the designers and developers.
Can one use the same group of real users for user testing of different services ?
Yes. However, you have to control effects of learning transfer which may occur when a single user is testing different services. To get valid evaluation results you have to define a proper experimental design. A good overview of how to do this is provided by Campbell, D.T., & Stanley, J.C. (1963). Experimental and quasi-experimental designs for research on teaching (pp. 445-632). In N.L. Gage (ed.), Handbook of Research on Teaching. Rand McNally. Chicago 1963.