Glossary of Terms
used in Usability Engineering
- Analytical method
is a method where formal specifications of the system, tasks and context of use serve as an input. The results of analytical evaluation can be seen as the output of a mathematical function which only depends on the formal input specifications. In general analytical methods are objective and access no empirical data. They can be applied very early in the design cycle. The reliability of measures calculated on the basis of these methods is not in question. Analytical methods are often based on simulation: the interaction of a (future) user with the system is simulated.
- Assessment criteria
are critical values for relevant measures which are the basis for the assessment of a service or product.
- BASELINE Usability Data
This term is synonymously used for Reference Values. BASELINE usability data is the result of pooling evaluation and validation information from comparable and representative systems. Only scaled measures with norms and reference values offer this possibility, and should be pursued whenever feasible.
- Benchmark task/test
A standard laboratory task (or a test which consists of a sequence of tasks) used by many companies in the evaluation of products.
- Context of use
is descibed by potential users, goals and tasks these users intend to perform with the service or product, and the technical, physical and social environment in which the application is used.
- Critical success factors
determine the success of an application for the organisation. Critical success factors can be product oriented (e.g. higher product quality, innovative design), development process oriented (e.g. more efficient and effective development process), standards oriented (e.g. product complies to standards), societal goals (e.g. product can be used by people with special needs).
- Electronic information applications
are either products or services that store, process, and deliver information required by users through electronic means.
- Measures and Metrics
Measures are operationalised quality factors. A measure can be subjective or objective, direct or indirect, analytical or empirical.
Metrics are measures posessing metric properties which express the degree or strength of a quality factor. Metrics are obtained by an objective measurement method. Their scale of measurement is known, they posess scale metric properties, known maxima, minima and reference values, their reliability and validity are known.
Metrics are interpreted according to the context in which they were measured. Metrics allow comparisons between applications, as well as between alternative versions of a single developing application, and comparisons with reference values. Conformance to standards and minimal requirements can be tested with metrics. Some metrics tell about the performance of the user applying an application to his work (e.g. efficiency of use, learning effort, errors). Other metrics predict quality of use factors on the basis of a user interface specification, prototype, or fully functional product, and an underlying user model.
- Measurement
means a repeatable, objective procedure for generating a measure.
- Objective measurement method
A method is classified to be objective if it is based on objective measurement procedures. However, the data can be of subjective or objective origin. A method based on empirical data (e.g. experiments, ratings of users) can be objective if there are objective measurement procedures. The results of the application of a subjective measurement method are influenced by decisions made by the evaluator in a high degree.
- Quality factors
are features by which a product can be assessed such as efficiency of use, task adequateness, cognitive workload, robustness, learning cost, user acceptance. Quality factors are the result of the decomposition of the term `quality of the application'. They are variables which reflect different independent quality aspects of the application. Validation questions must be formulated in terms of quality factors in order to allow meaningful measurement.
- Quality of use and usability
is used synonymously to usability. Usability as well as other traditional terms such as user friendliness, usefulness, ease of use, have in common that they are vague and fuzzy terms. They give the impression of just one single dimension. However, users have different needs and requirements and perform different tasks with an application, therefore an application which is usable by one user may be tedious to use by another user. In addition usability is a too narrow concept which does not take into account cost/benefit issues. Hence, the term 'quality of use' is prefered to usability.
Quality of use is a concept which consists of multiple, measurable dimensions (e.g. learning effort, efficiency, user satisfaction, enjoyability). An application's overall quality of use is then determined by specifying the dimensions which are relevant in a certain context of use, by adding priorities to the different quality dimensions (this is optional), and by defining assessment criteria for the dimensions. Quality of use describes the user-centered view of product quality. (There are other quality aspects, e.g. technical quality such as portability, maintanability, etc.)
- Reliability of a measure
describes the degree of stability, accuracy and error associated with a measurement procedure. It describes the extent to which a measurement procedure yields the same result if carried out on different occasions, possibly by different people on the same object. A measure is reliable if the application of the measure yields reproducable results. Factors which could reduce the reliability of a measure could be for example number or attributes of subjects involved in experimental tests or subjective decisions made by the evaluator.
- Task
is described in terms of the goals or a desired end-result of activities a user wants to achieve. More than one user procedure (a sequence of commands to be executed to carry out a task or to reach a goal) may exist to solve the task.
- Usability engineering
is a well defined process which is performed as part of the application development process. It can be part of the development process of any type of electronic information application. Although each development project is different, the approaches, methods, techniques and activities applied to achieve usability do not vary much.
- User-centred design
emphazises on early and continuous involvement of users in the design process. Participatory design and prototyping means making users part of the design team or letting them participating in the prototyping process as "subject matter experts". It is not reasonable to expect users to come up with design ideas from scratch. However, users are very good at reacting to designs and prototypes. Participatory design is particularly effective when designing customized software in-house.
- User validation
describes the gathering of quality of use information about an application which is used within a specified context of use. The user validation process describes a set of ordered activities that contribute to a defined objective of a validation project. A user validation process takes place over time and has precise objectives regarding the results to be achieved. A User Validation Process Model describes the structure and the elements of that validation process in terms of stages and steps, dependencies and data.
- Validation method
is a repeatable, systematic procedure to produce validation results. The specific aspect of validation methods as opposed to a general view of methods is that user validation always starts with an objective and assessment criteria, i.e. questions such as 'Is design alternative A more efficient to use than B ?', 'Does the electronic application fulfill the minimum user requirements?'. Quality factors such as enjoyability, cognitive workload and efficiency of task performance may be implied by these validation questions, and the resulting measures must be shown to be valid measures for these quality factors.
- Validity of a measure
A measure is valid for a given quality factor if there exists a corelation of the measurement values and that quality factor. Validity describes the extent to which a measurement procedure measures what it is intended to measure. It is a psychometric concept, and a number of different ways of testing it have been used. At present the following methods of validating a measure are frequently used:
- Content validity
Measure of the sampling adequacy of the rules of measurement;
- Criterion validity
The comparison of the measures obtained against external variables or criteria;
- Construct validity
This refers the rules of measurement to an underlying theory or model;
- Ecological validity
The extent to which the conditions simulated in the laboratory reflect real life conditions.
- Face validity
Does the measure appear to be measuring something sensible?
Return to BASELINE Homepage
Go to top of this page