Using ISO 13 407 as a guide to personal knowledge and competence

Dr J Kirakowski
Human Factors Research Group
University College Cork, Ireland.
jzk@ucc.ie
20th May, 2002.

This report is in the public domain and may be copied freely provided the author’s work is acknowledged as part of the copy or citation.

Overview

This paper is intended as a contribution to the debate concerning the knowledge and skills required of a person wishing to be certified as a Usability Professional.

Ever since the influential book by Norman and Draper (1986) put the issue into centre stage, discussion has taken place to attempt to define what exactly is required in order to do ‘user-centred (system) design.’ In 1999, the International Organisation for Standardization (ISO) published the now well-known standard ISO 13 407, Human-Centred Design Processes for Interactive Systems. This standard was proposed in 2001 as the basic definition of the sphere of work of a Usability Professional and so it therefore requires some examination to assess it for suitability for this role, and some short introduction so that the many interested participants in the certification process may become familiar with the broad outline of it.

The only authoritative source is copyright by the International Organisation for Standardization ISO, and is available for purchase through their web site at www.iso.ch through an interface which is at least mildly daunting to the first time user but which has the merit that it delivers promptly.

This paper has grown out of innumerable presentations made by the author to various interested groups both within industry and the research community and thus takes this experience into advantage to answer some of the more frequently voiced questions and requests for clarification made at these presentations. It is therefore a commentary and not a summary or digest of the standard: as such my personal biases and experience have indubitably crept in. Comments and criticism are welcome, especially from those who know ISO 13 407 much more thoroughly than I.

The paper consists of five sections. First of all, the background to the ISO organisation and the 13 407 standard is summarised. Then the background knowledge as stated in Clause 5 of the standard is considered, then the skills and knowledge needed to operationalise Clause 6, which is to do with planning the user-centred process, and Clause 7, which is the main substance of the standard, describing the four main activities of user centred design. There is a short conclusion and an acknowledgements section.

Why ISO 13 407?

In order to put the discussion into context, it is important firstly to understand what an ISO standard is. From the official ISO web site (at www.iso.ch) we find the following definition:

    Standards are documented agreements containing technical specifications or other precise criteria to be used consistently as rules, guidelines, or definitions of characteristics, to ensure that materials, products, processes and services are fit for their purpose (my italics).

Between what parties are such documents agreed?

    The mission of ISO is to promote the development of standardization and related activities in the world with a view to facilitating the international exchange of goods and services, and to developing cooperation in the spheres of intellectual, scientific, technological and economic activity (my italics).

This suggests a broad range of entities indeed, essentially, any party wishing to participate in the ‘international exchange of goods and services’ not just in economic activity but also in intellectual and scientific spheres. I find the order of the italicised list of spheres of co-operation at the end of the quotation is significant, and elsewhere on these introductory pages, we read that ISO’s aim is to ‘...facilitate trade, exchange and technology transfer...’

Thus the use of an ISO standard to define what intellectual and scientific knowledge is required to engage in economic activity within a given sector is not only acceptable, but would appear to be a large part of the essential mission of ISO. The selection of an ISO standard to adjudicate on the issue of what knowledge and skills are required of a professional working within the economic sector addressed by the standard would appear to be beyond dispute and indeed a strategically astute move.

Turning now to the subject matter of 13 407, in its Statement of Scope (Clause 1) we read:

    This International Standard provides guidance on human-centred design activities throughout the life cycle of computer-based interactive systems. It is aimed at those managing the design process... The main users of this standard will be project managers... nevertheless, all parties involved in human-centred system development, including the end users of systems, should find the guidance in this International Standard relevant.

There are a number of points worth amplifying in this statement.

  1. The standard is intended to apply to a wide range of computer-based interactive systems. In the Introduction we read this is even more explicitly given as ‘... hardware and software design processes.’ To what extent this standard is actually applicable or will really remain applicable to this wide range will have to be tested and seen.
  2. We see that the standard is ‘aimed’ at those ‘managing the design process’ that is, primarily at project managers. In that Usability Professionals are not always to be found in positions of responsibility as project managers this may be seen as a significant limitation.
  3. Notwithstanding this rather exclusive concern with managers, and fully in the spirit of what ISO sees as its collective mission, the standard then explains that all parties involved should ‘find the guidance... relevant.’ Thus in contrast to managers, at whom the standard ‘is aimed’ other actors in the design process should find relevant guidance in the standard.

It would appear that a strong case can be made from the above claims that the ISO 13 407 standard can indeed be used to provide relevant guidance as to the totality of what a Usability Professional should know in the sphere of the economic activity of the application of human-centred design activities throughout the life cycle of computer-based interactive systems.

The weakness remains that the claims may not be well substantiated. However, given the stringent review practices adopted by ISO working groups, and the need to agree by international voting on every single detail of a standard, in the absence of contrary evidence, ISO 13 407 must be given primacy of place as the best and most widely accepted definition we have available.

The standard itself contains three important sections: Clause 5 outlines the principles of Human Centred Design (HCD); clause 6 how to plan for HCD, and clause 7 describes the ‘four essential activities’ of HCD.

The rest of this paper will take each section in turn and outline the knowledge and skills necessary in the staff that a manager may have at their disposal in order to enable the manager to begin to substantiate a claim that the process for which they are responsible conforms with the 13 407 standard. Two important distinctions must be made:

  1. The manager need only have enough knowledge of the process in order to be able to assess that the process for which he is responsible is conformant; managers, by definition, do not need detailed technical knowledge but they rely crucially on advice and the quality of their assembled teams;
  2. To assert that any one person will have all this knowledge is not necessary; possession of the knowledge may be distributed among the team(s) and specialisms are bound to occur.

That is, the extent to which a Usability Professional should be able both to manage and carry out all the technical details of the entire process is a judgement call for those seeking to define the scope and competence of a Usability Professional: it is not given or implied anywhere in 13 407.

General principles

The standard makes it clear that the activities it describes are not to be interpreted as a design process, but that it is complementary to many existing design methods. How the activities are meant to be integrated is not so clear. There are usually two ways in which 13 407 is applied:

  1. As a ‘second process’ in parallel with the main (technical) design process, with well-defined points of contact with the main design process;
  2. As a ‘process within’ each stage of a larger (technical) process which latter process is characterised by a ‘waterfall’ type of methodology.

Some attempts have been made to make the activities suggested by 13 407 into a basic underlying process of system design (in the above language, a ‘first process’ which informs a technical process): these attempts however are all in the context of research projects and usually involve two large iterations over a period of 18 to 24 months.

There are four principles underlying the human-centred approach. For the manager, these represent four clear principles by which the manger should organise their work environment. For the Usability Professional when they are not in a senior management role, these are four areas in which they should be able to give advice and form an opinion in order to facilitate and inform management.

The active involvement of users and a clear understanding of user and task requirements

Although participation by the real end users in the various stages of the design activity is considered to be the most appropriate form of user involvement, it is clear that in some cases this may either not be possible, or may be economically prohibitive. The concept of ‘user representatives’ is broached: more modern methods may include ‘heuristics’, ‘personas’, ‘scenarios’ and indeed ‘design patterns.’ All of these methodologies, and new ones as they arise, must be considered as possibly ‘representing the users and their task requirements.’

If end users are not directly available, whatever approach is used to either supplement or replace direct end user involvement needs to be justified in terms of its adequacy to reflect the real requirements of the users and the tasks they carry out.

An appropriate allocation of function between users and technology

There are many lists of the ‘relative merits’ of humans versus machines. An appropriate allocation of function does not simply put to the machine those functions which are technically easy to implement and leave the rest to the human; an appropriate allocation of function will ensure that the human has meaningful tasks to carry out on the human side of the human-computer partnership.

How the human carries out their part of the tasks has to be considered in the overall design of the system. This is sometimes called ‘user modelling’ although there will be some Usability Professionals who may have concern over the implications of such terminology or query the methodologies involved.

In any case, feedback from the end users or their representatives is required: it is up to the Usability Professional to be able to show how in their chosen area of work in general, feedback from end users or their representatives is obtained and integrated into the technical aspects of the process, or to be able to critique their process constructively if there are things about it they cannot change at present.

Iteration of design solutions

User centred design seems to mandate iterative design practices, and this is possible both when the user centred activities are a ‘second process’ or a ‘process within’ (according to the terminology adopted at the start of this section.) The reason is that the number of influences on design are so many and so varied that it is invariably safer to test a concrete proposal with end users or their representatives than it is to attempt to second-guess by appeal to any body of theoretic knowledge. In any case, in the early stages of design, after user requirements have been collected, the synthesis of these requirements needs to be validated and this leads naturally to an iterative approach.

There are many approaches to design iteration that have been proposed, and new ones appear every year. The Usability Professional must understand those relevant to their chosen domain of specialisation and know the place of user centred activities within these iterative processes.

Multi-disciplinary design

As soon as a person representing user centred design enters the design team, a multi-disciplinary team is created (even if the team size is still only one person!) This stems from the realisation that the end user’s view and the technical developer’s view of the process are views from two different angles. In the standard, ‘multi-disciplinary’ is used both to describe specialists in the process, so for example:

  • application domain specialist, business analyst;
  • systems analyst, systems engineer, programmer.

But also people who play given roles within the process:

  • end-user;
  • purchaser, manager of user;
  • marketer, salesperson.

In order to be able to interact sensibly with a multi-disciplinary team, the Usability Professional must understand the process from the points of view of these other specialists and roles. The roles a Usability Professional may take themselves can be suggested as:

  • user interface designer, visual designer;
  • human factors and ergonomics expert, human-computer interaction specialist.

That is, the professional should know their field of work thoroughly, and be able to interact sensitively with professionals in fields of work impinging on theirs.

Planning the human-centred process

Although planning the system development process for an organisation is an activity which is usually reserved for high management, a Usability Professional should be able to advise on how human centred activities fit into an overall development process.

This requires knowledge, on the part of the Usability Professional, of the various main system design methodologies in current use in their chosen area of specialisation, or if none are used in practice, what should be the best models to adopt; and then an ability to fit human centred design practices within these models, using all the concepts and practices of general project management (resources, personnel allocation, milestones, cost estimation, risk management and so on.)

Note that, in keeping with the general concept of separating the necessary knowledge and skills from the actual practice, it is the manager’s role to ensure that the overall process which they seek to accredit against 13 407 has a human-centred process planned within it; it is firmly within the province of the Usability Professional however to know how this should be done, how responsibilities should be allotted and change control managed.

Human-centred design activities

This is generally acknowledged to be the substantive part of the standard. There are four activities that should take place during a system development project. These are outlined in turn. How many personnel are allotted to each activity, how these activities are managed, and what kinds of methods are used depends on the size and character of the project. In general, the first commandment of user centred design seems to be:

Test with users early in the project and frequently thereafter.

The standard also recognises that for any one project, not all activities will be of equal weight, so a Usability Professional should be able to recommend or order priorities for each of the four activities.

A question frequently asked is: what kinds of methods should our organisation adopt first? The Usability Professional should be able to weigh the needs and capabilities of the organisation seeking such advice and give the necessary contextual recommendations: this is not likely to be the same over all organisations and processes, although one would expect there to be some general broad lines of similarity, perhaps in relation to the general area of application for the organisation and the organisation’s level of quality maturity.

Understand and specify the context of use

There is a fairly formal method of reporting Context of Use recommended in the earlier ISO 9241 standard, part 11. This is already in need of updating as much of it does not apply for instance to internet based applications and different problems raise their heads in the domain of hand held devices. However, the basic questions:

  • Who will use the product (user or audience analysis)
  • What will they use it for (general task analysis)
  • Where will they use it (socio-technical context analysis)

remain universally applicable to most IS products. Context of Use (CoU) usually changes during a project as all the above factors are better understood and tested; early versions of the CoU will guide requirements gathering, later versions will inform product testing. The CoU report itself is one of the important connections between the Usability Professional and the technical design team.

The Usability Professional should be able to gather information for a CoU, produce and communicate their findings to other members of the design team, be able to verify the CoU findings, and know when and how to update the findings.

Specify the user and organizational requirements

There are in summary four main sources for a requirements statement:

  1. (Business): business and financial objectives, system and task performance, work design and organisation;
  2. (User): allocation of users’ tasks, human computer interface, workstation design, communication and co-operation between users;
  3. (Technical): technical feasibility of the system and feasibility of operation and maintenance;
  4. (Legal): legal and statutory requirements including those governing the well being, health and safety of the users.

The Usability Professional should be aware of each of these and be able to play a role in the process of assimilating requirements from all four sources, in addition of course, to their primary role of leading the work on the user’s side of the process (point 2) and in some contexts, the business side (point 1.) It should be noted that the standard sometimes talks of the ‘business and user requirements’ as if they were the same thing.

The statement of legal, technical and (at least some of the) business requirements is clearly beyond the scope of activity of the Usability Professional, but the statement, verification and ultimate testing of the user-centred requirements clearly lies at the heart of their competence. User-centred requirements should be stated as goals for which appropriate levels of attainment can be set, and against which performance may be ultimately measured, following the Context of Use.

In an iterative development situation, the nature of all the requirements may change, especially the user requirements as the users become more familiar with the technology and the applications. The Usability Professional should be able to notice and handle the required change.

The conclusion of this activity is usually the Requirements Document, to which the Usability Professional will have made a contribution. In some development environments this document exists only as a software prototype - it is not actually documented - and in others, the Requirements Document can be of legal standing as a contractual entity which is normally immune to change. The Usability Professional will have to fit in with the prevailing company practice but should also be able to suggest where and how such practice could be improved in order to create better products.

Produce design solutions

Solutions for the design of products are rarely completely under the control of the Usability team. They are usually a multi-disciplinary effort, and the Usability Professional should be able to play their part in this multi-disciplinary process. The process generally begins by using existing know-how, previous patterns etc. to create a mockup or model of a first version, which is then tested against user experience (using real users or user representatives as discussed above). The results from user testing are then fed back into the design, hopefully, until all the user requirements or goals have been met.

Again, it is not every Usability Professional who is privileged to work in such an environment. Usually cost and time restrictions limit the amount of iteration possible, often brutally. The Usability Professional should be able to fit their actions within the prevailing organisational climate of their client or employer, and should be able to critique this climate positively. However, they should be able to manage the development and testing of prototypes within the constraints imposed by their work, and be able to feed back the results to the technical process of design. It is rare to find that user testing at design stage is completely impossible.

Evaluate designs against requirements

The standard notes three senses of the concept of evaluation:

  1. to provide feedback during the design process,
  2. to assess whether the user and organisational objectives have been met,
  3. to monitor the product in the long term (post release).

In addition, we have noted that evaluation should also take place at the stage of Context of Use and Requirements gathering, although these have been referred to in this paper as ‘validation’ to emphasise their nature as checks within these phases of the human centred activity. The standard notes that it is ‘important to start evaluation as early as possible.’

Evaluation in the above three numbered senses at least needs an evaluation plan to guide the activity and it is the function of the Usability Professional to develop this: setting goals, benchmarks, suggesting appropriate methods, testing procedures etc.

However, the standard also lays an emphasis on the importance of reporting the results of evaluation, and issues of design, selection of appropriate apparatus or tests, use of verified procedures, adequate user samples, accuracy of analysis are all mentioned. There are a number of sources now available which give best practice guidelines on how to report usability results; the Usability Professional should be able to select and justify their choice of report format given the situation within which they habitually work.

The Usability report is an important document; for some, it may be the only visible sign of their presence in the design team.

Conclusion

Keeping in mind the intended Context of Use of this standard, it must be remembered that for the primary user - the manager of the development process - these notes are an extremely imprecise introduction. The manager who wishes to apply ISO 13 407 to his process must take the standard extremely literally, and especially be aware of the various shades of difference in a ISO standard between mandatory statements, optional statements and general background. Those unfamiliar with ISO terminology may need assistance with the level of literalness expected by an auditor.

However, ‘others involved in the process’ which of course includes the Usability Professional when they are not the manager of the entire development (a rare enough occurrence at present) ‘should find the guidance relevant.’ This usage is mandated by both the standard, which explicitly says so, and by the aims and objectives of the ISO Organisation. It is to these, the Usability Professionals involved in the process, to which these notes are addressed.

The reader will have noticed that there seems to be a natural distinction between overall knowledge that a Usability Professional should have encompassing the entire standard, and specific skills they should possess in order to be able to carry out certain portions of the standard. As yet there are far too few professional ‘usability’ tools available, and some of the skills mentioned in this standard can be carried out by mastering methods and techniques in common use in the IS industry.

While few would cavail about the knowledge elements summarised in the ‘general principles’ section and most would agree that in fact, a Professional should be knowledgeable about all of these, there is already a difference of opinion regarding the need for a Professional to be able to plan the overall process and to engage in the four areas of Human-Centred design activities.

  • There are those who maintain that, at the current fledgling state of knowledge in Usability, a Professional should have the skills to carry out planning, and activities in all four areas.

  • The contrary opinion is expressed by those who maintain that such an individual would be an ‘organisational Swiss-army knife:’ most workers in Usability already occupy specialised job niches which often prevent them from venturing forth into more than one of the four human-centred design activities.

In addition, although certification should assist employers and customers in choosing the best offer available for them, or setting appropriate conditions in a job advertisement or a tender for contract, many usability practitioners feel that the basis for certification should be individual, not organisational, and that the organisational/ managerial bias of the standard as it is should be softened. There are already excellent organisational derivatives of the 13 407 standard - most notably ISO 18 529.

However, these are open questions which can only be decided by an informed debate among Usability Professionals as they are now. I hope that these notes can stimulate and bring the debate forward.

Acknowledgements

As I fully expect to be informed of the shortcomings of this paper in no uncertain terms by my colleagues and best critics, and as I hope I will be able to answer them in subsequent editions, I will leave this section open for the time being.