Developing Information Systems

Getting the users involved

by Robin Beaumont

Contentss

1. Learning outcomes check list for the session *

2. Introduction *

3. Stakeholders and users *

4. Fundamental aim of systems development *

5. Importance of user involvement *

6. Should users always be involved in systems development? *

7. Mumfords levels of user involvement *

8. Commitment and Training Issues *

9. User groups *

10. ETHICS *

11. Users/Systems developers and Use: A development method *

11.2.1 Contract negotiation and standards setting *

12. Summary *

13. References *


1 Learning outcomes check list for the session

Each of the sessions aims to provide you with a number of skills (the 'be able to's' below) along with relevant information (the 'know what's' below). Details are listed below. After you have completed the session you should come back to these points ticking off those you feel happy with.

 

Learning outcome

Tick box

Know what a stakeholder is

 

Know what the different types of stakeholder are

q

Know the reasons why user involvement is important

q

Know the three levels of user involvement

q

Be aware of the educational requirements when considering user involvement

q

Have a broad grasp of the idea behind Enid Mumfords ETHICS (Effective technical and human implementation of computer systems) method

q

The advantages and disadvantages of user groups

q

Back to top of document

2 Introduction

This chapter investigates user involvement in computer systems development. It considers why it is so important and then describes two systems development methods that are based around user involvement.

Back to top of document

3 Stakeholders and users

The section on 'information systems development methods' provided an overview of the traditional waterfall approach, which precluded the user from involvement in most of the development process. The more recent iterative prototype development method encourages greater user involvement.

Most systems development methods including the two described above, of which there are literally hundreds, encourage the involvement of various 'stakeholders' of which one, may or may not, be the user. Often the first job an analyst does is a stakeholder analysis, literally finding out who they are and how powerful each is.

Stakeholders are considered to be those that have some power over the proposed system. This power may be in various guises:

Often systems, particularly those in the NHS, are implemented for political rather than organisational reasons. The first chapter concerning social issues provides examples of this. Frequently the 'purseholder' has requirements that are at odds with those of the other stakeholders. It is therefore often necessary to prioritise and consolidate the conflicting requirements of these stakeholders at the very beginning of the development process.

It should be noted that all the stakeholders besides the potential saboteurs are at the top of the organisational hierarchy. Yet these are the ones that hold possibly the most power when the system is implemented. Traditionally, probably due to the desire for short term success with the stakeholders, these 'top level' stakeholders where considered the most important in systems development. Unfortunately a large percentage of systems failed when they were implemented using this top down approach and as a consequence bottom up approaches have been developed.

A fundamental characteristic of the bottom up approach is the participation of the user throughout the development process.

 

Numerous frameworks have been developed to assess stakeholders. One such method is that of Johnson (G) & Scholb (K) 1992. They suggested that stakeholder analysis be concerned with assessing Power against several other characteristics, such as Predictability of behaviour and Level of interest these are shown above. The diagrams also indicate via the shaded area, those stakeholders that should play a part in decision making

Johnson (G) & Scholb (K) 1992 also suggest that each of these are assessed by considering actions of the stakeholders and their managers. In the matrix opposite the action considerations for each quadrant are shown.
Exercise:

Given the relative weak power base of 'data entry' personnel but pivotal role regarding data quality and possible subversive activities do you think the above type of matrix analysis is useful in the systems development context?

If you think not what alternative matrix's might you come up with?


Back to top of document

4 Fundamental aim of systems development

Possibly, the most important aim of systems development from the purseholders perspective is to achieve:

System use

However, few activities within most systems development lifecycles explicitly measure or present this as an area of focus. Instead, the specification and compliance of the system to it become the focus of activity. I feel this is because:

The common fallacy that adherence to a software specification is equivalent to system use, when in reality they often are diametrically opposed presents real problems for those systems developers who actually wish to end up with working systems.

Back to top of document

5 Importance of user involvement

Why is it so vital to ensure users are involved in the design process? Some of these issues have been mentioned in the chapter on systems development; danger of sabotage and failure of modellers (systems analysts) to understand the complexity of the system required. The need for end user involvement in the process of data / dynamic modelling and normalisation (for details see the chapter on normalisation) is because they often perceive as common sense what the system developers find incomprehensible, illogical or unnecessarily complex. Similarly, their role in the User Interface (UI) design aspects is vital.

At a more personal level involvement of the users creates a sense of ownership and often loyalty to whatever is developed. Unreal expectations are often a problem with users that are very difficult to modify. However, involvement in the design process is an excellent way of achieving a more realistic viewpoint.

Several studies have demonstrated that increased user involvement increases user satisfaction which in turn increases system use. I would also suggest that user education is also vitally important.

With the danger of labouring the point user involvement basically increases system use.

Back to top of document

6 Should users always be involved in systems development?

While it is ideal that users are always involved in systems development, the consequence of not involving them is directly related to three issues:

Each of these will be discussed in turn.

Back to top of document

6.1 User knowledge / expectations

A recent book concerned with developing systems, Nielson 1993, suggests that 'The user is not always right' (p11) when it comes to asking what they want and provides several examples:

 

I feel that Nielson is rather unfair on the user here. Possibly, what would be more appropriate is to suggest that the un-informed user is not always right. It is therefore important to educate users to obtain valid requirements. A typical sequence of events is shown below

The importance of prototypes, even non-functional screen mock-ups, for engaging users in the development process can not be over estimated.

Back to top of document

6.2 Complexity of organisation

Those organisations, which are highly structured and possess few autonomous professional groups, may get away with a low degree of user involvement particularly with regard to simple transaction processing systems. However, complex organisations with many autonomous groups would be heading for disaster if a minimal level of user involvement were used in the system development process.

Highly structured and few autonomous professional groups = simple

Un-structured and many autonomous professional groups = complex

Royal Mail

Mc Donalds

Marks & Spencers

Police force

Health Service

Criminal justice system

University

 

Back to top of document

6.3 Complexity of system

As with the type of organisation the type of system under development determines to some extent the appropriate level of user involvement required for a system to be successful.

The term 'complexity of system' needs some explanation in this context. Which is more to do with the complexity of the algorithms and uses the system is supposed to undertake rather than the size or geographical coverage of the system. For example it would be much more important to involve users in the development of a specialist system to help with diagnosis of skin conditions than a nation wide billing system for a computer supply house. Transaction processing systems (often of the back office type) are simple in comparison to those designed to assist professionals carry out complex largely self-determined tasks. Churchill, Kempster & Uretsky suggested in 1969 that systems could be related to the level of structure of the tasks undertaken. The simplest systems working with the most structured tasks and the most complex working with unstructured individually defined tasks. While the simpler type of systems has become common place since 1969 few, if any computer systems, to support the relatively unstructured tasks have emerged. Friedman & Cornford provide a wonderfully insightful discussion of the history of systems development

This can clearly be seen in the medical context. Large numbers of GPs use computer systems for routine highly structured tasks such as prescribing and patient registration. However it is a very different story concerning 'expert systems' that is those that carry out part of the diagnostic process. Despite literally hundreds of projects demonstrating the diagnostic accuracy of these systems compared to appropriate doctors, none have gained wide spread use. The feeling now is that the best one can hope for in this context is to develop systems that 'Support' these unstructured tasks rather an automate them. The reasons for this are complex, involving at least professional and system adequacy factors.

Back to top of document

7 Mumfords levels of user involvement

Enid Mumford has been writing for the last twenty or so years on 'participative design'. She has been involved in large systems developments including international banking. More importantly she has produced several books describing how her method has been used in the Health sector (1993) Enid Mumfords approach to systems development is based upon the idea that any system should fulfil two critical criteria:

Related to the above criteria she introduces two additional concepts; the idea of 'Job enrichment' which the aim is to improve the relationship between the individual and his work. Secondly the 'Socio-technical method' which is the idea of small units of workers that develop very close joint working relationships and take control over a specific area.


Exercise:

List five ways a computer system might enrich your job and increase your job satisfaction


Mumford 1983, identified three levels of user participation:

  1. Consultative participation - The users are consulted but most of the decision making is still left with the traditional systems design group (top level stakeholders and the designer).
  2. Representative participation - Design group formed from representatives of all grades of staff in each department. Members selected by management. 'Representative participation has the advantage that clerks and systems analysts work together on equal terms. It has the disadvantage that it assumes that representatives really do represent the interests of their constituents and this may not always be the case.' (Mumford 1983 p6).
  3. Consensus participation - Attempts to involve all staff. Design group formed from representatives of all grades of staff in each department but these time members selected by elections from the shop floor.

Exercise:

Which of the above three levels of participation do you feel is most appropriate in your organisation? Provide reasons.


Back to top of document

 

8 Commitment and Training Issues

Within a complex organisation many levels and types of training are required. For example the NHS a special authority with regional offices co-ordinates the process. (http://www.enablingpp.exec.nhs.uk) along with the numerous professional bodies (e.g. Royal colleges etc).

Clearly if users of all levels are to be involved in the design process to develop valid user requirements training will be required. This also requires additional resourcing to allow people to drop out from their present work commitments. Related to this is the issue of commitment. Using this approach to systems development means that the users can not be passive but must 'take control' of the development process.

Unfortunately if eager users are not effectively educated, they can become problematic (often called disparagingly 'fiddlers'), Within the GP community this problem is managed to some extend by the user groups.

Potential groups may include:

IM&T professionals within the organisation - Clearly the organisation must have the infrastructure or develop one to support these activities. Most IM&T professionals will require training in participative development techniques. The use of training consultants should at all costs be avoided, as the aim is to develop skills in house.

Managers - Education regarding modern systems development methodologies should be a priority for these.

Professional bodies / Unions - These can often provide either the necessary carrot or stick to encourage user involvement.

Movers and shapers - As part of a long term strategy potential 'movers and shapers' should be identified and given highly desirable incentives to undertake computing / informatics courses. These preferably should be full time and lead to degree or MSc. status.

Ground level users - These should be the focus of the educational process.

Students - It is most important to involve students in the process. This may well result in innovative requirements and use of data such as production of log books for training accreditation.

Clients / Patients / Cases - If the system is intended to be used in situations that involve any of these groups they will also require training.

I will not mention the training requirements in detail for any of the groups. However training for the last three groups involves:

Quite a hefty task for most people, emphasising the degree of commitment required by top level stakeholders for this method to work. A significant number of people will need a significant amount of time out for training and subsequent system design activities and for complex systems the process will undoubtedly be ongoing. This emphases the requirement to develop the necessary infrastructure rather than depend upon consultants.

Commitment may need to be bought by offering financial incentives.

Back to top of document

9 User groups

A more pragmatic approach taken regarding user participation for long term system developments is the setting up of user groups. These groups, which are often set up by software companies, provide a number of advantages. These include providing a method of filtering and prioritising design suggestions and provision of a set of individuals to try out new software before going on general release (beta testing). The disadvantage of such groups results from the lack of power they often have, suggestions are ignored or the user group is just political window dressing for the software company.

Within the NHS user groups exist for most of the large PAS systems, as well as for Read codes. Also 'care pathways' have a user group which has representatives from over 150 hospitals. All the major GP systems have user groups.

Most user groups provide a number of methods for communicating with and between users.

Back to top of document

9.1 Communications Strategies

These may include:

The next sections disscuss two systems development methods which focus on user involvement.

Back to top of document

10 ETHICS

ETHICS is an acronym for Effective Technical and Human Implementation of Computer Systems, it is a method that Enid Mumford has been developing over the past fifteen years. In contrast to most systems design methods, in ETHICS user participation is fundamental at every stage.

ETHICS is also unusual in that it forces the search for solutions that take into account 'social' as well as technical factors. The whole process can be considered as a method of balancing the costs and benefits of social and technical solutions upon job satisfaction and efficiency.

The ETHICS methodology contains six stages which are further divided into twenty-five steps (Mumford, 1983b). These are shown below.

Back to top of document

 

Stage 1 Essential Systems analysis (steps 1 to 11)

This is the preliminary phase of the ETHICS method. It involves identification of the boundaries of the system (e.g. a particular clinical department) as well as a description of the current system, if any is present. The key tasks and objectives of the proposed system are stated. Each of the interested groups then rank the list of objectives on a scale of one to five. Both the efficiency and job satisfaction requirements are initially considered separately before being consolidated.

Stage 2 Socio-technical systems design (steps 12 to 20)

This stage attempts to reconcile the social side with the technical side of systems design. Firstly the business, social and technical constraints are identified. Two different groups are formed (one focusing on the social aspects of the system; the other the technical aspects) whose job it is to find socially and technically desirable design options. After identification of the social and technical constraints, the resources available for each of the socially and technically desirable options are considered. The objectives (in ranked order) are then checked for comparability before actual technical and social systems decisions are taken. Revision may be necessary before this final step is completed.

Stage 3 Setting out alternative solutions (steps 21 to 23)

This stage involves the evaluation of any alternative technical or social solutions. These are set out in a matrix form evaluating possible advantages and disadvantages as well as overall compatibility with the established objectives. As in the previous stage, each will be evaluated against three criteria, priority, constraints, and resources. Once doubtful solutions are eliminated, a short list of technical solutions and one of social solutions is drawn up.

Stage 4 Setting out compatible solutions (step 23)

This stage merges the short lists from stage three to see which solutions are most compatible. Incomplete solutions are discarded. Technical and social solutions found to operate well together are entered into an evaluation matrix for the next stage.

Stage 5 Ranking socio-technical solutions (step 24)

The matrix set up from the previous stage is now ranked using information generated in stage 3, while ensuring all socio-technical solutions meet the criteria outlined in stages 1 and 2.

Stage 6 Prepare a detailed work design (step 25)

A detailed list and description of all tasks people would perform under a particular socio-technical solution's implementation is drawn up. Tasks are ranked in order of simplicity and attempts are made to provide a balanced spread of required skills and complexity of tasks. Checks are made to ensure that created jobs are as interesting and satisfying as possible using a set of 'issues of concerns'. If the highest ranking socio-technical solution scores high on these issues while achieving the technical objectives, it is accepted as the final solution. If this is not the case, another short listed solution is tried in the same manner.

ETHICS adopts a number of special methods for systems development. For example, there is a special diagramming method used for describing work layout. There is also a job diagnostic questionnaire which is used to elicit views on the job situation. More importantly, ETHICS, employs a facilitator who seeks to find a consensus on the systems development exercise using special questionnaires.

Other papers in this area include an interesting discussion regarding the measurement of job satisfaction, by Wanus & Lawler 1972 and the large body of literature that has developed over the last decade concerning stress and work. One recent study is that by Arnetz, Arnetz and Petterson concerning stress from violence in Swedish nurses.

Back to top of document

11 Users/Systems developers and Use: A development method

Most systems development methods have several problems: they assume limited input from end users; they encourage the centrality of the specification, and they assume the commissioning body has the ability of to carry out prolonged detailed negotiations with systems developers.

 

The proposed development method described below is derived that which has been successfully adopted by Prodigy (http://www.schin.ncl.ac.uk/prodigy). An iterative software development for GPs to support prescribing.

Back to top of document

11.1 Independent evaluation / contract negotiation team

Within the development method there is a need for an independent Evaluation / contract negotiation team. The team provides the following functions:

Back to top of document

11.2 Process

This can be thought of an iterative development process which is covered in more depth elsewhere. The key stages are:

  • Initial prototype development with very limited functionality
  • Evaluation
  • Formulation of new specification
  • Contract negotiation
  • Development
  • Evaluation
  • Formulation of new specification
  • Contract negotiation
  • Development
  • Evaluation
  • Etc. No end point
  • The relationship between the various evaluation / communication, specification activities is shown below.

    Good project management is vitally important as the results from the various evaluation activities feed into the specification development process for the next iteration.

    Back to top of document

    11.2.1 Contract negotiation and standards setting

    In contrast to traditional contract negotiations that centre wholly around system design specification this development method additionally specifies minimum acceptable levels of Use and user satisfaction.

    Realistic targets for the key items from the questionnaire(s) should be mutually agreed with the developers along with penalty clauses and/or bonus incentives dependent upon the results. The quality, quantity and format of training should be included along with a wish list for users in the questionnaire(s).

    Typical headings from a system evaluation questionnaire might include:

    Some typical questions are provided below from the Prodigy evaluation questionnaires:

    How would you rate the trainer's assessment of your particular learning needs for the system?

    extremely limited limited adequate good expert

    How would you rate the trainer's ability to communicate effectively when answering questions about the system?

    extremely limited limited adequate good expert

    How useful did you find the documentation as a . . .

    General introduction to the system not at all little use useful extremely useful

    Definitive reference not at all little use useful extremely useful.

    'How to do it' not at all little use useful extremely useful.

    Source of exercises etc. not at all little use useful extremely useful.

    Code reference not at all little use useful extremely useful.

    Guide of where to get additional help / documentation not at all little use useful extremely useful.

    How do you rate the layout of the output grossly inadequate inadequate adequate good excellent

    Can you print output while with a client yes no

    The independent evaluation team should administer the questionnaire.

    Numerous generic (e.g. QUIS see Chin et al 1988) and domain specific questionnaires (e.g. in the health arena Bailey 1990) have been developed. Anderson et al 1994 provides an appendix (p.97) of survey instruments.

    Back to top of document

    12 Summary

    This chapter has investigated both why and how users should be involved in systems development. A detailed description of two methods was given. The resource implications of such approaches were discussed. The more traditional, pragmatic approach of long term user groups was introduced.

    Back to top of document

     

    13 References

    Anderson J G, Aydin C E, Jay S J. 1994 Evaluating health care information systems - Methods and Applications. Sage publications.

    Arntez J E. Arntez B B. Petterson I L. 1996 Violence in the nursing profession. Work and stress 10 (2) 119 - 127

    Bailey J E 1990 Development of an Instrument for the management of computer user attitudes in Hospitals. Methods Information Med. 29 51-56

    Chin J P, Diehl V A, Norman K L 1988 Development of an Instrument Measuring User satisfaction of the Human-computer Interface. CHI (may 15-19) Also a computerised version "QUIS" available from Klnorman@umd2.umd.edu

    Churchill N C Kempster J H Uretsky M 1969 Computer based information systems for management. National association of accountants. New York

    Friedman A L Cornford D S 1989 Computer systems development: History, organisation and implementation. John Wiley & sons Chichester

    Hirschheim Rudy, Klein Heinz K, Lyytinen Kalle. 1995 Information systems development and data modelling: Conceptual and philosophical foundations. Cambridge university press. UK

    Mumford Enid 1983 Designing participatively: A participative approach to computer systems design. Manchester business school. UK

    Mumford Enid 1983b Designing human systems - The ETHICS method. Manchester business school. UK

    Mumford Enid 1993 Designing human systems for Health care - The ETHICS method. 4C corporation [ISBN 90 74687 01 6]

    Nielsen J 1993 Usability Engineering AP professional London

    Wanus J P. Lawler E E 1972 Measurement and meaning of job satisfaction. Journal of applied psychology 56 (2) 95 - 105

    Back to top of document

    Written by Robin Beaumont e-mail: robin@robinbt2.free-online.co.uk Date: 03/06/99 Tel 0191 2731150

    Document management info:

     

    Source: C:\HIcourseweb new\chap12\s4\des2.doc