|
|
We report on an examination of work practice in a knowledge-based, document-intensive organisation and describe the role of paper in that work. We show how such an examination can provide a resource for (1) the determination of system design modifications that can be undertaken in the short term; (2) the determination of entirely new systems design requiring longer term research and development; and (3) helping to specify where paper will continue to be used in future document-related work practice.
paper, ethnography, knowledge work, diary study, collaboration, design, organisations, IMF
For many years now the paperless office has been held out as the goal for organisations: with such an office, organisations will be able to create, distribute, store, and use information in new and more effective ways. Furthermore, organisational theorists explain that without paper, new organisational forms will emerge, curbing the shocking waste of wood pulp and the ever increasing cost of paper storage. Yet paper has obdurately remained a conspicuous fact in organisation life: even the most "hi-tech" environments find themselves increasingly burdened down by it. The emergence of yet another digital document form, the Web, though introducing new possibilities for document access and delivery, seems unlikely to alter the situation.
Why is this? Is it that a cocktail of cultural inertia and non-progressive attitudes is forcing organisations to continue to rely on paper for much of their documentation? Of course, from our own organisation's point of view, the continuing use of paper is good news: paper is a major source of revenue. But from a research and design perspective, the issues are quite different. For one thing, the continuing use of paper begs questions about how adequate technological alternatives to paper have been over the past thirty years or so. For another, it draws attention to the question of whether researchers have properly understood the needs of users in organisational settings. Beyond all this, paper is an awkward subject to investigate since it is a symbol of the uninteresting past, not the exciting future. As a result, there is very little research that has looked systematically at the role of paper in organisational life, and its continuing use has remained largely unexplained in the literature.
But paper does not need to be viewed this way. We contend that its continuing use can be seen not as a problem but as an analytic resource. In other words, an examination of why and how paper is used in existing work processes can be seen as a way of directing and inspiring the design of new technologies. It may do this in three ways:
Our approach, then, is to use the existence of paper in organisational life as a tool in analysis, not as a problem. That is, by understanding how the properties of paper support document-related tasks, we might then be able to determine the potential for other document technologies in those tasks. This is not to say that digital alternatives need to mimic the properties of paper, but rather that they need to take into account what one might call for the sake of convenience the affordances of paper in those tasks, and attempt to provide those affordances perhaps in other ways. This approach also entertains the possibility that paper itself may be the best technological tool for some kinds of tasks, and that this is likely to be the case for the foreseeable future.
We are carrying out such investigations systematically with a combination of laboratory [12] and field studies. We report here on one of the field studies - an examination of the role of paper in the work practices of the International Monetary Fund (IMF) in Washington, DC. This study is part of a long term investigation in which we have also focused on implications for particular kinds of technology (e.g., for groupware [8]; and workflow tools [15]).
The IMF is a financial "club" whose members are the countries of the world. These countries are obliged to pool resources which are used to provide multi-currency loans to members who find themselves in balance of payments crises.
The IMF has some 3,000 staff, of which 900 are professional economists. These economists analyse the circumstances that lead to a balance of payments crisis, and determine criteria for the making of loans. These assessments and criteria are contained in the IMF's "staff reports". These are used by the organisation's Executive Board for its decision-making.
With regard to technology, the IMF's 3000 users are connected by a Novell network. The economists have PC laptops with docking stations and their own printers. All staff are trained in WordPerfect but are gradually being converted to Microsoft Word running on Windows. The main spreadsheet tools are Excel and Lotus 1-2-3, and time series tools include AREMOS. Users have access to electronic templates for the construction of IMF standard documents and tables.
The IMF is an ideal place to study the role of paper for several reasons:
The work we will focus on here relates mainly to the IMF's mission process. In brief, this process involves a team from the IMF visiting a member country. The team normally involves: a desk officer (an economist responsible for gathering and storing data about a member country throughout the year), a chief (also an economist and the desk officer's boss), three other economists, and one or two administrative support staff. Over a period of about two weeks, the mission team prepares an analysis of that country's macroeconomic situation. This analysis, which may include recommendations for policy changes by the member authorities or for a loan by the IMF, is later presented to the IMF's Executive Board in a staff report and associated documentation.
The IMF study relied mainly on ethnographic techniques, the first part of which involved six months field work, observation of a mission, and interviews with 138 personnel [7]. This field work was later supplemented with a "daily activity study" conducted over 5 days in May of 1995. This was a detailed study of 25 of those people previously interviewed.
To ensure representativeness, these people had a range of jobs: 7 administrative staff, 2 research assistants, 11 economists, and 5 chiefs or deputy chiefs. These individuals were also responsible for the three main types of IMF work, namely, work related to the analysis of major economies, minor economies, and developing economies. A final criterion was that these staff were operating within the main stages of the annual work cycle: preparation for a mission, post-mission work and the relatively quiet period in-between missions.
The daily activity study was essentially a method of enquiry relying on diary-keeping by the people involved. Part of its purpose was to focus specifically on the role of paper in the working lives of these 25 people. For a period of five consecutive days, we asked each member of staff to list their planned activities for the day first thing in the morning, and to keep a log of their activities during the course of the day. Then, at the end of each day, we interviewed each staff member to ensure a complete log of which activities they carried out and when. During these interviews we focused on issues such as the nature of the activities they carried out, how long these activities took, and the kinds of documents and document-related tools they used, including the medium (i.e., paper or electronic).
Data from the ethnographic and daily activity studies were combined to generate a picture of the routine activities of the IMF's professional workers and those who support that work. This picture included an understanding of the practical reasoning of these workers and detailed data enabling comparison and quantification of tasks across a section of workers within the organisation.
We could find no data in the literature describing the degree to which contemporary office work is based on the use of paper versus electronic document technologies. While there are plenty of marketing data on general paper consumption, there are no studies which attempt to quantify and explicate paper use in the context of the day to day activities of individuals in an organisation. Further, while there are many attempts to analyse the various activities of office workers (e.g., [9]), such analyses do not link the activities to the way in which they are carried out, or to the document media used.
With the data from the daily activity study we were able to carry out a fairly comprehensive quantitative analysis of the activities and the document media used by at least a subset of our sample. We first constructed a taxonomy of their activities (see Figure 1). This began to some extent by relying on the workers' own descriptions of what they were doing (e.g., "editing a document", "dealing with the mail", etc.). We then compared our initial taxonomy to other existing taxonomies [2,3], and finally, with the two other members of the research team, agreed on one which seemed to be a good general tool for classifying the activities of all 25 people in the study. In carrying out this analysis we also categorised each activity according to whether it involved no documents, paper documents, on-line tools, or a mixture of paper and electronic tools.
The analysis confirmed these workers' heavy reliance on documents, and in particular on paper documents. In terms of general statistics on document use, for these 25 people we found:
For reasons cited earlier concerning the importance of studying knowledge work, of particular concern to us was the IMF's economists (this includes desk officers and chiefs). These are the individuals who create most of the IMF's documents in the first instance, and thus their selection of document medium has an impact for those who support their work processes, such as administrative and secretarial staff.
Although there were 16 economists in the sample, due to their busy schedules and days away from the office, only 8 of them provided complete data on which to base our quantitative analysis. Figure 1 shows the activity profile we were able to construct. The profile shows that a large proportion of their time was spent on authoring activities, as one might expect. The figure also shows the extent to which these processes relied on paper, or a combination of paper and electronic tools. In particular, collaborative authoring processes, either in co-authoring a document or in reviewing the documents of others, were heavily paper-based. Paper was also often present in the drafting and editing of their own text and data, although this tended to be in conjunction with on-line tools.
One can also see from Figure 1 that over half of conversations and the majority of meetings were supported by paper documents. Of further interest is the fact that it also tended to be the preferred medium for reading documents, for document delivery, for thinking and planning activities, and for document organisation. We now examine why this was so by combining our daily activity study data with ethnographic analyses for a selection of these paper-based activities.
Figure 1. The activity profile for 8 desk officers and chiefs.
One of the results of the daily activity study was to show that economists spent far more time editing and reviewing documents (71% of their authoring time) than actually drafting documents (29% of their authoring time). Further, we found that while drafting was done by individuals when alone, editing and reviewing of documents was either done alone or in collaborative situations. These editing and reviewing activities involved both their own documents or the documents of others.
The ethnographic work showed that the large amount of time spent editing and reviewing has to do with the fact that almost all of the major documents that economists at the IMF deal with (most especially staff reports) are co-authored. This means that much of the authoring work is concerned with the integration of sub-sections of documents created by different people. A central feature of this process is the fact that these documents are much more than the sum of their individual parts. When the constituent parts of a document are brought together (such as different chapters and tables for a staff report), there is often a need for extensive negotiation and editing of that document's subsections. It is vitally important that the content of each section be checked and iterated with reference to the larger document.
One reason this integration is a labour-intensive process is that, as we have commented on elsewhere [8], the sub-sections that the IMF's economists produce involve significant degrees of professional judgement. As such, these judgements must maintain consistency with the judgements of other people so that any single document presents a commonly agreed set of interpretations. Although extensive re-drafting and review occurs with individual sections of documents, it often occurs at the point when authors start to integrate the sections of their documents.
This helps to explain the fact that while the economists in our study mainly revised and edited their own sections of a document on-line, 89% of the time this was done in conjunction with paper. These paper documents consisted mainly of other sub-sections of the larger document, or of staff reports from previous years. Cross referral to these other documents helped to ensure consistency and continuity of content. Having them in paper form meant that they could be kept simultaneously available for quick reference and comparison to the on-line document in progress. Limited screen size, and the problems this creates in terms of the obstruction of one document window by another, make this very difficult to do on a workstation. Having said that, the electronic authoring tools they used were useful in ensuring consistency in the format of their documents, if not in content.
The activity analysis also showed that much of the editing and reviewing process involved the review of other people's documents. This was always done by marking up the document on paper (although 10% of the time it was done in conjunction with electronic files). The reasons for this have to do with the affordances of paper for this kind of task.
One affordance of paper that emerged as important is that marking up a document preserves the distinction between a reviewer's comments and the original text. This is true in two senses. In one sense the distinction is a true separation between suggested modifications and actual modifications. In other words, by marking on paper, actually implementing the changes is left to the discretion of the author and owner of the document. In another sense the distinction is perceptual - it is easy to perceive the comments as distinct from the text, so that areas where changes are suggested can be seen at a glance.
Some currently available electronic authoring tools do offer a markup capacity with some of these features. However, a second point that arose is related to the fact that the markability of paper provides a richness of representation that most current electronic systems do not support. For example, in cases where the same paper might be reviewed by more than one person, the identity of the reviewer can be conveyed by the nature of the markings. One can also convey ideas in a variety of ways, by easily adding text, graphics, and sketches of tables, for example. Proofreader-like symbols can also be used to indicate the ways in which text might be moved, deleted, or otherwise modified. This kind of mark-up language provides much more flexibility for annotation than most on-line text editors support.
We have described the ways in which the economists in our sample edited and reviewed their own and others' work when they were alone. But Figure 2 shows the degree to which document editing and review was also collaborative, involving meetings in which documents were discussed and revised. Specifically, we found that nearly half (44%) of these activities were carried out in face-to-face meetings.
Figure 2 also shows that this collaborative review process was especially reliant on use of paper. In our data it was centred around paper 82% of the time, the other 18% of the time involving on-line tools in conjunction with the use of paper documents.
Figure 2. Time spent revising and editing both alone and with other people as a function of different media.
Figure 3. Two days of data from the daily activities of 5 people involved in the production of a document: a director, chief, deputy chief (D.C.), and 2 administrative assistants (AA). White boxes refer to activities that mainly involve the use of paper; shaded boxes refer to activities which mainly involve the use of on-line tools. The order of activities is represented along the horizontal axis. Dotted vertical lines indicate face-to-face meetings.
The kind of collaborative authoring undertaken is further illustrated by looking in more detail at an example of the process over time. Figure 3 shows the document-related activities for the creation and review of a note for the Executive Board taken from our daily activity logs. This case involved a director, a chief, a deputy chief, and two administrative assistants. Note that, while much of the drafting and editing of the text took place on-line by the deputy chief alone, there was extensive editing and review of the document on paper and in meetings with either the director or the chief. The process was iterative, involving several collaborative review sessions interspersed with on-line editing and then printing.
The ethnographic analysis showed similar processes for other types of documents when they were reviewed. Observation of review meetings and interviews with staff indicated that paper was essential to these collaborative processes because it supports the social mechanisms that occur during these activities. One reason is that the flexibility and tailorability of paper is non-disruptive - discussion can be easily carried on in parallel with the marking up and concurrent viewing of parts of a document.
Second, in these situations paper is a medium that provides at-a-glance information so that people who are co-present can easily discern the activities of their colleague with respect to the document being discussed. The physicality of paper means that reviewers sitting around a desk can tell whether the other person is turning toward or away from the document, helps them to see approximately where in the document they are, and tells them whether a colleague is flicking through pages, or is setting it aside. Having a sense of these activities helps reviewers to co-ordinate and focus their discussion. Similar kinds of observations have been made in a variety of other work settings [10,14,16].
Another activity in which paper played a key role was in the delivery of documents. By key role we do not mean here that hand delivery of paper was a very frequent activity (as Figure 1 attests), but rather that when it occured, the activity was perceived to be an important and significant event by the people involved.
At the IMF, once a important document like a staff report has been drafted, it must be delivered to various other functional departments such as the Policy Development and Review (PDR) department for external review. If we look closely at the nature of document delivery within the IMF, it becomes clear that there are important issues to do with the process of handing over documents between the departments that produce them and the departments that review them. For, although automatic routing is part of current practice, on many occasions, producers of documents want to be involved at the point of delivery and prefer to hand deliver paper documents. This is especially the case in relation to documents for review by PDR. There are a number of reasons for this:
The final paper-based task we will discuss is reading. Reading is critical to the work of economists at the IMF, both as part of the authoring process, and as an activity unto itself. Because it is embedded within other activities, it was difficult to quantify the extent to which it was paper-based. However, there were some "pure" reading activities (appearing as "Reading Only" in Figure 1) which were heavily reliant on paper. For example, desk officers are routinely passed a folder of news stories from their member countries to help them keep on top of current events. One of the jobs of a research assistant, at least in some departments, is to select out the items of interest from on-line news services such as Reuters, and then to print them out and distribute in paper form.
However, we found the printing out of long documents for the purpose of reading to be much more pervasive in economists' work. In the course of their authoring work, whenever they needed to read and understand a document for the subsequent structuring of ideas, or for the composition of another document, it tended to be printed.
Motivated by these findings, just why reading from paper is preferred to reading on-line became the subject of further enquiry in our laboratory. Unfortunately, the literature seemed to offer little in the way of explanation: experimental tasks seemed too contrived to be meaningful, and the differences between paper-based and on-line reading were often subtle and unremarkable [see 1,4 for reviews]. This led us to design our own experiment using a task which we felt was more representative of the work of economists, and analysing the data in a broadly descriptive way [12]. This highlighted important differences in the ways that users interact with paper compared with on-line documents. Unlike much of the existing literature comparing paper to screen, none of these issues have do with issues of screen resolution, contrast or viewing angle. Rather, the critical differences have to do with the major advantages that paper offers in supporting annotation while reading, quick navigation, and flexibility of spatial layout. We found that these, in turn, allow readers to deepen their comprehension of the text, extract a sense of its structure, create a plan writing, cross-refer to other documents, and interleave reading and writing.
This analysis has focused on current work practices and the reasons why paper provides support for a range of important tasks. Our view is that such examinations can highlight those areas where current technology may be rapidly altered to more readily fit users' needs, can help specify longer term research and development goals to ensure that digital technologies offer an effective alternative to paper, and can specify those places in which paper is likely to continue to play a role in organisations. Of course, in practice there will be a blurring of the boundaries between these three different outcomes with some short term developments moving into the long term, for example. In any case, we believe the findings from this single case study already offer some important design implications. It is to those that we now turn.
Document creation is largely transferring to the electronic world for a variety of good reasons including document modifiability and re-use. However, the IMF study has shown that paper continues to play a role in at least two ways: in providing simultaneous, quick access to other documents during the drafting and editing process, and as the medium through which other people's documents are reviewed and annotated.
In the short term, the development of larger displays or the use of multiple displays may improve access to other documents during on-line authoring. However, merely increasing the space within which other documents can be displayed on-line will not necessarily solve the problem of quick access. As we have found in our studies of reading [12], paper supports much more efficient movement to and fro between different parts of the same document. We would therefore expect that major improvements in navigational techniques would also need to occur in conjunction with changes in display size before paper is abandoned in the on-line authoring process. In the meantime, systems which can electronically capture information from paper documents may benefit this sort of situation [e.g., 11].
With regard to mark-up and review of documents, indications are that commercial authoring tools are beginning to turn their attention to providing support for these processes (e.g., the Revisions feature in Microsoft Word). Such tools maintain a distinction between suggested modifications and the original document, and allow authors to retain control over which changes get implemented.
However, collaborative markup systems which combine these aspects with stylus-based marking techniques are rarer, and seem mainly confined to the research world (e.g., [6]). Stylus-based input technologies support a richer, more flexible means of annotation, and handwritten markings provide the added benefit of being a perceptually distinct layer on top of typed text. This could be greatly improved by considering the use of dimensions such as colour and pressure-sensitivity to control thickness of a line. As further support for this kind of marking activity, it seems wise to consider how displays and input devices should also be modified. The use of portable, wireless, pressure-sensitive displays could add to the "markability" of digital documents.
This study has found that paper is essential in supporting the social mechanisms that occur during collaborative authoring and review. Some of these issues have to do with the tailorability of paper and the ease with which it can be marked up during ongoing discussion. As outlined above, advances in stylus-based interaction, and in the markability of digital display surfaces will likely impact this aspect of collaborative use.
However, paper also affords the perception of information at-a-glance. It does so because the document is embodied in the physical medium of its display - i.e. the paper. Furthermore, and related to this, there is a fixity to paper based documents. In effect, these properties mean that paper renders information tangible. These properties, combined with the ease with which paper can be spread out in space, means that people's actions in relation to their documents are made visible because of the ways in which they orient toward and interact with paper. This in turn helps to coordinate collaborative action and support social processes.
In general, designers of collaborative systems have tended to pay much more attention to the support of shared editors for people who are remote rather than co-proximate (e.g., Aspects, Timbuktu). Technologies specifically designed for people co-authoring in face-to-face meetings are rare but can be found in the research world [e.g., 13]. Such systems may involve providing access to electronic documents through multiple workstations. However, Luff et al. [10] have pointed out that, unlike paper, interacting with documents within a limited, fixed screen causes actions to be localised and thus less visible to one's co-participants. Advances in mobility, size, and viewing angle of digital display screens may be able to decrease this localisation problem to some extent. However, the fact that digital documents are dynamic in nature may act to undermine these processes. If information is displayed dynamically, then an onlooker will have less certitude about what a colleague is orienting to, or turning away from. In short, the "at a glance" property of documents is compromised by the dynamic nature of digital documents. An alternative is the "shared white board" approach where participants attend and interact through one large electronic display (e.g., Xerox's Liveboard). This is a more general kind of meeting support tool, and its use for collaborative authoring needs to be demonstrated in real work settings.
Advances in the speed and connectivity of digital networks, and in document exchange standards and interoperability will have a significant impact on the speed and ease with which documents can be distributed and shared. However, this study has outlined a number of reasons why people may still want to be involved at the point of delivery. In particular, the findings underline the need for discussion when documents are delivered, the symbolic and ritual importance of hand delivery, and the need to confirm delivery. Even though fast, effortless delivery is often cited as one of the major advantages of the use of electronic document systems, there is no reason why electronic documents cannot also be hand delivered. Handing over floppy disks or CD ROM's, or providing ways of "handing over" applications on PDA's could support this function. While this would fulfil some of the above requirements, one way in which the process would differ is that discussion could not also be accompanied by visually "walking through" a document together. PDA's are too small to view documents effectively, and floppy discs and CD ROM's obviously offer no viewing at all.
But the implications of these findings can go beyond what happens in co-proximate environments. We may also use these findings to consider how one might develop electronic document delivery systems that take into account the need for discussion and the presence of people in the handing over process over large distances. One possible approximation to this is the use of video "telepresence" in document delivery between remote sites. Unfortunately, most commercial videoconferencing systems do a poor job of supporting the sharing of documents during ongoing face-to-face discussion. Most require making a choice between viewing a document or viewing one's co-participants, and there is no support for shared pointing and gesturing over documents. These shortcomings are not so much due to limited bandwidth as a failure to recognise the importance of document-centred activity in the course of work. Systems in the research world are making progress [e.g., 5] but it is likely to be some time before video systems give up their "talking head" model of conferencing. When they do, we predict that advances in such systems will have important utility for document delivery. At the IMF for example, one can imagine how the mission process might be better supported by integrating video links with document viewing and delivery services.
Documents which do not rely heavily on their linear structure, and which are primarily used for search and retrieval tasks for small sections of text and well-defined pieces of information (e.g., dictionaries and technical manuals) are in many ways much better suited to use in the digital realm than paper. We are seeing that such things are rapidly and successfully going on-line, and they will continue to do so.
However, the development of digital alternatives in support of reading, especially for the reflective reading of long, linearly structured documents, presents a major design challenge. The continued reliance on paper in such reading activities at the IMF indicates that for digital alternatives to rival the affordances of paper, there will need to be changes to many different aspects of hardware and software. These include improvements to support annotation while reading, in speeding navigation, and in improving the flexibility of spatial layout. This will involve systematic research and development into more flexible and less constrained input techniques, better display technologies, faster system response times, and the provision of multimodal feedback, to name but some suggestions (see [12] for more detail). Because of the sheer complexity of the problem, we predict that paper in support of some kinds of reading tasks will be one of the hardest paper-based tasks to shift to the digital domain.
Our study has shown that there are "good reasons" for the continuing use of paper in organisational life in a wide range of tasks. Our research at the IMF, combined with work undertaken in numerous other settings (e.g., hospitals, police organisations, manufacturing, air traffic control, and communications companies) has led us to conclude that the role of paper is partly related to the institutional setting in which that work occurs but also reflects general practices within all organisations. What differentiates the institutionally-specific from the generally-applicable is essentially a matter of degree. So, for example, we have argued [8] that the extent and importance of collaborative authoring at the IMF reflects the difficulty of the analyses it undertakes. In other settings less difficulty is associated with such work, and so paper plays a smaller role in institutional practice in that capacity. Similar observations can be made about reading, document referral and annotation processes, and document delivery practices.
Finally, we believe we have only begun to scratch the surface of the role that paper plays in organisational life. For example, we have not yet begun to consider how personal preferences or generational differences may affect that role. Nonetheless, while recognising the limits of our research, we believe we have demonstrated how a focus on paper use in real work settings can serve as a resource in guiding the design of new technologies. Perhaps by being more sensitive to the affordances of paper, shifts toward digital office environments can be achieved more effectively. At the very least, recognition of why people continue to use paper should signal to designers and strategists the need to take seriously the inevitability of a mix of paper and electronic documents well into the future.
Many thanks to Marge Eldridge and William Newman of RXRC Cambridge who formed the other half of the research team for the daily activity study. We greatly appreciate the help of the IMF, especially Christa Dubb, Warren Minami, Chris Yandle, and the individuals in the study.
1. Dillon, A. (1992). Reading from paper versus screens: A critical review of the empirical literature. Ergonomics, Vol. 35(10), 1297-1326.
2. Eldridge, M.A., Barnard, P.J., & Bekerian, D.A. (1993). Autobiographical memory and daily schemas at work. Memory, 2(1).
3. Engel, G., Groppuso, J., Lowenstein, R., & Traub, W. (1979). An office communications system. IBM Systems Journal, 18, No. 3.
4. Gould, J.D., Alfaro, L., Barnes, V., Finn, R., Grischowsky, N., Minuto, A. & Salaun, J. (1987). Reading is slower from CRT displays than from paper: Attempts to isolate a single variable explanation. Human Factors, 29, 269-299.
5. Gutwin, C., Greenberg, S., & Roseman, M. (1996). Workspace awareness support with radar views. Conference Companion to CHI 96, 210-211.
6. Hardock, G. (1991). Design issues for line driven text editing/annotation systems. Graphics Interface'91.
7. Harper, R.H.R. (forthcoming). The IMF within: An ethnography of documents, technology and organisational action. London: Academic Press.
8. Harper, R. H. R., & Sellen, A.J. (1995). Collaborative tools and the practicalities of professional work at the IMF. Proceedings of CHI '95, 122-129.
9. Hirscheim, R.A. (1985). Office automation: A social and organisational perspective. New York: John Wiley and Sons.
10. Luff, P., Heath, C., & Greatbatch, D. (1992). Tasks-in-interaction: Paper and screen based documentation in collaborative activity. Proceedings of CSCW '92, 163-170.
11. Newman, W., & Wellner, P. (1992). A desk supporting computer-based interaction with paper documents. Proceedings of CHI '92, 587-592.
12. O'Hara, K., & Sellen, A.J. (1997). A comparison of reading paper and on-line documents. Proc. of CHI '97.
13. Olson, J.S., Olson, G.M., Storrosten, M., & Carter, M. (1992). Proceedings of CSCW '92, 91-106.
14. Rouncefield, M., Hughes, J. & Rodden, T. (1994) Working with constant interruption: CSCW and the small office. Proceedings of CSCW '94.
15. Sellen, A.J., & Harper, R.H.R. (1996). Can workflow tools support knowledge work? RXRC Tech. Rep.
16. Shapiro, D.Z. Hughes, J.A. Randall, D. & Harper,R. (1991). Visual re-representation of the flight data strip in air traffic control. Proceedings 10th Interdisciplinary Workshop on Informatics and Psychology.
17. Taylor, J. R. & Van Every, E. (1993). The vulnerable fortress: Bureaucratic organisation and management in the Information Age. Toronto: University of Toronto Press.
|
|