windows, as well as incoming instant messaging and email alerts. We describe ... disruption and resumption tracking tool
Disruption and Recovery of Computing Tasks: Field Study, Analysis, and Directions Shamsi T. Iqbal Department of Computer Science University of Illinois Urbana, IL 61801 U.S.A
[email protected] ABSTRACT
We report on a field study of the multitasking behavior of computer users focused on the suspension and resumption of tasks. Data was collected with a tool that logged users’ interactions with software applications and their associated windows, as well as incoming instant messaging and email alerts. We describe methods, summarize results, and discuss design guidelines suggested by the findings. Author Keywords
Interruption, Attention, Task Switching, Notifications. ACM Classification Keywords
H.5.2 [Information Interfaces and Presentation]: User Interfaces. INTRODUCTION
Task switching is common in computing. Several decades ago, Bannon et al.[3] noted that computer users often switched among multiple active tasks. The diversity and numbers of applications supported by personal computers has grown since the Bannon study and multitasking has now become a salient feature of modern computing. Today, computer users often run programs simultaneously to support multiple tasks, including word processing, financial analysis, searching, browsing, and communications. Card and Henderson [4] attempted to characterize useful attributes of designs for computer-based task management, stressing the need to allow for efficient task switching and resumption, and to provide methods for assisting with refreshing a task context. Today’s major operating systems include tools in line with these recommendations, such as providing multiple means for switching among tasks. However, efficient shifting ability does not mean that a suspended task will be resumed efficiently. Multiple active Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. CHI 2007, April 28–May 3, 2007, San Jose, California, USA. Copyright 2007 ACM 978-1-59593-593-9/07/0004...$5.00.
Eric Horvitz Microsoft Research One Microsoft Way Redmond, WA 98052 U.S.A
[email protected] computing tasks and the opportunity to spawn new tasks compete with returns to specific tasks, interfering with the resumption of tasks following their disruption. The timing of shifts among related and disjoint computing tasks is often self-directed, occurring in the absence of explicit external influences. However, task switching may be affected by external signals and events [9]. Such influences include alerts delivered to computer users from applications that are not at the focus of a user’s attention. For example, a computer user may be drawn to switch from a spreadsheet program to their email application after hearing or seeing an alert about incoming email or receiving an instant message. We sought to characterize task suspension and recovery among information workers in the course of their normal daily computing tasks. We developed and deployed a disruption and resumption tracking tool to monitor the use of software applications and associated windows at the focus of computer users’ activities, as well as to log incoming instant messaging and email alerts. Rather than seek only to measure the specific effect of an alert on a task at focus, we also pursued patterns and understanding of user behavior before and after interruptions. We have particularly worked to understand the chain of diversions whether likely caused by an alert or by a self interruption, and the path and timing back to the resumption of tasks. The work includes an analysis of behaviors of users before they suspend tasks, and to examine behaviors that would suggest a preparation for more efficient resumption of a task upon return. We also sought to better understand the relationships between actions prior to the suspension and time taken to resume suspended tasks, and factors that promote returns to suspended applications. We first review related work. Then, we review the methods that we used to study task interruption, diversion, and resumption in real-world computing situations. We summarize results of analyses of the logged activity and of interviews of subjects. Finally, we provide a set of design guidelines based on the lessons gleaned from the data and from interviews of participants. RELATED WORK
We first motivate our work by providing some background on several studies of interruption and recovery.
Interruptions and their disruptive effects
Maintaining information awareness and near instant communication in the workplace is becoming increasingly important to knowledge workers to support collaborative practices and increase productivity [6, 10, 11]. However, the pursuit of awareness and rapid communication often injects increased numbers of notifications and potential interruptions into ongoing tasks [9, 15, 24]. Several research teams have explored interruptions of computing tasks. Recent efforts come in the context of a rich history of research in cognitive psychology on the influence interruptions on human memory and planning, going back to the protean efforts of Zeigarnik and Ovsiankina [25, 32]. Czerwinski, Cutrell, and Horvitz in a series of studies have explored the effects of external interruptions on task switching behavior and performance, and have also investigated the impact of varying timing and type of interruption [5, 6, 9]. Iqbal and Bailey have shown that interruptions during periods of higher mental workload cause users to take longer to resume their suspended tasks and have larger negative affect [19]. Mark et al. have sought to understand the influence of interruptions on task switching and found that users frequently switch between tasks and 57% of their activities are interrupted [22]. Others have also investigated effects of interruption on error rates [21], decision making [28] and affective state such as frustration, annoyance and anxiety [2, 33]. Researchers have also investigated methods that could decrease the cost associated with communication alerts. Approaches explored to date include reducing the frequency and costs of interruptions through identifying the attentional state of users [13, 16, 18] and deferring or scheduling notifications in an intelligent, strategic manner [13, 20, 23], and providing support for recovering from interruption [27, 29]. Recovering from interruptions
Research has shown that inopportune interruptions can increase task performance time, primarily due to increases in the time to resume suspended tasks [1, 19, 20]. Cognitive models suggest that when the workload of the ongoing task is high, interruptions cause users to divert cognitive resources to the interrupting task [31]. On return from the interrupting task, users have to reallocate resources to the suspended task, which becomes increasingly difficult if the resource demands were high to begin with. The result is higher resumption lag, which affects recovery. With users typically suspending sets of applications [9, 22], recovery is often confounded with a cycling through and visiting of multiple suspended applications on the way to resuming a task. Designs for recovering from interruptions
Software tools have been developed across a variety of application domains with the goal of supporting ease of resuming suspended applications after responding to an interruption [12, 14, 26, 27]. These tools provide visualizations of suspended application states and group
applications based on time proximity, leveraging the use of visual cues, or interfaces for interactive dialogue to help users quickly regain the suspended task context. We believe that the challenge is not only one of resumption of the suspended application but also one of turning one’s focus of attention to the suspended tasks, given other tasks competing for the user’s attention. Beyond characterization of the suspension and resumption behavior, we seek to determine (i) how we might best help people to break away from potentially costly ‘chains of diversion’ following suspension, so as to return to suspended tasks within a time they would desire and (ii) how to help them quickly resume where they left off, once they return to continue on a task. OVERVIEW OF STUDY
We conducted a field study to better understand task suspension and resumption in practice. We were interested in the influences of computer-based alerts on users’ task execution behaviors. Specifically, we sought to explore effects of interruption on task switching and the path and timing back to the resumption of suspended primary tasks. By primary tasks we refer to normal daily tasks that users perform as their primary responsibility while in the computing environment. For our study population, this typically entailed programming or content generation tasks, e.g., document editing or creation of presentation material. By alerts, we refer to notification cues generated by email clients and instant messaging applications. In the simplest case, an alert influences the probability that a user will switch to the alerting application with a concomitant suspension of the ongoing primary task and, some time later, will resume the primary task after responding to the alert. However, when users suspend a task because of an alert or for other reasons, they may take advantage of the break in the execution of the primary task offered by the switch to interact with other peripheral applications, and perhaps turn to other tasks. We sought to gain a deeper understanding of how users prepare for the context switch from the primary task to the alert response, how a succession of diversions after a task switch may interfere with a return to their primary tasks, and how they eventually pass through a chain of diversions on the way to resuming a suspended primary task. More specifically, we explored the following hypotheses: H1: Users prepare to address alerts in their regular task execution by stabilizing their current task state before switching to the alerting application. H2: Users are less focused on applications visited during the ‘chain of diversion’ and during resumption. H3: The chain of diversion mostly consists of rapid interactions with communication and awareness applications. H4: Availability of cues about suspended tasks assists with resumption of tasks.
H5: Users have difficulty with resuming interrupted desktop computing tasks. H6: The time to resume a primary task is influenced by the recency and focus of attention on a task before suspension. Our study was designed to gather evidence from users in situ to investigate these hypotheses as well as gain a basic understanding of the prevalence of alerts in practice and the length of time users typically spend on chains of diversions initiated by these alerts. We began with defining a task disruption and resumption lifecycle. Each phase in the cycle signifies a distinct user goal along the path of suspending and returning to an interrupted task. We then defined a set of task state attributes to characterize behaviors across these different phases. We developed a disruption and recovery logging tool by extending an existing user-activity monitoring system. The tool was deployed to log data from users over a period of two weeks. The collected data was analyzed and findings were corroborated through interviews of the study participants. Finally, the findings from the study were distilled into key results and a set of design guidelines for enhancing the recovery of suspended tasks. PHASES OF AN INTERRUPTION LIFECYCLE
We divide the time following an alert into distinct temporal segments or phases. Our intent is to measure the impact of the interruption by comparing behavioral changes across these phases as users sequentially move through a cycle including focused attention on a primary task, alert arrival, response and diversion, return from diversion and the resumption of original task. A related categorization of aspects of interruption is provided in [14]. One of our key goals was to better understand natural user behavior during each phase so as to inform the design of tools that might assist computer users with multitasking. Figure 1 displays phases of the interruption lifecycle. We define the initial phase of the interruption lifecycle, which we call the preparation phase, as the time between an alert and the concomitant suspension of ongoing tasks. Based on prior research showing response time to be a function of task state [13], we hypothesize that during this phase, the user may consciously or subconsciously perform activities that leave the primary task in a more stable state, before switching to the alerting application. Phase 2 is the diversion phase, defined as the time between the switch from the primary task to respond to the alert and the return to the primary task after the response. During this period the goal is to access the interrupting application but users may also explore other peripheral applications. Phase 3 is the resumption phase, where the user finishes interactions with interrupting and peripheral applications and seeks to a return of conceptual context and focus to become active once again in the primary task. Since it is difficult to identify exactly when the resumption phase may begin, we used cues indicating user intent to terminate the diversion and resume suspended work, e.g,. minimizing or
Figure 1. Phases of the interruption lifecycle. a) User begins interaction with two applications on a primary task, continuing through a pre-interruption phase; b) alert arrives and user enters a response preparation phase; c) user suspends primary task and switches to interrupting application, and may become diverted to other peripheral applications; d) user returns to resume primary task.
closing applications accessed in the diversion phase and starting to resume applications from the suspended group. As users can be active in a task in the absence of computing activity (e.g., reading text), we used a simple heuristic to determine resumption: we considered users to resume a suspended task if they had spent more than 15 seconds on the suspended application, which is more time than required for rapid application switches, e.g., tabbed browsing. To compare users’ actions in the aforementioned phases to behaviors seen during task execution behavior, we defined an additional phase, pre-interruption, which refers to a predefined time segment of activity before the arrival of an alert. Behaviors during this period provide a baseline for comparison to the same activities in the other phases, thereby providing an opportunity to demonstrate the influences of interruptions on users’ task execution. FORMULATING TASK ATTRIBUTES
With an eye to investigating our hypotheses, we defined a set of events that promised to allow us to probe sets of relevant activities during the different phases of the interruption lifecycle. The definition of events was guided by intuitions about behavioral patterns we expected to see at different phases. For example, we were interested to see if users tended to perform activities during the preparation phase that provided evidence that they were attempting to leave the ongoing task in state that could be resumed with more efficiency (H1). We defined a set of events that promised to provide insights about such potential efforts. These include the number of save operations, the completion of edit operations (sentence or paragraph
completion), and the completion of pending tasks lacking physical representation (pasting content copied into a memory clipboard). Also, since alerts originate largely from communication applications, we wished to explore whether suspension of primary tasks was associated with users interacting with communication-centric applications, beyond switching to the alerting application, during the diversion phase (H3). To gather evidence on the diversion phase, common email interactions, including mail opens, writes, and sends, were monitored To study the potential influence of visual cues on the timing of transitions from the diversion to the resumption phase, we developed attributes for capturing the visibility of open windows. Such monitoring promised to allow us to examine the effects of cues associated with suspended task windows on the time to return to those tasks (H4). We sought to better understand the difficulty that people had with resuming applications that has been suspended as a result of responding to an alert (H5). We decided to quantify difficulty in terms of the time taken by users to not only return to the suspended application after responding to an alert, but also to restore context and state, and to resume the activity they were engaged in before switching tasks in response to the alert. We were also interested in exploring how the intensity of the focus of attention on a task and task recency influenced the time until task resumption after an interruption. We created attributes capturing the duration of time and the last time that users were focused on a particular task, with a goal of exploring the relationship between these coarse measures of focus and salience to the time until a user would completely return to a task (H6). As a related task state attribute, we defined the rate of task switches at each phase - a potential indicator of user focus. We hypothesized that users would switch tasks at a higher rate as they sought to return to suspended applications, especially if they were searching for a window associated with a primary task application among multiple open windows, as part of an attempt to regain task context (H2). A DISRUPTION AND RESUMPTION TRACKING TOOL We developed a monitoring tool named DART (for
Disruption and Recovery Tracker). The tool was developed on top of the Eve event-monitoring infrastructure, a constellation of user and system monitoring components that have been evolving for over a decade at Microsoft Research [17]. Eve components have been employed in prior research on interruptions in computing settings, including efforts on inferring the cost of interruption and on guiding alerts and information awareness based on costbenefit analyses [16, 18]. DART runs as a background process, and continues to logs the name, size, and location of all windows on a computing system, noting the opening and closing of windows. The system also logs user activities, including when users are actively engaged with the software, keyboard and mouse activity, and switches
among windows as well as actions of saving, cutting, and pasting. To protect privacy, window titles were truncated, and only a subset of keyboard events were recorded, including the input of periods and carriage returns (which can indicate sentence or paragraph completion and data entry), and shortcuts for saving, cutting, and pasting. The tool also logs alerts from email and IM systems. DART logged user actions relevant to the aforementioned
task state attributes and corresponding timestamps. Logged data files were periodically flushed to a server, where the data was preprocessed to generate the attributes and stored in a SQL database for further analysis. DEPLOYMENT OF TOOL AND COLLECTION OF DATA We deployed DART on the primary machines of 27 people
at our organization, whose job descriptions ranged from program manager, administrator, and researcher to software developer. On recruiting subjects, we sought a balance of people who focused on different kinds of tasks as primary, including software development, working on productivity applications, and/or managing large numbers of external communications. We did not screen users for whether they used alerts in communications, but excluded from analysis those subjects who did not have alerts enabled. We collected 2,267 hours of activity data over a period of 2 weeks, resulting in 974 sessions (M(session length)=2h, 17m, S.D= 410.37 m). A session was defined as delimited by either the logging on and off or by the unlocking and locking of a machine. Collected data included logs of application access, window sizes and configurations, file retrieval and archival, percentage visibility of open application windows, key events corresponding to content manipulation, e.g. cut, copy and paste, file open and save, and completion of text generation. Users were informed a priori about the overall nature of the data that was to be collected and informed that they would be able to quit the study at any point if they were not comfortable or if the software was perceived to influence the performance of their computers. Users were not informed that the study was investigating disruption and resumption of tasks. As in any field study, the knowledge of being studied potentially may have had influences on the behavior of the subjects. However, we believe the study had little influence on participants. A number of the subjects mentioned during interviews that they had forgotten about the tool running in the background. ANALYSIS AND RESULTS
We focused on characterizing the behaviors of users in response to alerts generated by Outlook, a widely used email client, and IM clients, including Windows Messenger, MSN Messenger, and Office Communicator. It is difficult in the general case to determine with certainty whether an interruption of a current task is a direct consequence of an alert or if a switch results from users making a decision to switch away from a task largely
independent of recent notifications. We employed a simple heuristic to identify suspensions likely to have been driven by alerts: switches to a notifying application (Outlook or IM) occurring within 15 seconds of the alert were considered as being caused by that alert. Our later interviews with users further raised our confidence about the robustness of this heuristic. Our analysis showed that for such switches, users take on average 2.35 seconds (S.D =1.39s) to switch to Outlook and 1.72 seconds (S.D.=1.1s) to switch to the IM client. We distinguish between immediate and delayed responses to alerts in our presentation of the results so as to explore differences in activities during the preparation phase. As user actions for email and IM alerts could vary based on such influences as social conventions and expectations, we analyze email and IM results separately. We also examine results across developers, researchers and managers and note if a significant influence of job role is found. Rather than measure effects over the entire pre-interruption phase, as the baseline condition we consider 5 minutes of activity prior to the interruption. Preliminary analysis showed that, on average, the maximum time spent on an application before switching to another is just above 4 minutes, with an average of below a minute. Distribution of alerts
Overall, we found that, on an hourly basis, a user’s primary tasks were interrupted by an average of 4.28 email (S.D.=5.56) alerts and 3.21 IM (S.D.=4.31) alerts, with an overall average rate of 3.74/hour (S.D.=4.94). For IM alerts, our system did not discriminate between conversational pings, and sign-in and presence status alerts. We are more interested in attempts to initiate or continue conversation as these pings pose a social obligation to respond; nonetheless, sign-in alerts also affect awareness of the user and may serve as a subtle trigger to self-interruption, e.g., if the user wishes to communicate with the person who just signed in. Job roles did not significantly affect the number of alerts. Time to respond to alerts
After an alert was delivered, users took, on average, 4 minutes, 59 seconds (S.D.=8m, 43s) to suspend their primary task and switch to switch to Outlook, 7 minutes, 54 seconds (S.D.=16m, 50s) to switch to MSN messenger, 7 minutes, 13 seconds (S.D.=13m, 49s) to switch to Communicator, and 34 seconds (S.D.=1m, 5s) to switch to Windows Messenger. There were no significant differences in the response times of the different alerts, nor were there any significant effect of job roles on the response times. Immediate and Delayed Responses to Alerts
For 40.8% (2344/5747) of the email alerts, users responded immediately (
