Workflow interruption and nurses’ mental workload in electronic health record tasks: An observational study

A prospective observational method approach was employed in this study, and naturalistic observation and questionnaire investigation were conducted. The study setting was a tertiary teaching hospital which provided specialist and sub-specialist care and was affiliated to a medical university). The hospital was divided into four clinical areas: the internal medicine department, surgical medicine department, emergency room and intensive care unit. Nurses involved in EHR tasks in the internal medicine and surgical medicine departments were observed and investigated in our study because the work patterns of the emergency room and intensive care unit were different. The observations were conducted at nurse stations where the desktop computers were located, and usually nurses performed EHR tasks there even though tablets were introduced. The EHR system in this hospital had been installed only one year ago as the previous one was unsuitable for paperless office. Nurses completed the assessment and care plan using a template in the EHR, and could only view documents completed by other healthcare professionals. According to Cohen’s sample size recommendation [30] and using G*Power [31] and the 10 times rule of thumb, a minimum of 98 participants were required for this study.

Non-participatory observation and investigation were conducted from June 1st to October 31st, 2021. A total of 152 nurses (response rate 98.70%) were initially permitted to participate and were observed in all EHR tasks during their shifts (8 h). Day shifts were scheduled from 8:00 A.M. to 3:30 P.M. or 8:00 A.M. to 5:00 P.M., and the night shift was from 5:00 P.M. to 1:00 A.M. Questionnaires including several scales were distributed at the end of the EHR task observation to measure nurses’ mental workload for the task and to explore the potential influencing factors on mental workload and task performance. Initial permission was obtained from various department heads and hospital administrators before the study was conducted. Prior to each observation, individual nurses were informed about the study objectives and procedure, and written consent was obtained.

A sociodemographic questionnaire was designed to collect information on characteristics, including gender, age, marriage status, education, years of clinical practice, professional title, and practice department.

A self-designed observational tool was applied to document EHR tasks (e.g., assessment, care planning and evaluation) and concurrently classify the source and content of each observed interruption event (e.g., interruptions from the environment, patients, nurses’ colleagues, nurses themselves, patients’ family members, doctors, technical malfunctions, information impediments, telephone/equipment alarms, clinical teaching) and reaction (task switching or concurrent multitasking) over time. The self-designed observational tool was firstly drafted based on a three-month field observation and interviews of twelves nurses, and a literature review on studies of nursing workflow interruptions to identify the core elements and classification of interruptions. Expert consultation with ten health care professionals for content revision and validity was then conducted. A twelves-nurse pilot observation was carried out to test the revised version of the observational tool and then developed the final version (Appendix Observational tool for EHR tasks 1). The total EHR task time was calculated from the beginning to the end of the process (one shift), while medicine administration or other usual care with fixed times was observed and recorded with free text and then eliminated during calculation by observer.

Performance was measured based on the incidence of errors and near errors. Potential error and near errors included (1) the input of incorrect patient data or diagnosis; (2) the use of incorrect nursing terminology; (3) the use of incorrect templates; (4) the unexpected exit from the EHR due to maloperation; and (5) a slow log-in processes. Drawing on the definition of near error in human factors and ergonomics research [32], a near error in EHR task is an unintentional incident in which no real mistake is made though self-checking and revision with prior completed assessment or care plan, but when there is a slight shift in time or position, error may occur easily. An index of performance was calculated, with an error counted as 2 points and a near error counted as 1 point. The total point in one shift represents the nurse’s index of performance.

Mental workload data were obtained using the Chinese version of the National Aeronautics and Space Administration Task Load Index (NASA-TLX) [33]. The NASA-TLX is a well-validated and widely used measure in human factors and ergonomics that comprises six subscales or dimensions regarding different aspects of workload (mental demands, physical demands, temporal demands, performance, effort, and frustration). The Chinese version was translated by Liang L et al. [34]; in this version, the items are rated on a 20-point bipolar scale that ranges from 0 to 100. For five of the six dimensions, i.e., mental demands, physical demands, temporal demands, effort, and frustration, a score of 0 indicates the lowest task load; however, the performance dimension is reverse-scored, with 0 indicating the most successful performance of the task and the highest level of satisfaction with one’s performance. The Cronbach’s ɑ of the total Chinese version of the scale is 0.782 [34]. In this study, we used the total (mean) mental workload score rather than the weighted workload score.

The difficulty of EHR tasks was evaluated by the Chinese version of the Difficulty Index of Clinical Nursing Operation Technique, which was developed by Liyan Liu [35]. The instrument consists of 11 items rated on a 5-point Likert scale, ranging from “lowest” to “highest”. The subscales of this instrument consist of the subordinate concepts of task complexity, operator requirements, operating conditions, task intensity and operational risk.

The usability of EHR system was measured using a system usability scale originally created by John Brooke in 1986, which is a widely used self-report scale regarding to technique products including hardware, software, mobile devices, websites and applications, ranging from 1 (strongly disagree) to 5 (strongly agree). Five statements are positively formulated (items with odd numbers), and 5 statements are negatively formulated (items with even numbers) [36].

Professional competency was measured using the Chinese version of the Competency Inventory for Registered Nurses developed by Ming Liu et al. [37]. The subscales of the instrument consist of the subordinate concepts of critical thinking and research aptitude, clinical care, leadership, interpersonal relationships, legal/ethical practice, professional development, and teaching coaching. The scale has 58 items, and the responses are rated on a 5-point Likert scale, with 4 representing the highest level of the concept. At the time of the scale’s development, the total scale internal consistency of Cronbach’s ɑ was 0.91, and the Cronbach’s ɑ coefficients of the dimensions ranged from 0.77 to 0.87. In this study, the subscales of critical thinking and research aptitude and clinical care, which include 18 items, were employed.

Self-efficacy was evaluated with the Chinese version of the General Self-Efficacy Scale [38], which comprises 10 items rated on a 4-point Likert scale, ranging from “completely incorrect” to “completely correct”. The total score of the scale ranges from 10 to 40, with higher scores indicating higher confidence. The Cronbach’s ɑ was 0.883 in the present study [39].

Nurses’ emotions during the EHR task were measured with the Chinese version of the Positive and Negative Affect Scale [40], a self-reported instrument comprising positive and negative domains with 10 items each. Responses are rated on a 5-point Likert scale ranging from 1 (very slightly or not at all) to 5 (extremely), with higher scores indicating a higher level of either positive or negative affect. The Cronbach’s ɑ coefficients were 0.93 and 0.90 for the positive and negative domains, respectively [41].

The psychometric properties of the measurement tools have been described above. Moreover, the observer (first author) was a registered nurse who had 6 years of clinical practice experience and was a senior lecturer at a medical university. To ensure familiarity with the EHR tasks, the observer was familiar with each patient’s courses of disease, diagnosis, treatment, nursing care and prognosis. Additionally, to increase sensitivity to task errors, the observer reviewed 234 EHRs randomly selected from the last quarter in this hospital under the guidance of staff of the quality control department.

The counts of observed interruptions were calculated as rates per shift of observation overall. For each interruption, the source and the nurses’ reaction (task switching or concurrent multitasking) over a certain time period were calculated in terms of the frequency. Statistical analysis was carried out with SPSS 21.0 software, and p < 0.05 (two-tailed) was considered statistically significant. Descriptive statistics, including the number (n), percentage (%), mean and standard deviation (SD), were used to analyse the demographics, interruptions, task time, task difficulty, system usability, professional experience (title and competency), self-efficacy, mental workload, and task performance (error and near error). Tests for normality and equal variance for all the study variables were performed using Shapiro–Wilk’s and Bartlett tests. Pearson's correlation was applied to evaluate correlations between variables according to the hypothetical model. Path analysis was used with the maximum-likelihood method implemented in SPSS Amos v23.0 software (IBM Corp) to examine the hypotheses [42]. The fit indices were examined to determine the appropriated model, including χ², root mean square error of approximation (RMSEA), goodness-of-fit index (GFI), adjusted goodness-of-fit index (AGFI), comparative fit index (CFI), Tucker-Lewis index (TLI), normed fit index (NFI) and incremental fit index (IFI). The path coefficient estimated the intensity between two study variables and was analysed using a standardized regression coefficient (β weight).

In our study, the total number of nurses involved in EHR tasks in the internal medicine and surgical medicine departments was 198; 44 nurses were absent during the period of our investigation for anti-epidemic tasks outside of the hospital. Therefore, the targeted sample was 154. With consent of the participants, 152 observation shifts with 152 nurses were conducted, and complete data were collected for 145 of the shifts, of which 32.41% were night shifts. Overall, 138 (95.17%) of the observed nurses were female, 48.28% ranged in age from 20–30 years, and 31.72% had 6–10 years of clinical practice. The demographic characteristics of the participants are shown in Table 1.

Among the 145 participants, the total mean score of the NASA-TLX was 44.57 (SD 14.08). The task performance index score was 1.09 (SD 1.31). The task difficulty index score was 33.30 (SD 6.27). The system usability score was 69.23 (SD 17.65). The professional experience score was 56.94 (SD 13.71). The self-efficacy score was 29.92 (SD 5.45). The positive effect score was 27.10 (SD 7.31), and the negative affect score was 15.65 (SD 3.81). The minimum, maximum and mean scores and standard deviations for the variables are shown in Table 3.

A path analysis approach was employed to test the hypotheses according to the recommendations by Preacher and Hayes [42]. The overall effect of interruptions, mental workload and task performance was analysed according to hypothesis 1 (Model 1 in Tables 5 and 6). The results suggested that the EHR task time (β = -0.46, p < 0.001) had a significant impact on nurses’ mental workload. In addition, the mediating effect of self-efficacy on the pathway between professional experience (professional title or competency) and mental workload was tested according to hypothesis 2, however, the pathway between self-efficacy and mental workload was not statistically significant (β = -0.127, p = 0.125). We have modified the model 2 with variables of professional experience (professional title or competency), task difficulty, mental workload and task performance according the initial hypothesis 2 and 5 (Model 2 in Tables 5 and 6). Path analysis showed that task difficulty mediates the effect of professional title and mental workload, and professional title could influence task performance directly (β = -0.224, p < 0.001). Then, the relationships between negative affect and mental workload (β = 0.413, p < 0.001), and the relationship between task performance and negative affect (β = 0.170, p = 0.042) were confirmed (Model 3 in Table 5 and 6). Furthermore, the relationships between system usability and task difficulty (β = -0.397, p < 0.001) and mental workload (β = 0.170, p = 0.044) were confirmed (Model 4 in Table 5 and 6). Because of the above findings, the best fitting model including the variables professional title, task switching incidence, multitasking incidence, task time, task difficulty, system usability, negative affect, mental workload, and task performance were confirmed (Fig. 1). The fit indices of the model were χ² = 30.168, df = 25, p = 0.218, RMSEA = 0.038, GFI = 0.924, AGFI = 0.987, CFI = 0.982, TLI = 0.974, NFI = 0.906, and IFI = 0.983 (Model A in Table 5 and 6).

This study identified three important findings. First, to our knowledge, this is the first study to provide a description of a large number of EHR task interruptions. The findings of this study showed that the magnitude of interruption in EHR tasks was related to the incidence of errors and near errors; fortunately, 68.35% of errors could be corrected by nurses themselves. Second, the study highlighted the two major reactions of nurses facing an EHR task interruption, namely, task switching and concurrent multitasking, of which task switching was related to task performance. Third, this study employed a mental workload theoretical framework to explore the multilevel factors associated with nurses’ mental workload and task performance, that is, interruptions, task time, nurses’ working experience, EHR usability, task difficulty and negative affect, providing a comprehensive understanding of the cognitive process in EHR tasks in dynamic clinical settings.

In our study, the average frequency of interruptions was 14.03 times per hour, while relevant studies have reported an average of 8.13 interruptions per hour in medication administration [43] and an average of 5.40 to 10.90 interruptions per hour in nursing care in emergency departments [6, 44]. Although there is a lack of studies on interruptions in EHR tasks, the interruption incidence found for EHR tasks in this study was much higher than that of other nursing procedures or general daily nursing care. This is explainable, as EHR tasks are considered the lowest priority compared with patients’ requirements and medical orders [2], which is why we observed a proportion of nurses performing EHR tasks in their off-hours. Our finding calls for attention to the high incidence of interruptions, as longer task time caused by interruptions contributes to higher mental workload and thus to increased task errors in our study.

In this study, we confirmed that task switching is far more frequent than concurrent multitasking in EHR tasks. Task switching occurs when nurses face human interaction, such as treatment requirements from patients and interpersonal communication, or when the interruption is a task that requires a larger amount of time (with an average duration of 1.53 min); on the other hand, multitasking tends to occur with shorter tasks with an average time of 0.92 min and is more likely to occur when nurses engage in simple communication. As the path analysis model indicates a linkage from concurrent multitasking and task switching to task time, the incidence and duration of the interruption seems to be one of the major causes for mental workload. This could be explained by the fact that when the nurses are interrupted in their task, their attention is shifted from the initial process to the new process with a decay of the memory of the initial process; this leads to an increase in response time when returning to the initial task or even the nurse starting the initial task over and hence a likelihood of overwhelmed cognitive capacities [14] and decreased productivity and accuracy in the EHR task [45, 46].

Apart from interruptions, we also measured other factors, including task difficulty, system usability, professional experience and competency, and self-efficacy according to the human factors and ergonomics perspective of mental workload [13], to describe cognitive processes and explore the predictors of mental workload and performance in EHR tasks. The total mean mental workload score in EHR tasks was 46.21 (SD 14.76), indicating a medium level of mental workload. Regarding objective workload, the net duration of EHR task time was an average of 32.56 min per shift, which was close to the restriction (30 min) on shift task hours by the National Health Commission of the People's Republic of China [47]. However, the average total time spent on EHR tasks was 84.69 (SD 56.68) minutes per shift due to the high number of interruption events, which indicates that interruptions could influence task efficiency, leading to a higher level of temporal demands. In addition, the task difficulty index (33.30 SD 6.27) reflected a moderate to high complexity and difficulty of EHR tasks, contributing to a higher self-reported level of cognitive demands. Hence, it is reasonable to suggest that task difficulty is a predictor of mental workload, and this result is consistent with other studies [48, 49].

In our study, the path analysis indicated that task difficulty is partially affected by professional title. This finding may be due to the role of nurses in the study hospital, with those with higher professional titles tending to be in charge of complicated cases. In addition, professional title is a protective factor for task performance in the path analysis model. In this study, professional experience was reflected by two variables: professional title and competency. We did not verify that professional competency was associated with mental workload or task performance. It is reasonable that objective indicators would be far more accurate than subjective indicators to reflect real professional experience because participants may exaggerate their competency in a self-report scale. One reason why professional title was related to task performance but not to mental workload might be that experienced nurses will ask for help from colleagues when necessary, thus reducing their error and mental workload, while novices may not realize their mistakes; thus, errors occurred without the elevation of mental workload [21]. Coincidentally, the observation showed that concurrent multitasking events generally involved interactions with colleagues, among which most were problem-solving interactions. Hence, interruptive events may also be beneficial and necessary for the quality of the task [4].

The usability of EHR system is a major concern in modern nursing care, and suboptimal usability has been proven to be associated with clinician burnout and patient safety events [26]. In our study, the total mean system usability score was 56.94 (SD 13.71), indicating a medium level of reported usability. Even though the investigated hospital employs one of the most intelligent EHR systems in China, which allows offices to be paperless, information overload and information conflict were the major complaints in the post-observation conversations with participants. This result seems to be in line with findings in the study from Beasley and colleagues [50]. In our study, system usability was associated with mental workload directly or mediated by task difficulty, which is consistent with the finding in a simulated scenario test and could be explained by the fact that suboptimal EHR usability is associated with the elevation of task effort [26]. Therefore, organizations should develop explicit policies and procedures for enhancing EHR usability.

Moreover, the results of this study showed that task performance and mental workload interacted with each other, which is consistent with ergonomic studies [51]. Given the effect of mental workload on performance, insufficient stimulation is known to lead to underload, boredom, and decreased performance [52]. Conversely, overload is also known to decrease task performance [53]. Therefore, the relationship between mental workload and performance is curvilinear, while in this study, it was linear. This might be because the range in mental workload in our study was approximately moderate to high, which reflects only half of the curvilinear model. Another possible reason is that this level of mental workload might be appropriate for good working performance, which is well illustrated in the mental workload model of Hancock and Chignell [51, 52]. Regarding the effect of performance on mental workload [54], perceived performance, especially errors, influences mental workload by requiring more information processing resources and elevating emotional workload [55]. Hence, the model presents a trend in that the more errors that occur, the higher the negative affect will be.

In our study, the total mean self-efficacy score for EHR tasks was 29.92 (SD 5.45), and the range was at a high level, meaning that the majority of participants believed they were confident and competent in performing EHR tasks. However, this variable was not included in the path analysis model. According to the technology acceptance model [22], which is one of the most common social cognitive theories used in behavioural research on health care professionals, social, individual and contextual factors interact with each other to predict behaviours related to EHR tasks, among which individual factors generally encompass variables such as self-efficacy and professional experience [56]. Hence, the typical variable selection and hypothetical model in our study was reasonable. The exclusion of self-efficacy could indicate that in terms of individual variables, professional experience is far more important for positive EHR task behaviour than self-efficacy. It is obvious that knowledge, skills and situational awareness are essential for quality practice and for developing self-efficacy [6].

To our knowledge, this is the first study showing and describing the magnitude of nursing workflow interruptions in electronic health record tasks. This study employed a mental workload theoretical framework to explore the multilevel factors associated with nurses’ mental workload and task performance in real-world clinical settings. Nevertheless, several limitations of this study should be noted in the interpretation of our findings. First, we conducted a real-world study by employing the core variables from the theoretical framework of mental workload to explore a predictive model for EHR task cognition and behaviour; however, mental workload was statically assessed, which may have involved recall bias and limited inferences concerning the dynamic problems that evolve over time in EHR tasks. Future studies should employ objective and longitudinal evaluation tools to reflect multilevel variables and provide a coherent picture of flexible adaptation to dynamic care settings. In addition, the technique of non-participatory observation may have modified participants’ behaviour under the data collection conditions even though the observer established a good cooperative relationship with participants. Since the observer acted as a research tool, fatigue due to the long observation duration may have led to the neglect of some important clinical information. Although the observer was knowledgeable and well trained in EHR tasks, real-time error recognition is limited, especially in this kind of environment with a fast working pace.

Based on the study findings, reducing harmful interruptions to decrease task time can avoid negative outcomes. Nursing leaders should comprehensively understand the characteristics of interruptions in EHR tasks, which are contextual, multifarious, complex and dynamic, to develop targeted interventions. Training nurses to cope with interruptions and mitigate the impact of interruptions has the potential to decrease nurses’ mental workload and improve task performance. As perceived task difficulty is one of the key variables influencing nurses’ mental workload, which shows differences in work experiences, and EHR usability has been found to induce task difficulty and mental workload, sufficient training related to EHR implementation and task operation would be particularly important in nursing management. Moreover, improving system usability is beneficial to nurses to reduce mental workload and increase the efficiency and accuracy of EHR tasks.