Workload during a complex ICU monitoring task: the effect of level of patient’s sedation level and repetion of the task

Authors: Aslanidis Th. MD¹ , Chatzis A. CSE², Kontos A. MD¹, Grosomanidis V. MD, PhD¹, Karakoulas K. MD, PhD¹, Chatzisotiriou A. MD, PhD³

¹Department of Anesthesiology and Intensive Care Medicine, AHEPA General University Hospital, Thessaloniki, Greece.
²Student, Department of Computer Science and Engineering, University of Ioannina, Greece.
³Laboratory of Physiology, Medical School, Aristotle University of Thessaloniki, Thessaloniki, Greece.

ABSTRACT

Patient care in Intensive Care Units is characterized by high demanding tasks, which leads in daily high workload. The aim of the study is to evaluate the effect of patient’s sedation level to workload for the certain task. It also examines whether workload lowers over time, as an effect of the experience gained by the repetition of the task. NASA- TLX tool was used as workload assessments method during a complex monitoring task in an adult Intensive Care Unit environment. The latter included monitoring and recording of skin conductance variability, noise level, hemodynamic and respiratory parameters were monitored during 4 hour routine in two groups of patients. The group was defined by the sedation level (Ramsay sedation score); otherwise no major differences were spotted in their characteristics. Both raw and weighted data of the NASA-TLX tool were included in the analysis, which was performed with MS Excel 2007 (Microsoft Co, USA) and Rstudio® IDE v.0.99.903 (Rstudio Inc, Boston, MA, USA). Patients’ sedation level did not affect NASA-TLX measured workload. The former was valid both for raw values and weighted data of the subscales of the NASA-TLX tool.  In the second part of the analysis where the raw values were treated as time series data, it was shown that some subscales (Ment, Phys) had a tendency towards lower values, others (e.g. Temp, Ef) had a relative stability and others  (Per) increased over time. The total workload (OW) did not seem to lower over time. While the patient’s sedation level does not affect workload of the specific task, several subscales of the NASA-TLX index do reveal a tendency over time; a fact that may be used as learning curve/ experience assessment for a given task. However, further studies are needed in order to define its future utility.

INTRODUCTION

Intensive Care Unit (ICU) is very complex environment, where the continuous integration of both technological and medical progress on one hand, and the dynamic character of the clinical condition presented in each ICU case on the other, poses great challenges to its staff. Patient care is characterized by high demanding tasks, which leads in daily high workload^1-2. The latter has been identified as a major occupational stressor and has been related to several adverse effects, for ICU staff as well as for their patients^3-5.

Mental workload monitoring is identified early as the key point in order to assure higher levels of comfort, satisfaction, efficiency, and safety in this workplace⁶.

Several tools have been developed for this purpose and there is a trend of creation of more oriented indices^7-8. Most of these methods fall into the three following categories (a) performance-based measures, (b) subjective measures, and (c) physiological measures. The practical advantages of subjective procedures include their ease of implementation, non-intrusiveness and their capability to provide sensitive measures of operator load^{7, 9-10}. Therefore, they are the more often used in the literature.

NASA Task Load Index (TLX) is a subjective workload assessment tool to allow users to perform subjective workload assessments on operator(s) working with various human-machine interface systems. It derives an overall workload (OW) score based on a weighted average of ratings on six subscales: a) Mental Demand (Ment), i.e. how much mental and perceptual activity was required? Was the task easy or demanding, simple or complex? b) Physical Demand (Phys), i.e. how much physical activity was required? Was the task easy or demanding, slack or strenuous? c) Temporal Demand (Temp), i.e. how much time pressure did you feel due to the pace at which the tasks or task elements occurred? Was the pace slow or rapid? d) Performance (Per), i.e. How successful were you in performing the task? How satisfied were you with your performance? e) Effort (Ef), i.e. How hard did you have to work (mentally and physically) to accomplish your level of performance? and f) Frustration (Fr). How irritated,stressed, and annoyed versus content, relaxed, and complacent did you feel during the task?

Coincidentally, these dimensions also correspond to various theories that equate workload with the magnitude of the demands imposed on the operator or the operator’s ability to meet those demands¹¹.Originally developed as a paper and pencil questionnaire, it is currently used as computerized version. In each subscale the score varies between 0-100 (no workload to extreme workload), with 5-point steps. Results can be analyzed both as raw data or weighted scores. The observer evaluates the contribution of each factor (its weight) to the workload of a specific task, thus providing diagnostic information about the nature of the workload imposed by the task.

Several other methods exist for operator-based subjective workload: the Cooper-Harper Scale, the perceived workload scale, the Subjective Workload Assessment Technique (SWAT), the Workload Profile (WP), the Rating Scale Mental Effort (RSME) and the NASA-Task Load Index (NASA-TLX). Yet, literature shows that the latter is a reliable and valid instrument and is actually more reliable and valid than other subjective workload instruments^7,13.

Standardization and repeatability of complex tasks often reduce mental workload. That’s the reason that few studies have used mental workload monitoring as evaluation tool in a given learning curve for a specific task¹².

The aim of the study is to examine the latter theory in a case study of a complex monitoring task in an adult ICU environment and to evaluate the effect of patient’s sedation level to workload for the certain task.

MATERIAL AND METHODS

This prospective observational study was conducted at the adult general ICU, at AHEPA General University Hospital, Thessaloniki, Greece. A total of 25 4h-measurements took place in 16 critically ill adult patients, under sedation. Other inclusion criteria included administered mechanical ventilation>24h and constant sedation level under midazolam or propofol continuous intravenous infusion (c.i.v.). On the contrary, patients with Ramsay sedation score (RSS) 1, diagnosed or with history of hearing problems, psychiatric disorders, neurological diseases, neuro~ or myopathy, delirium, CNS or spinal cord injury, were excluded. Also as exclusion criteria were considered pregnancy, hemodynamic/respiratory instability, edema of the upper limbs (place of measurement) and the presence of sensitive electrical life-sustainable devices such as cardiac pace, renal replacement therapy devices, intra-abdominal aortal counterpulsion pump, extracorporal membrane oxygenation and artificial liver. Measurements were divided into 2 categories according to sedation level: Group A- RSS 2-4 and Group B –RAS 5-6.

The task included monitoring skin conductance (SC) variability, anesthesia depth, noise level, selected hemodynamic and respiratory parameters during 4 hour routine daytime intensive care nursing and treatment.  Events that could influence the above measurements were also recorded (e.g. nursing turning, voice stimulus, drug administration, hand-over, physiotherapy, etc). Demographics and lab data were recorded at the beginning of each task Check and appropriate setting of the monitoring devices was also included. Measurements which were monitored on screen photographed every 60 min. The task demanded continuously (4 hour) presence of the observer.

Med Storm Pain Monitor System (MED Storm® Innovation AS, Oslo, Norway) was used as SC monitor¹³. Three single use Ag/Cl electrodes were attached at the palmar surface of the hand: on the thenar eminence (current), on the hypothenar eminence (measurement) and just below 2^nd and 3^rd digits (reference). In order to minimize artifacts, the hand least likely to move, with no intravenous or intra-arterial lines was chosen. SC was measured by alternating current of 66Hz and an applied voltage of 50mV. SC parameters recorded were: absolute SC (in μS), peaks/sec or number of SC fluctuations per second (NSCF), the average peak (micro Siemens seconds – μSs), the rate of increase or decrease from the start to the end of the measurement window (rise time, in micro Siemens per second – μS/s), area huge peaks (μSs), area small peaks (μSs) and the larger of the two measures (referred as Area under curve- AUC, in μSs). In case of area huge peaks establishing a horizontal base line from the first peak minimum in the time window. The area calculated is the accumulated difference between the conductance values at the registration curve and the established baseline when they are larger than the baseline. The measure of area small peaks is calculated by establishing a line between two adjacent peak minimum points. The area is the accumulated difference between the line and the skin conductance registration curve values when they are larger than the line. Cut off for NSCF counting was >0.005, much more sensitive than the >0.02 μS used in relative pain monitoring literature¹¹. Signal quality <80% was considered artifact and the measurement was also excluded. Event – input was performed manually (via keyboard) by the investigator.  Noise level was measured at distance 30 cm from the head of the patient via Sound Level Meter GM13656 (Shenzhen Jumaoyuan Science & Technology® Co., China)¹⁴.  The rest of the parameters were monitored via Bedside Monitor BSM 9101K and Monitor CNS 9601 (Nihon Kohden® Ltd., Japan). Bispectral index monitor (Covidien®, USA) was also in place. Mechanical ventilation parameters and arterial blood gases were recorded hourly or in case of an “event”.

Observer was a consultant with proven previous experience and thorough knowledge of every device used for the task.

Descriptive statistics are presented as mean ( ),  standard deviation (s).  Shapiro-Francia normality test was conducted for each parameter and then two comparison designs were followed: one that examined possible difference between the 2 groups and one examined workload along time (time series analysis). Data analysis was performed with MS Excel 2007 (Microsoft Co, USA) and Rstudio® IDE v.0.99.903 (Rstudio Inc, Boston, MA, USA).

Results

General characteristic of patients in each group of measurements is illustrated in Table 1. Different averages of APACHE II score, Extended Glasgow Outcome Score (GOSE) and PaO₂/FiO₂ are partially explain the different sedation level.

Descriptive statistics in for the measurements conducted in each group are displayed in Table 2, while boxplots of each subscale for both groups is shown in Graph 1.

 

 

Descriptive statistics of weighted scores for each subscale is displayed in Table 3, while boxplots of each subscale for both groups is shown in Graph 2.

Comparison between the 2 Groups did not reveal any differences (Table 4).

In the second part of the analysis we examined the workload over time. From the raw data we can see that the general overall trend is towards smaller workload, yet not in every subcategory (Graph 3).

In detail, trend extrapolation (linear, exponential.,logarithmic) reveal a tendency for lower workload in Ment, Phys, Per, Ef and Fr subscales ;yet an increasing trend for Pe subscale and for Total workload (OW) (Graph 4). Yet, the coefficient of determination R² for all trends are low to moderate (Table 5).

 

Further on, we treated each subscale raw data as univariate time series. First we used average, naïve and drift forecasting methods for each time series fir extrapolating another 11 measurements into the future.

Then, Exponential Smoothing State method (ets model) was used for forecasting the value of each subscale for the same number (11) of measurements. The model chosen manually for all parameters was ETS (M,N,N), i.e. simple exponential smoothing with multiplicative errors. Yet, an automatic selection of model was also examined (Suppplemental File).

Accuracy of each method with calculation of  Mean Error (ME), Root Mean Squared Error (RMSE), Mean Absolute Error (MAE), Mean Percentage Error (MPE), Mean Absolute Percentage Error (MAPE), Mean Absolute Scaled Error (MASE) and Autocorrelation errors at lag 1 (ACF1) are displayed in Table 6.

 

For Ment, Temp subscales, ETS (M,N,N) seem to be the best method, for the Per, Fr subscale and OW  the  ETS (A,N,N)  and for Phys subscale ETS (A,A,N) method was respectively automatically selected as the best model. Only for Fr subscale the mean method, ETS (M,N,N) and ETS (A,N,N) seem to be equally reliable. Yet, only Phys subscale shows a clear tendency towards lower values over time.

Discussion

Several associations with patient’s condition or ICU environment and workload have been revealed in previous reports. Thus, e.g. higher workload demand was associated in the past with physiological instability (respiratory failure) and multiple severe trauma injuries in male patients¹⁴_. On the contrary, higher nursing workload seems to have a protective role for the development of pressure ulcers¹⁵_. Other studies report that administrative problems, high ratio of patients: nurse and mismatch of the mismatch between the capacity of wards and the number of patients may increase workload¹⁶. The type of the ICU and the shift also affects workload: thus, lower scores are reported during night shifts, in weekends and in Medical ICU patients and higher during morning shifts in Surgical ICU patients^17-18_. In Greece, there are few studies that relate high nursing workload with high mortality¹ and fever in ICU¹⁹_. This the first study that uses NASA TLX index and examines patients’ sedation level and workload. No significant relation was found. Yet, further studies are needed with more investigators and workload scales, either operator-based subjective ones^7,13 or scores measuring activities (e.g. TISS-28, NAS)²⁰_, to reach a more definite conclusion.

In the second part of the analysis where the raw values were treated as time series data, it was shown that some subscales (Ment, Phys) had a tendency towards lower values, others (e.g. Temp, Ef) had a relative stability and others  (Per) increased over time. The total workload (OW) did not seem to lower over time.

Those results can be partially explained by the “familiarization”/adaptation of the investigator to the time frame and the manual tasks needed over time (i.e. Ment and Phys) and the increasing focusing to efficiency (i.e Per).  Previous studies also report that performance did not depend on experience; thus, enforcing the former hypothesis^21-22. There are few studies in the literature about the use of workload as an evaluation method of the learning curve of a specific task^{12, 23-25}. On the contrary, increased mental and physical workload was associated with inferior task performance and higher likelihood of errors²⁶.Physical signs of increased mental workload and frustration during specific task were also described²⁷; and psychological strategies have also been proposed as a measure of reducing mental workload²⁸. The current study does reveal a tendency towards lower mental and physical workload over time (i.e. adaption=experience or step up in the learning curve of the specific task), though the OW stays relatively constant. That suggests that though the use of the NASA-TLX subscales may be useful, the OW is yet to prove its utility.

Future studies with a variety of tasks, observers and methods are needed to define which tool/tools could be used both as workload and experience (learning) evaluation. Suggested objective electrophysiological methods could also be of help in this direction²⁹.

Conclusion

No significant relation was found between workload (as measured by NASA-TLX index) for performing a complex monitoring task in ICU environment, and the patient’s sedation level. Furthermore, several subscales of the NASA-TLX index do reveal a tendency over time; a fact that may be used as learning curve/ experience assessment for a given task. However, further studies are needed in order to define its future utility.

Acknowledgements

The authors wish to thank Dr. Maria-Giannakou Peftoulidou, director of the ICU and Prof. Dimitrio Vasilako, director of the Department of Anesthesiology and Intensive Care Medicine in which the study took place; and the medical and nursing staff of the unit for their assistance.

Ethics

The study is part of a thesis project, approved by AHEPA General University Hospital Research Committee and by No 16/09-07-2013 General Assembly of Special Composition of Medical School, Aristotle University of Thessaloniki (Ref. No.8220/10-07-2013)

REFERENCES                                                                                                                                           

1. Kiekkas P, Sakellaropoulos GC, Brokalaki H, et al. Association between nursing workload and mortality of intensive care unit patients. J Nurs Scholarsh. 2008; 40(4):385-90.
2. Schaufeli W, Le Blanc P. Personnel. In: Miranda DR, Ryan DW, Schaufeli WB, Fidler V, Eds. Organisation and Management of Intensive Care: A Prospective Study in 12 European Countries. Springer; 1998. pp. 169–205.
3. Llenore E, Ogle KR. Nurse-patient communication in the intensive care unit: a review of the literature. Aust Crit Care. 1999; 12(4):142–5.
4. Lysaght RJ, Hill SG, Dick AO, et al. Operator workload: Comprehensive review and evaluation of operator workload methodologies (No ARI Tech Report 851) Fort Bliss, TX: U.S. Army Research Institute; 1989.
5. McManus IC, Keeling A, Paice E. Stress, burnout and doctors’ attitudes to work are determined by personality and learning style: a twelve year longitudinal study of UK medical graduates. BMC Med. 2004;2:
6. Sheridan T. Mental workload-What is it? Why bother with it? Human Factors Society Bulletin. 1980;23(2):1–2.
7. Rubio S, Díaz E, Martín J, et al. Evaluation of Subjective Mental Workload: A Comparison of SWAT, NASA-TLX, and Workload Profile Methods. Applied Psychology: an International Review. 2004; 53(1):61–86.
8. Mark R.W, Jamie M.P, Neha M, et al. Development and Validation of a Surgical Workload Measure: The Surgery Task Load Index (SURG-TLX). World J Surg. 2011; 35(9): 1961–9.
9. Belkić K. The occupational stress index: an approach derived from cognitive ergonomics and brain research for clinical practice. Cambridge (United Kingdom): Cambridge Interna­tional Science Publishing; 2003.
10. Gaillard AWK: Comparing the concepts of mental load and stress. Ergonomics. 1993;36:991–1005.
11. Hart S. NASA TLX workload index: 20 years later. Proc Hum Factors and Ergonomics Soc Ann Meeting. 2008;50 (9): 904–8.
12. Mohamed R, Raman M, Anderson J, et al. Validation of the National Aeronautics and Space Administration Task Load Index as a tool to evaluate-the learning curve for endoscopy training. Can J Gastroenterol Hepatol. 2014;28(3):155-60.
13. Hoonakker P, Carayon P, Gurses A, et al. Measuring workload of icu nurses with a questionnaire survey: the NASA task load index (TLX). IIE Trans Healthc Syst Eng 2011; 1(2): 131–43.
14. Nogueira L de S, Domingues C de A, Poggetti RS, et al. Nursing Workload in Intensive Care Unit Trauma Patients: Analysis of Associated Factors. Salluh JIF, ed. PLoS ONE. 2014;9(11):e112-25.
15. Cremasco MF, Wenzel F, Zanei SS, et al. Pressure ulcers in the intensive care units: the relationship between nursing workload, illness severity and pressure ulcer risk .J Clin Nurs. 2013; (15-16):2183-91.
16. Bahadori M, Ravangard R, Raadabadi M, et al. Factors Affecting Intensive Care Units Nursing Workload. Iranian Red Crescent Med J. 2014;16(8):e20072
17. Debergh DP, Myny D, Van Herzeele I, et al. Measuring the nursing workload per shift in the ICU. Intensive Care Med. 2012; 38(9):1438-44.
18. Kiekkas P, Poulopoulou M, Papahatzi A, et al. Workload of postanaesthesia care unit nurses and intensive careBr J Nurs.2005;14(8):434-8.
19. Kiekkas P, Sakellaropoulos GC, Brokalaki H, et al. Nursing workload associated with fever in the general intensive care Am J Crit Care 2008; 17(6):522-31.
20. Campagner AOM, Garcia PCR, Piva JP. Use of scores to calculate the nursing workload in a pediatric intensive care unit. Rev Bras Ter Intensiva. 2014;26(1):36-43.
21. Schulz CM, Schneider E, Kohlbecher S, H et al. The influence of anaesthetists’ experience on workload, performance and visual attention during simulated critical incidents. J Clin Monit Comput. 2014; 28(5):475-80.
22. Byrne AJ, Murphy A, McIntyre O, et al. The relationship between experience and mental workload in anaesthetic practice: an observational study Anaesthesia 2013;68(12):1266-72.
23. Ruiz-Rabelo JF, Navarro-Rodriguez E, Di-Stasi LL, et al. Validation of the NASA-TLX Score in Ongoing Assessment of Mental Workload During a Laparoscopic Learning Curve in Bariatric Surgery. Obes Surg. 2015; 25(12):2451-6.
24. Schulz CM, Skrzypczak M, Schneider E, et al. Assessment of subjective workload in an anaesthesia simulator environment: reliability and validity. Eur J Anesthesiol 2011;28(7):502-5.
25. Byrne A, Soskova T, Dawkins J, et al. A pilot study of marking accuracy and mental workload as measures of OSCE examiner performance. BMC Med Educ. 2016;16:191.
26. Yurko YY, Scerbo MW, Prabhu AS, et al. Higher mental workload is associated with poorer laparoscopic performance as measured by the NASA-TLX tool. Simul Healthc. 2010;5(5):267-71.
27. Zheng B, Jiang X, Tien G, et al. Workload assessment of surgeons: correlation between NASA TLX and blinks. Surg Endosc. 2012 ;26(10):2746-50.
28. Anton NE, Howley LD, Pimentel M, et al. Effectiveness of a mental skills curriculum to reduce novices’ stress. J Surg Res. 2016;206(1):199-205.
29. Raduntz T. A new method for the objective registration of mental workload. In Halle KS, Stanney KM, Eds. Advances in Neuroergonomics and Cognitive Engineering, , Springer International Publishing Gr, 2017. pp.279-90.

 

Author Disclosures:

Authors Aslanidis Th.,  Chatzis A., Kontos A., Grosomanidis V., Karakoulas K. and Chatzisotiriou A., have no conflicts of interest or financial ties to disclose.

Corresponding author:

Theodoros Aslanidis,

4 Doridos street,
PC 54633, Thessaloniki, Greece,
tel.: +306972477166,
e-mail: