Usability evaluation of obstetrics and gynecology information system using cognitive walkthrough method
Keywords:
Health information systems, Usability testing, Cognitive walkthrough, Obstetrics and gynecologyAbstract
Background and aim: Obstetrics and gynecology information system is a critical component of the HIS in social security organization health centers. The objective of this study was to evaluate the usability of this system using the cognitive walkthrough method. Also, the present study provided a detailed formal description of how the cognitive usability evaluation can be applied and reported for a health care information system. Methods: This study was conducted at the Mashhad University of Medical Sciences' usability lab from March 2016 to June 2017. A two-phase approach was used to conduct the cognitive walkthrough evaluation: preparatory and evaluation. The preparation was done in three stages: first, we investigated users’ capabilities and background knowledge through a semi-structured interview. Second, the evaluation scenario was developed based on the most common tasks in routine workflow of users. Finally, each task was broken down into sequences of actions. In the evaluation phase, three usability experts independently assessed each action using a four-item checklist. Problems were categorized thematically and were reported from three different perspectives: Question-based, Task-based, and Evaluator-based. The data were then analyzed to understand the contribution of each task, along with its mean severity score. Results: Evaluators’ responses were compared and any conflict was resolved in an expert panel. A total of 116 usability problems were identified based on the consensus of the evaluators. Inadequate system feedback was found to be the main source of 43% of the problems, and resulted in users confusion. Conclusion: Since the system was evaluated in its pilot implementation phase, there was an opportunity to prevent future potential usability problems. The use of a mixed quantitative and qualitative approach in this usability study provided a more comprehensive perspective of the system problems. This study provided a detailed description of conducting CW usability evaluation which can be used as a practical guide for future studies.
References
Kohn L, Corrigan J, Donaldson M. To err is human: building a safer health system. National Academy of
Science, Institute of Medicine. 2002. PMID: 25077248.
Borycki E, Kushniruk A, Carvalho C. A methodology for validating safety heuristics using clinical
simulations: identifying and preventing possible technology-induced errors related to using health
information systems. Computational and mathematical methods in medicine. 2013; 2013.
Rahimi B, Vimarlund V. Methods to evaluate health information systems in healthcare settings: a literature
review. J Med Syst. 2007; 31(5): 397-432. doi: 10.1007/s10916-007-9082-z. PMID: 17918694.
Bullard M, Holroyd BR, Meurer DP. How usability of a web-based clinical decision support system has the
potential to contribute to adverse medical events. human-computer interaction. 2008; 16: 17.
Wharton C, Rieman J, Lewis C, Polson P. The cognitive walkthrough method: A practitioner's guide.
Usability inspection methods. John Wiley & Sons, Inc; 1994.
Wechsung I. An evaluation framework for multimodal interaction. T-Labs Series in Telecommunication
Services. Springer. 2014; 10: 978-3. doi: 10.1007/978-3-319-03810-0.
Guise JM, Segel S. Teamwork in obstetric critical care. Best Pract Res Clin Obstet Gynaecol. 2008; 22(5):
-51. doi: 10.1016/j.bpobgyn.2008.06.010. PMID: 18701352, PMCID: PMC4987289.
Hajialiafzali H, Moss J, Mahmood M. Efficiency measurement for hospitals owned by the Iranian social
security organisation. J Med Syst. 2007; 31(3): 166-72. doi: 10.1007/s10916-007-9051-6. PMID:
Fariba Begdeli SJT, Amir Ashkan Nasiripour , Farhad Ghaffari Assessing Hospital Governance in Iran
Social Security Organization Journal of Applied Environmental and Biological Sciences 2015;5(9):290-5.
Sinding SW. Population, poverty and economic development. Philos Trans R Soc Lond B Biol Sci. 2009;
(1532): 3023-30. doi: 10.1098/rstb.2009.0145. PMID: 19770153, PMCID: PMC2781831.
Computer Skills: Levels of Proficiency. In: university C, editor. 2011.
Ellsworth MA, Dziadzko M, O’Horo JC, Farrell AM, Zhang J, Herasevich V. An appraisal of published
usability evaluations of electronic health records via systematic review. J Am Med Inform Assoc. 2017;
(1): 218-26. doi: 10.1093/jamia/ocw046. PMID: 27107451.
Edwards PJ, Moloney KP, Jacko JA, Sainfort F. Evaluating usability of a commercial electronic health
record: A case study. International Journal of Human-Computer Studies. 2008; 66(10): 718-28. doi:
1016/j.ijhcs.2008.06.002.
Choi J, Bakken S. Web-based education for low-literate parents in Neonatal Intensive Care Unit:
Development of a website and heuristic evaluation and usability testing. Int J Med Inform. 2010; 79(8):
-75. doi: 10.1016/j.ijmedinf.2010.05.001. PMID: 20617546, PMCID: PMC2956000.
Joshi A, Arora M, Dai L, Price K, Vizer L, Sears A. Usability of a patient education and motivation tool
using heuristic evaluation. J Med Internet Res. 2009; 11(4): e47. doi: 10.2196/jmir.1244. PMID: 19897458,
PMCID: PMC2802560.
Pressler TR, Yen PY, Ding J, Liu J, Embi PJ, Payne PR. Computational challenges and human factors
influencing the design and use of clinical research participant eligibility pre-screening tools. BMC Med
Inform Decis Mak. 2012; 12(1): 47. doi: 10.1186/1472-6947-12-47. PMID: 22646313, PMCID:
PMC3407791.
Horsky J, Kaufman DR, Oppenheim MI, Patel VL. A framework for analyzing the cognitive complexity of
computer-assisted clinical ordering. Journal of biomedical informatics. 2003; 36(1): 4-22. doi:
1016/S1532-0464(03)00062-5.
Dybå T, Dingsøyr T. Empirical studies of agile software development: A systematic review. Information
and software technology. 2008; 50(9): 833-59. doi: 10.1016/j.infsof.2008.01.006.
Nielsen J, Landauer TK. A mathematical model of the finding of usability problems. Proceedings of the
INTERACT'93 and CHI'93 conference on Human factors in computing systems; 1993: ACM. doi:
1145/169059.169166.
Khajouei R, Ahmadian L. Methods Used for Evaluation of Health Information Systems in Iran. Journal of
Health Administration. 2013; 16(53): 7-21
Published
Issue
Section
License
Copyright (c) 2020 KNOWLEDGE KINGDOM PUBLISHING
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
