A fast and clear overview of all relevant information during a surgical procedure can be critical to the success of the operation. The Surgical Panel offers OR personnel a central interface to internal hospital systems such as HIS and PACS and when used in conjunction with the Caliop software, it simplifies management of video. The two elements serve as a core for interacting with hospital systems from within the operating room. "We are keenly interested in finding out what difficulties might occur when using our hardware and software in the OR environment," said James Berge, Product Manager at EIZO GmbH. "By better understanding these issues, we can optimize our products and minimize related risks during the actual operation." For this reason, EIZO began preparing a usability study in early 2018 with the industry-in-clinic platform M3i, which specializes in customer-oriented product development. While EIZO provided its Caliop software and Surgical Panels for the test, M3i took care of the agile integration of nursing staff and surgeons as testers.

Two series of full-fledged usability tests using different test groups have been performed since late 2018. M3i organized access to a one-way mirror lab with two fully equipped operating rooms at the Institute of Emergency Medicine and Medical Management (INM) at the Medical Center of the University of Munich. The study participants were asked to perform a series of common tasks in simulated OR situations without any prior instruction on using the hardware and software. "In the study, we examined the standard workflow of a surgical procedure in all phases: from system start to patient assignment and video management, through to archiving the recorded images, videos and reports from the simulated OR," said Berge. “In addition to the one-way mirror, we observed and recorded participants with cameras positioned at two angles, and we captured the operation of the Caliop software and use of the touchscreen integrated into the Surgical Panel. This allowed us to later follow and analyze the testers’ actions and responses." The examinations focused primarily on participant interaction with the video management elements, and whether medical intervention was impacted by operation of the Caliop software.

Afterwards, the testers were interviewed about their use of the system and were asked to evaluate various aspects of the system on a scale of 1 (easy to use) to 5 (difficult to use) in order to determine its usability based on the System Usability Scale (SUS). The first test run achieved ratings of very good to excellent user friendliness. "The technical affinity within the first test group seemed particularly high, enabling these users to grasp operation very quickly," said Berge. "For this reason we set the bar much higher for the second test group by specifically selecting OR personnel with little prior knowledge of technical systems. Nevertheless, our products still achieved good to very good results according to SUS." Based on the initial data, some useful insight was gained regarding the intuitive operation of the Caliop software and Surgical Panels. For example, the initial layout icon was changed to avoid confusion with other functions. In addition, ambiguous terms such as "delete" and "remove" were formulated more clearly and in some cases replaced with checkboxes. Using feedback from the participants, the way in which patient names are displayed was adapted to facilitate quick and certain identification of the patient, and the selection of recordings for archiving was redesigned to be easier to use. "In addition to the insight we gain from clinical experts, finding lead users through this type of cooperation is particularly important for the continued development of these products using agile methods." said James Berge. "Not only do they provide feedback on using the current software, they offer useful suggestions and ideas for new functions and a better understanding of the challenges facing medical technologists."

Additional testing is planned for the coming months, and evaluation of the data should be finalized in the second quarter of 2020. "When this study is completed we hope to use these findings in the further development of all of our CuratOR products," said Berge in summary. "We also plan to present the results of this study in a white paper sometime before the DMEA (Connecting Digital Health) in Berlin in April of next year." While OR personnel were the focus of the previous usability study, EIZO intends to expand its questions to hospital IT administrators in a separate survey. The experts for medical imaging hope to thereby ensure that their video management systems are of even greater benefit in the increasingly interconnected OR environment.