Purpose Even though workflow analysis in the operating room has come a long way, current systems are still limited to research. In the quest for a robust, universal setup, hardly any attention has been given to the dimension of audio despite its numerous advantages, such as low costs, location, and sight independence, or little required processing power.Methodology We present an approach for audio-based event detection that solely relies on two microphones capturing the sound in the operating room. Therefore, a new data set was created with over 63 h of audio recorded and annotated at the University Hospital rechts der Isar. Sound files were labeled, preprocessed, augmented, and subsequently converted to log-mel-spectrograms that served as a visual input for an event classification using pretrained convolutional neural networks.Results Comparing multiple architectures, we were able to show that even lightweight models, such as MobileNet, can already provide promising results. Data augmentation additionally improved the classification of 11 defined classes, including inter alia different types of coagulation, operating table movements as well as an idle class. With the newly created audio data set, an overall accuracy of 90%, a precision of 91% and a F1-score of 91% were achieved, demonstrating the feasibility of an audio-based event recognition in the operating room.Conclusion With this first proof of concept, we demonstrated that audio events can serve as a meaningful source of information that goes beyond spoken language and can easily be integrated into future workflow recognition pipelines using computational inexpensive architectures.

Swallowing problems (dysphagia) is associated with significant morbidity and mortality therefore diagnosis and treatment of dysphagia is important. Diagnostic tests include screening procedures, clinical swallowing examinations, and instrumental examination procedures. A non-invasive diagnostic option is auscultation of the swallowing act. However, there are different statements about the reliability and validity of the manual execution of this test. We developed a mobile hardware system to record cervical sounds using two microphones on the neck to acquire audio a data set. To generate ground truth data, fiberendoscopic swallow examinations were performed simultaneously to identify dysphagia. In order to diagnostically assess the swallowing sounds a spectrogram based classification pipeline was developed. In a first step this enabled us to identify different swallowing patterns in healthy individuals. With an accuracy of ~95%, we were able to reliably detect swallows within audio recordings, while the classification of types of swallow (dry, water, solid food) indicate the need for further improvements within the project ahead. In the future, we anticipate AI based analysis of auscultated swallowing sounds to detect swallowing disorders and aspirations.

PURPOSE:Acoustic information can contain viable information in medicine and specifically in surgery. While laparoscopy depends mainly on visual information, our goal is to develop the means to capture and process acoustic information during laparoscopic surgery. METHODS:To achieve this, we iteratively developed three prototypes that will overcome the abdominal wall as a sound barrier and can be used with standard trocars. We evaluated them in terms of clinical applicability and sound transmission quality. Furthermore, the applicability of each prototype for sound classification based on machine learning was evaluated. RESULTS:Our developed prototypes for recording airborne sound from the intraperitoneal cavity represent a promising solution suitable for real-world clinical usage All three prototypes fulfill our set requirements in terms of clinical applicability (i.e., air-tightness, invasiveness, sterility) and show promising results regarding their acoustic characteristics and the associated results on ML-based sound classification. CONCLUSION:In summary, our prototypes for capturing acoustic information during laparoscopic surgeries integrate seamlessly with existing procedures and have the potential to augment the surgeon's perception. This advancement could change how surgeons interact with and understand the surgical field.

Abstract Objectives Due to the coronavirus disease 2019 (Covid-19) pandemic, all scientific conferences in the year 2020 had to be adapted in their form of presence to accommodate for safety regulations, postponed, or canceled entirely. As organizers of the annual Computer Assisted Radiology & Surgery International Conference & Exhibition (CARS)-Conference 2020, we decided to hold a “hybrid” conference, i.e., a virtual conference with partial presence to mitigate the drawbacks of a purely virtual conference. It is the purpose of this paper to describe the results and experience gained by our first hybrid conference. Methods Besides technical necessities like an online conferencing tool, we introduced additional personal namely the technical chairs and communication officers ensuring a smooth flow of presentations. To measure the success of the hybrid conference, we assessed various parameters during the conference (e.g., counting of adverse events, delays, and no-shows) and sent a questionnaire to participants for evaluation after the conference. Results We offered four types of presentation formats, whereas the majority of speakers presented their pre-produced videos including live discussions. Significant delays in sessions occurred during the morning sessions, which could be reduced during lunch breaks. The analysis of the influence of the distribution of the audience’s location/time zone toward the attendance rate showed a high relevance for the American zone and only little influence for the Asian-Pacific region. Based on the questionnaire, 60% of responders considered the hybrid approach as superior and 12% as inferior to purely virtual conferences. Conclusions Most scientific associations in 2020 had to struggle with a dramatic change: Regular, traditional meetings with personal communication and exchange, networking, and creation of new visions became obsolete almost instantly. As an alternative, virtual conferences became increasingly popular, and are offering additional advantages (e.g., reduction of cost for travel, lodging, and time on transit). To overcome the drawbacks of purely virtual conferences, we introduced a hybrid concept for the CARS-Congress. While certainly, those with the privilege to take part personally on-site did benefit most from the hybrid format. Facing upcoming waves of the Covid-19 Pandemic, with ongoing changes to the regulations on meetings and transit, hybrid conferences are a viable option for scientific conferences for the future.

We propose a novel method to tackle the visual-inertial localization problem for constrained camera movements. We use residuals from the different modalities to jointly optimize a global cost function. The residuals emerge from IMU measurements, stereoscopic feature points, and constraints on possible solutions in SE(3). In settings where dynamic disturbances are frequent, the residuals reduce the complexity of the problem and make localization feasible. We verify the advantages of our method in a suitable medical use case and produce a dataset capturing a minimally invasive surgery in the abdomen. Our novel clinical dataset MITI is comparable to state-of-the-art evaluation datasets, contains calibration and synchronization and is available at [1].

Background Digitalization affects almost every aspect of modern daily life, including a growing number of health care services along with telemedicine applications. Fifth-generation (5G) mobile communication technology has the potential to meet the requirements for this digitalized future with high bandwidths (10 GB/s), low latency (<1 ms), and high quality of service, enabling wireless real-time data transmission in telemedical emergency health care applications. Objective The aim of this study is the development and clinical evaluation of a 5G usability test framework enabling preclinical diagnostics with mobile ultrasound using 5G network technology. Methods A bidirectional audio-video data transmission between the ambulance car and hospital was established, combining both 5G-radio and -core network parts. Besides technical performance evaluations, a medical assessment of transferred ultrasound image quality and transmission latency was examined. Results Telemedical and clinical application properties of the ultrasound probe were rated 1 (very good) to 2 (good; on a 6 -point Likert scale rated by 20 survey participants). The 5G field test revealed an average end-to-end round trip latency of 10 milliseconds. The measured average throughput for the ultrasound image traffic was 4 Mbps and for the video stream 12 Mbps. Traffic saturation revealed a lower video quality and a slower video stream. Without core slicing, the throughput for the video application was reduced to 8 Mbps. The deployment of core network slicing facilitated quality and latency recovery. Conclusions Bidirectional data transmission between ambulance car and remote hospital site was successfully established through the 5G network, facilitating sending/receiving data and measurements from both applications (ultrasound unit and video streaming). Core slicing was implemented for a better user experience. Clinical evaluation of the telemedical transmission and applicability of the ultrasound probe was consistently positive.

BACKGROUND Digitalization affects almost every aspect of modern daily life including a growing number of healthcare services along with telemedicine applications. 5th. generation mobile communication technology (5G) has the potential to meet the requirements for this digitalized future with high bandwidths (10 GB/s), low latency (< 1ms) and high quality of service, enabling wireless real-time data transmission in telemedical emergency health care applications. OBJECTIVE We present the results of a 5G field test framework enabling preclinical diagnostics with mobile ultrasound for emergency patients using 5G network slicing technology. METHODS A bi-directional audio-video data transmission between ambulance car and hospital was established, combining both 5G-radio and -core network parts. Besides technical performance evaluations also medical assessment of transferred ultrasound image quality and transmission latency was examined. RESULTS Telemedical and clinical application properties of the ultrasound probe were rated very good – good (VAS). The 5G field test revealed an average End-2-End round trip latency of 10 ms. The measured average throughput for the ultrasound image traffic was 4 Mbps and for the video stream 12 Mbps. Traffic saturation revealed a lower video quality and a slower video stream. Without core slicing, the throughput for the video application was reduced to 8 Mbps. Deployment of core network slicing facilitated quality and latency recovery. CONCLUSIONS Bi-directional data transmission between ambulance car and remote hospital site was successfully established through the 5G network, facilitating sending/receiving data and measurements from both applications (ultrasound unit and video streaming). Core slicing was implemented for better user experience.

We propose a new benchmark for evaluating stereoscopic visual-inertial computer vision algorithms (SLAM/ SfM/ 3D Reconstruction/ Visual-Inertial Odometry) for minimally invasive surgical (MIS) interventions in the abdomen. Our MITI Dataset available at [https://mediatum.ub.tum.de/1621941] provides all the necessary data by a complete recording of a handheld surgical intervention at Research Hospital Rechts der Isar of TUM. It contains multimodal sensor information from IMU, stereoscopic video, and infrared (IR) tracking as ground truth for evaluation. Furthermore, calibration for the stereoscope, accelerometer, magnetometer, the rigid transformations in the sensor setup, and time-offsets are available. We wisely chose a suitable intervention that contains very few cutting and tissue deformation and shows a full scan of the abdomen with a handheld camera such that it is ideal for testing SLAM algorithms. Intending to promote the progress of visual-inertial algorithms designed for MIS application, we hope that our clinical training dataset helps and enables researchers to enhance algorithms.

Purpose:Anastomosis creation after resective gastrointestinal surgery is a crucial task. The present review examines the techniques and implants currently available for anastomosis creation and analyses to which extent they already address our clinical needs, with a special focus on their potential to enable further trauma minimization in visceral surgery. Methods:A multi-database research was conducted in MEDLINE, Scopus, and Cochrane Library. Comparative controlled and uncontrolled clinical trials dealing with anastomosis creation techniques in the intestinal tract in both German and English were included and statistically significant differences in postoperative complication incidences were assessed using the RevMan5.4 Review Manager (Cochrane Collaboration, Oxford, UK). Results:All methods and implant types were analyzed and compared with respect to four dimensions, assessing the techniques' current performances and further potentials for surgical trauma reduction. Postoperative outcome measures, such as leakage, stenosis, reoperation and mortality rates, as well as the tendency to cause bleeding, wound infections, abscesses, anastomotic hemorrhages, pulmonary embolisms, and fistulas were assessed, revealing the only statistically significant superiority of hand-suture over stapling anastomoses with respect to the occurrence of obstructions. Conclusion:Based on the overall complication rates, it is concluded that none of the anastomosis systems addresses the demands of operative trauma minimization sufficiently yet. Major problems are furthermore either low standardization potentials due to dependence on the surgeons' levels of experience, high force application requirements for the actual anastomosis creation, or large and rigid device designs interfering with flexibility demands and size restrictions of the body's natural access routes. There is still a need for innovative technologies, especially with regard to enabling incisionless interventions.

Purpose In current clinical practice, intraoperative repositioning of mobile C-arms is challenging due to a lack of visual cues and efficient guiding tools. This can be detrimental to the surgical workflow and lead to additional radiation burdens for both patient and personnel. To overcome this problem, we present our novel approach Lidar-based X-ray Positioning for Mobile C-arms (RAY-POS) for assisting circulating nurses during intraoperative C-arm repositioning without requiring external aids. Methods RAY-POS consists of a localization module and a graphical user interface for guiding the user back to a previously recorded C-Arm position. We conducted a systematic comparison of simultaneous localization and mapping (SLAM) algorithms using different attachment positions of light detection and ranging (LIDAR) sensors to benchmark localization performance within the operating room (OR). For two promising combinations, we conducted further end-to-end repositioning tests within a realistic OR setup. Results SLAM algorithm gmapping with a LIDAR sensor mounted 40 cm above the C-arm’s horizontal unit performed best regarding localization accuracy and long-term stability. The distribution of the repositioning error yielded an effective standard deviation of 7.61 mm. Conclusion We conclude that a proof-of-concept for LIDAR-based C-arm repositioning without external aids has been achieved. In future work, we mainly aim at extending the capabilities of our system and evaluating the usability together with clinicians.

Purpose Surgical documentation is an important yet time-consuming necessity in clinical routine. Beside its core function to transmit information about a surgery to other medical professionals, the surgical report has gained even more significance in terms of information extraction for scientific, administrative and judicial application. A possible basis for computer aided reporting is phase detection by convolutional neural networks (CNN). In this article we propose a workflow to generate operative notes based on the output of the TeCNO CNN. Methods Video recordings of 15 cholecystectomies were used for inference. The annotation of TeCNO was compared to that of an expert surgeon (HE) and the algorithm based annotation of a scientist (HA). The CNN output then was used to identify aberrance from standard course as basis for the final report. Moreover, we assessed the phenomenon of ‘phase flickering’ as clusters of incorrectly labeled frames and evaluated its usability. Results The accordance of the HE and CNN was 79.7% and that of HA and CNN 87.0%. ‘Phase flickering’ indicated an aberrant course with AUCs of 0.91 and 0.89 in ROC analysis regarding number and extend of concerned frames. Finally, we created operative notes based on a standard text, deviation alerts, and manual completion by the surgeon. Conclusion Computer-aided documentation is a noteworthy use case for phase recognition in standardized surgery. The analysis of phase flickering in a CNN’s annotation has the potential of retrieving more information about the course of a particular procedure to complement an automated report.

Purpose During the COVID-19 pandemic, a threatening bottleneck of medical staff arose due to a shortage of trained caregivers, who became infected while working with infectious patients. While telemedicine is rapidly evolving in the fields of teleconsultation and telesurgery, proper telediagnostic systems are not yet available, although the demand for contactless patient–doctor interaction is increasing. Methods In this project, the current limitations were addressed by developing a comprehensive telediagnostic system. Therefore, medical examinations have been assessed in collaboration with medical experts. Subsequently, a framework was developed, satisfying the relevant constraints of medical-, technical-, and hygienic- aspects in order to transform in-person examinations into a contactless procedure. Diagnostic steps were classified into three groups: assisted procedures carried out by the patient, teleoperated examination methods, and adoptions of conventional methods. Results The Telemedical Diagnostic Framework was implemented, resulting in a functional proof of concept, where potentially infectious patients could undergo a full medical examination. The system comprises, e.g., a naso-pharyngeal swab, an inspection of the oral cavity, auscultation, percussion, and palpation, based on robotic end-effectors. The physician is thereby connected using a newly developed user-interface and a lead robot, with force feedback control, that enables precise movements with the follower robot on the patient’s side. Conclusion Our concept proves the feasibility of a fully telediagnostic system, that consolidates available technology and new developments to an efficient solution enabling safe patient-doctor interaction. Besides infectious situations, this solution can also be applied to remote areas.

Abstract In context of the Corona pandemic, telemedicine acquired a new significance. Whereas previously the aim was to override given barriers, now, in the case of a pandemic, the main idea is to create an intentional distance between patients and healthcare professionals in order to avoid cross-infection. To meet the needs of a fully diagnostic examination, a robotic based system was designed. However, collaborative robotic systems bear new risks, that have to be dealt with. To prepare future physicians for telediagnostics, we developed a training curriculum for the telemedical examinations. It is based upon multiple stages including a skill trainer, healthy volunteers, supervised examinations of real patients and an exam. In a first proof of concept, we demonstrated the existence of a learning curve and significant better performance after the passed curriculum compared to an untrained collective.

The major advantages of 5G networks for eHealth use cases are the low latency transmission and network slicing. In this paper, we explore the feasibility of using 5G to enhance remote mobile ultrasound. We demonstrate a medical use case to show how eHealth applications can be enhanced with 5G radio technology and network slicing. By using a flexible, reconfigurable test-bed, we examine the benefits of ultra low latency communications (URLLC) and slicing in a real world field test. This field test not only shows the advantages, but also reveals additional requirements of using 5G for eHealth sector. Our field test examines the impact of communication latency and slicing for performance evaluations. The reported results validate the feasibility of using 5G technology for eHealth applications. Meanwhile, comprehensive evaluations give the first impression of how 5G-enhanced eHealth applications perform in the real world scenario.

A core principle of modern health care is the compliance of hygienic and aseptic techniques in areas that are sensitive to contamination through bacteria, dust, aerosols, and fallout, primarily in operating theatres or around patients with contagious diseases. Keeping track of potentially contaminated surfaces in an environment is a major concern, especially when protecting from COVID-19. This work proposes a novel concept in using 3D sensing technology to track human movement within an indoor area and identifying high-risk contaminated surfaces in real-time. It combines recent Augmented Reality display technology, which allows keeping track of decontaminated surfaces during the cleaning process using an interactive visualization method. The proposed concept of Clean- AR is implemented in a clinical environment used for observation in COVID-19 scenarios. We discuss key challenges and outline further research direction in effectively reducing the risk of contamination using the proposed concept.

A part from the tremendous increase in the demand for telemedicine during the COVID-19 pandemic, the use of telemedical technology offers many advantages, such as better coverage of rural areas and improved access to specialists. While current telediagnostic possibilities are often limited to a verbal consultation, the field of surgery has already made use of robotics for one of the most challenging areas of medicine: invasive procedures. Since comprehensive diagnostics are a prerequisite for each surgery, we built upon the knowledge gained in telesurgery and developed a telediagnostic system that allows for an extensive perioperative and emergency examination. It is based on a robotic platform consisting of a remote lead robotic arm at the physician's site and a follower robot at the patient's site. Mirroring all movements directly and using force-feedback, both parties can precisely interact, enabling tasks such as auscultation, percussion, and palpation without the need for extensive training. Our overall setup also includes the possibility to measure and monitor all relevant vital parameters and can be used to perform ear and nasopharyngeal inspections as well as an automatic swab to screen for COVID or other contagious diseases prior to hospital admission. In this paper, we focus on the potential of this technology for the surgical community by demonstrating the ease of adding an ultrasound probe to our modular setup to perform a high-quality emergency ultrasound examination. While the system is not yet ready for everyday use in a hospital and drawbacks such as a high cost persist, our setup paves the way for the future use of telediagnostics in surgery.

PURPOSE:While demand for telemedicine is increasing, patients are currently restricted to tele-consultation for the most part. Fundamental diagnostics like the percussion still require the in person expertize of a physician. To meet today's challenges, a transformation of the manual percussion into a standardized, digital version, ready for telemedical execution is required.METHODS:In conjunction with a comprehensive telemedical diagnostic system, in which patients can get examined by a remote-physician, a series of three robotic end-effectors for mechanical percussion were developed. Comprising a motor, a magnetic and a pneumatic-based version, the devices strike a pleximeter to perform the percussion. Emitted sounds were captured using a microphone-equipped stethoscope. The 84 recordings were further integrated into a survey in order to classify lung and non-lung samples.RESULTS:The study with 21 participants comprised physicians, medical students and non-medical-related raters in equal parts. With 71.4% correctly classified samples, the ventral motorized device prevailed. While the result is significantly better compared to a manual or pneumatic percussion in this very setup, it only has a small edge over the magnetic devices. In addition, for all ventral versions non-lung regions were rather correctly identified than lung regions.CONCLUSION:The overall setup proves the feasibility of a telemedical percussion. Despite the fact, that produced sounds differ compared to today's manual technique, the study shows that a standardized mechanical percussion has the potential to improve the gold standard's accuracy. While further extensive medical evaluation is yet to come, the system paves the way for future uncompromised remote examinations.

In this paper we introduce OperA, a transformer-based model that accurately predicts surgical phases from long video sequences. A novel attention regularization loss encourages the model to focus on high-quality frames during training. Moreover, the attention weights are utilized to identify characteristic high attention frames for each surgical phase, which could further be used for surgery summarization. OperA is thoroughly evaluated on two datasets of laparoscopic cholecystectomy videos, outperforming various state-of-the-art temporal refinement approaches.

PURPOSE:Today's hospitals are designed as collections of individual departments, with limited communication and collaboration between medical sub-specialties. Patients are constantly being moved between different places, which is detrimental for patient experience, overall efficiency and capacity. Instead, we argue that care should be brought to the patient, not vice versa, and thus propose a novel hospital architecture concept that we refer to as Patient Hub. It envisions a truly patient-centered, department-less facility, in which all critical functions occur in the same building and on the same floor.METHODS:To demonstrate the feasibility and benefits of our concept, we selected an exemplary patient scenario and used 3D software to simulate resulting workflows for both the Patient Hub and a traditional hospital based on a generic hospital template by Kaiser-Permanente.RESULTS:According to our workflow simulations, the Patient Hub model effectively eliminates waiting and transfer times, drastically simplifies wayfinding, reduces overall traveling distances by 54%, reduces elevator runs by 78% and improves access to quality views from 67 to 100% for patient rooms, from 0 to 100% for exam rooms and from 0 to 38% for corridors. In addition, the interaction of related medical fields is improved while maintaining the quality of care and the relationship between patients and caregivers.CONCLUSION:With the Patient Hub concept, we aim at rethinking traditional hospital layouts. We were able to demonstrate, alas on a proof-of-concept basis, that it is indeed feasible to place the patient at the very center of operations, while increasing overall efficiency and capacity at the same time and maintaining the quality of care.

Das Positionspapier „Digitalisierung in der Chirurgie“, das im Auftrag der DGCH und in Mitwirkung durch zahlreiche Experten erstellt wird, beabsichtigt, einen Statusbericht über den aktuellen Transformationsprozess und die eingebundenen Technologien zu geben. Zugleich soll es aber dazu dienen, Handlungsempfehlungen zu generieren, die als Thesen bereits formuliert wurden. Die sich hieraus ableitenden Konsequenzen betreffen verschiedene Teilbereiche unseres Faches und sollen in diesem Manuskript erläutert werden.