Every minute delay in life-saving intervention increases mortality risk in injured patients. Given this relationship, quality measurement of trauma resuscitation includes the timing of provider decisions and interventions. The current approach of manual recording of patient arrival and other timestamps may be inaccurate due to providers underestimating elapsed time during resuscitation. We introduce a frame-based computer vision system to automatically detect and classify two specific phases of patient arrival: (1) when the patient enters the trauma resuscitation room, and (2) when the patient is moved onto the bed. The proposed system consists of two stages. The first stage uses a pattern-based method to detect when a patient enters the room. The second stage uses both side-view and top-view video feeds to determine when the patient has been moved to the bed, addressing occlusion issues and enhancing robustness and precision. To minimize labeling effort and accelerate detection, we introduce PA-YOLO, a frame-based classification model, in both stages of the system. We evaluated our system using 5-fold cross-validation on 60 trauma resuscitation cases. Our results show that this approach achieves an accuracy of 0.92 +/- 0.01 for patient-at-door detection and 0.93 +/- 0.03 for patient-on-bed detection, with an average detection delay of 2.15 +/- 1.24 s. The proposed system outperforms SlowFast by up to 5 percentage points and our previous I3D-based system by up to 7 percentage points in detection accuracy, while reducing detection delay from over 5 s to about 2 s. Compared to baseline model YOLO11, our PA-YOLO improves detection performance by 2.0% while reducing floating point operations per second (FLOPS) by 51.5%.

We describe the design and initial evaluation of a video-based explainable interface (XUI) for personal protective equipment (PPE) compliance monitoring in a dynamic medical setting of trauma resuscitation. The system integrates video input, a computer vision model for detecting PPE status, backend processing, and the video-based XUI. The initial UI underwent a major redesign and was evaluated with five nurse educators. Participants interacted with the XUI features, provided feedback on visual elements, and compared and ranked alternate design options. Using the results, we determined visuals for alert symbols, PPE status, and compliance metrics that support rapid PPE compliance monitoring among care providers. While standard computer vision outputs, such as bounding boxes, improve explainability in object detection models, we suggest moving beyond these model representations to enhance system usability in practice.

While recent advances in activity recognition have achieved remarkable success, these methods depend on large-scale datasets with extensive annotations. Developing fine-grained activity recognition for domain-specific tasks presents unique challenges, including limited labeled data due to privacy concerns and labor-intensive annotation processes. Distinguishing visually similar activities remains difficult, as fine-grained actions often involve subtle differences (i.e., slight variations in hand positioning and tiny object interactions). In trauma resuscitation, critical procedures such as intravenous (IV) placement and temperature measurement occur within small regions of the scene and in complex, crowded environments, further complicating activity recognition. To address these challenges, we reframed medical activity recognition as an image classification task instead of relying on temporal modeling. We introduce TextAidAR, an activity recognition method that incorporates expert-defined domain knowledge to improve visual representation learning. By leveraging vision-text alignment, our approach enhances spatial feature learning using structured medical descriptions provided by domain experts. We also introduce a lightweight module, the MedSpec adapter, to refine pre-trained text embeddings for medical contexts and propose two key loss functions—class-conditioned alignment loss and MedDistinct loss—to enhance the robustness of visual-text alignment. Our studies show that TextAidAR surpasses existing video-based and image-based methods, achieving state-of-the-art performance with a mAP of 0.72 in detecting 13 fine-grained medical activities in real-world trauma resuscitation. On the NurViD dataset, TextAidAR achieves a 41.2% improvement in overall performance compared to the strongest existing baseline.

Personal protective equipment (PPE) is critical for infection control in healthcare, protecting workers and patients from infection risks. The COVID-19 pandemic further highlighted the importance of correct PPE use, yet adherence to U.S. Centers for Disease Control and Prevention guidelines remains inconsistent. Continuous human monitoring of PPE adherence is impractical because it is labor-intensive and may expose observers to infection risk. Automated monitoring is a promising alternative, but reliable PPE assessment in clinical videos remains difficult due to occlusion and subtle differences between adherence levels. To address these challenges, we propose SAFE - Smart Adherence detection Framework for PPE, a cascaded computer vision system for real-time monitoring of PPE wearing status, including complete, incomplete, and absent cases, with a focus on gowns and masks. SAFE uses a two-stage design: Stage 1 detects gown status and localizes head regions, and Stage 2 classifies mask status from head crops. We evaluate SAFE on R2PPE, a ceiling-view trauma-room simulation dataset with dense PPE annotations and complex scenes. SAFE improves overall average precision from 0.48 to 0.67 and increases mask-class average precision by 0.33 compared to a baseline one-stage detector. We further validate SAFE across modern detector backbones, including transformer-based detectors, and on real-case trauma-room data using class-level and alarm-level criteria, improving class-level mask accuracy from 0.59 to 0.65 while maintaining a high alarm-level recall of 0.98. SAFE could enhance PPE monitoring with minimal human intervention, providing a scalable solution for improving infection control in healthcare settings.

We describe the design and initial evaluation of a video-based explainable interface (XUI) for personal protective equipment (PPE) compliance monitoring in a dynamic medical setting of trauma resuscitation. The system integrates video input, a computer vision model for detecting PPE status, backend processing, and the video-based XUI. The initial UI underwent a major redesign and was evaluated with five nurse educators. Participants interacted with the XUI features, provided feedback on visual elements, and compared and ranked alternate design options. Using the results, we determined visuals for alert symbols, PPE status, and compliance metrics that support rapid PPE compliance monitoring among care providers. While standard computer vision outputs, such as bounding boxes, improve explainability in object detection models, we suggest moving beyond these model representations to enhance system usability in practice.

Object detection in Synthetic Aperture Radar (SAR) imagery remains a challenging task due to strong speckle noise, low semantic texture, and a significant domain gap from natural images. While recent approaches have addressed these challenges through complex pretraining strategies and transformer-based architectures, they often incur high computational costs, limiting their applicability in real-time and edge scenarios. In this work, we propose a lightweight SAR-specific detector built upon the YOLO11s framework. The proposed model integrates an enhanced HGNet backbone for stronger feature extraction, a lightweight structural design to reduce computational cost, and a dynamic upsampling method. On the two public datasets SARDet-100K and SSDD, the proposed method consistently outperforms existing approaches. Compared with the YOLO11-s baseline, our method achieves an average improvement of approximately 3.4% in mAP@50:95 and 0.9% in mAP@50, while significantly reducing model complexity, with about a 17% reduction in parameters and a 1 GFLOP decrease in computational cost. These results demonstrate that a streamlined architecture, carefully adapted to SAR characteristics, can achieve strong performance without relying on heavy pretraining or large-scale backbone networks.

We propose Traffic VILA, a framework for traffic safety description and analysis, addressing challenges in understanding complex interactions between pedestrians and vehicles in multi-view traffic videos. Our proposed method combines several innovative concepts for effective video analysis. We integrate a phase-aware selection of camera views, utilize a frame augmentation strategy that combines focused image regions, and develop a module that identifies the most relevant moments for answering questions about the scene. By leveraging a multimodal large language model, TrafficVILA effectively analyzes a traffic scene while preserving fine-grained details. Experiments on the Woven Traffic Safety dataset demonstrate the effectiveness of our framework in generating accurate captions and answering visual questions. Overall, trafficVILA achieved the 5th rank with a final score of 57.46 in the 9th AI City Challenge Traffic Safety Description and Analysis track. Our source code is openly accessible at https://github.com/BaamPark/TrafficVILA

Labeling medical activities from visual datasets requires extensive time and effort for practitioners. Deep learning methods have been widely used to train partially labeled datasets for predicting activity labels. Cotraining, a semi-supervised learning technique that utilizes predictions from two different models, is used in conjunction with active learning to mitigate sampling bias. There is a need to optimize manual labeling efforts with active learning while improving the prediction performance. In this paper, we developed the SCALE-TAG methodology, based on single-view cotraining and active learning, to label concurrent activities in video records cost-effectively. ScALE-TAG utilized the cosine similarity metric to enhance diversity within the subsets and selected informative samples based on prediction uncertainty for retraining the models. We evaluated the performance of scale-tag for six trauma resuscitation activities. During the training stage, the average labeling time required for selected samples was 10.93 % of the manual annotation time for all unlabeled samples. We then applied scale-tag to a test set. The F1 scores for test samples were over 0.7 for four activities, where the labeling time required for ambiguous predictions of test samples was 8 % for one of them, 17 % for another, and around 20 % for the other two. The improvement in prediction accuracy and decrease in labeling time we obtained with ScAleTAG are promising for developing deep learning-based methods to support healthcare providers with effective activity labeling.

Monitoring adherence to personal protective equipment (PPE) guidelines is critical for infection control in clinical environments. Automated methods for monitoring require precise localization of PPE in complex real-world videos. While recent video segmentation models like SAM2 have shown strong performance, they underperform in healthcare settings due to cluttered back-grounds and frequent occlusions of small PPE items such as masks and gloves. We have identified two core limitations of SAM2 in this context: (1) difficulty in distinguishing PPE objects from complex backgrounds, and (2) tracking drift during occlusion. To address these issues, we propose MAPS: Morphology Aware PPE Segmentation, a training-free extension of SAM2 that in-corporates two novel components: (1) a morphology-aware memory module that leverages shape descriptors to selectively retain reliable memory features and (2) a person-aware filtering module that removes predictions that do not align with detected person regions. MAPS achieves consistent improvements across multiple SAM2 model scales and outperforms recent SAM2-based extensions on a newly introduced PPE object tracking dataset. The code and the new dataset are available at https://github.com/yangwanzhao/MAPS.

Manual annotation of unlabeled data for model training is expensive and time-consuming, especially for visual datasets requiring domain-specific experience for multi-labeling, such as video records generated in hospital settings. There is a need to build frameworks to reduce human labeling efforts while improving training performance. Semi-supervised learning is widely used to generate predictions for unlabeled samples in a partially labeled datasets. Active learning can be used with semi-supervised learning to annotate unlabeled samples to reduce the sampling bias due to the label predictions. We developed the aselmar framework based on active and semi-supervised learning techniques to reduce the time and effort associated with multi-labeling of unlabeled samples for activity recognition. aselmar (i) categorizes the predictions for unlabeled data based on the confidence level in predictions using fixed and adaptive threshold settings, (ii) applies a label verification procedure for the samples with the ambiguous prediction, and (iii) retrains the model iteratively using samples with their high-confidence predictions or manual annotations. We also designed a software tool to guide domain experts in verifying ambiguous predictions. We applied aselmar to recognize eight selected activities from our trauma resuscitation video dataset and evaluated their performance based on the label verification time and the mean ap score metric. The label verification required by aselmar was 12.1% of the manual annotation effort for the unlabeled video records. The improvement in the mean ap score was 5.7% for the first iteration and 8.3% for the second iteration with the fixed threshold-based method compared to the baseline model. The p-values were below 0.05 for the target activities. Using an adaptive-threshold method, aselmar achieved a decrease in ap score deviation, implying an improvement in model robustness. For a speech-based case study, the word error rate decreased by 6.2%, and the average transcription factor increased 2.6 times, supporting the broad applicability of ASELMAR in reducing labeling efforts from domain experts.

Delays and process inefficiencies during trauma resuscitation can contribute to adverse patient outcomes. While tracking elapsed time may improve the trauma team's temporal awareness and reduce delays, reliance on manual activation of stop clocks can introduce variability. To address this limitation, we implemented a computer vision-powered automatic stop clock designed to activate upon patient arrival without requiring manual input. We conducted a retrospective video review of 50 trauma resuscitations to assess how the clock was used in practice, followed by semi-structured interviews with nine trauma team members to elicit their feedback and perceptions. This study contributes to the broader discussion on AI-assisted clinical tools, highlighting the role of automation in supporting trauma teams, reducing variability in time tracking, and improving process efficiency.

We examine the use of personal protective equipment (PPE) in two interdisciplinary medical settings to inform the design of just-intime alerts and reminders for correcting PPE noncompliance. We reviewed videos of 26 pediatric resuscitations occurring over the course of the COVID-19 pandemic at an urban pediatric teaching hospital. Through video review, we identified causes for PPE non-compliance, activities that were frequently performed without PPE, instances in which PPE was intentionally removed, and mechanisms by which healthcare providers corrected PPE noncompliance. We also interviewed 18 registered nurses working in the hospital's emergency department (ED) and intensive care unit (ICU) to better understand observed PPE behaviors and practices. Our results suggest that alert design will require considering the urgency of correcting PPE noncompliance against the urgency of tasks being performed. We discuss our findings through the lens of the COM-B framework and conclude by exploring design opportunities for just-in-time alerts and reminders for prompting PPE noncompliance corrections in dynamic medical work.

Deep learning models rely on large-scale labeled datasets, but collecting such data is expensive and time-consuming. Semi-supervised learning (SSL) mitigates this issue by learning from a small set of labeled samples along with a large pool of unlabeled data. However, existing SSL methods struggle with fine-grained classification when dealing with visually similar classes, as they rely solely on visual features and ignore the semantic information within label names. This paper introduces SemiVisBooster, an SSL enhancement approach that utilizes semantic information from label names to guide visual feature learning, addressing the challenges of fine-grained classification. By aligning text embeddings from label names with visual features, our method helps the model capture subtle visual distinctions that purely visual representations may overlook. To enhance robustness, we propose two key components: (1) text embedding de-similarity (TEDS) to reduce confusion caused by similar text embeddings across different class names, and (2) class-aware visual-text alignment loss to accurately define positive and negative pairs during visualtext alignment. Our method achieves state-of-the-art performance on the latest SSL benchmarks. Additionally, on the challenging Food-101 dataset, which contains many visually similar classes and uses only 404 labeled images, our approach improves performance by approximately 13.6% over the second-best method. Code is available at SemiVisBooster

Proper personal protective equipment (PPE) use is critical to prevent disease transmission to healthcare providers, especially those treating patients with a high infection risk. To address the challenge of monitoring PPE usage in healthcare, computer vision has been evaluated for tracking adherence. Existing datasets for this purpose, however, lack a diversity of PPE and nonadherence classes, represent single not multiple providers, and do not depict dynamic provider movement during patient care. We introduce the Resuscitation Room Personal Protective Equipment (R2PPE) dataset that bridges this gap by providing a realistic portrayal of diverse PPE use by multiple interacting individuals in a healthcare setting. This dataset contains 26 videos, 10,034 images and 123,751 bounding box annotations for 17 classes of PPE adherence and nonadherence for eyewear, masks, gowns, and gloves, and one additional head class. Evaluations using newly proposed metrics confirm R2PPE exhibits higher annotation density than three established general-purpose and medical PPE datasets. The R2PPE dataset provides a resource for developing computer vision algorithms for monitoring PPE use in healthcare.

Objective Trauma resuscitation is the initial evaluation and management of injured patients in the emergency department. This time-critical process requires the simultaneous pursuit of multiple resuscitation goals. Recognizing whether the required goal is being pursued can reduce errors in goal-related task performance and improve patient outcomes. The intention to pursue a goal can often be inferred from ongoing and completed treatment activities, but monitoring goal pursuit is cognitively demanding and prone to errors. We introduced an interpretable deep learning-based approach to aid decision making by automatically recognizing goal pursuit during trauma resuscitation. Methods We developed a predictive model to recognize the pursuit of two resuscitation goals: airway stabilization and circulatory support. We used event logs of 381 pediatric trauma resuscitations from August 2014 to November 2022 to train a neural network model with a dual-GRU structure that learns from both time-level and activity-type-level features. Our model makes predictions based on a sequence of activities and corresponding timestamps. To enhance the model and facilitate interpretation of predictions, we used the attention weights assigned by our model to represent the importance of features. These weights identified the critical time points and contributing activities during a goal pursuit. Results Our model achieved an average area under the receiver operating characteristic curve (AUC) score of 0.84 for recognizing airway stabilization and 0.83 for recognizing circulatory support. The most contributing activities and timestamps were aligned with domain knowledge. Conclusion Our interpretable predictive model can recognize provider intention based on a limited number of treatment activities. The model outperformed existing predictive models for medical events in accuracy and in interpretability. Integrating our model into a decision-support system would automate the tracking of provider actions, optimizing workflow to ensure timely delivery of care.

Background Cervical spine (c-spine) injuries can lead to significant disability and mortality. Although stabilization is the primary management for suspected c-spine injuries, lapses in stabilization frequently occur during trauma resuscitation. To facilitate evaluation of c-spine management, we developed a computer vision system to detect stabilization techniques. This system would enable scalable monitoring, including the timing and duration of c-spine stabilization. Methods We developed a 2-stage computer vision system to detect prehospital rigid c-collar, hospital semi-rigid c-collar, and manual in-line stabilization. The system was trained, tested, and validated using image frames extracted from 86 pediatric trauma resuscitation videos at a level 1 pediatric trauma center from October 2022 to May 2023. The first stage identified the patient in each image, and the second stage classified the stabilization technique. A 5-fold cross-validation was performed on the first 68 resuscitation videos for training/testing, with the latest 18 cases reserved for validation. System performance was evaluated using accuracy, precision, recall, F1 score, and Matthews correlation coefficient (MCC). To assess system potential for manual in-line detection, 10 simulation videos were added (eight for training, two for testing). Results In the 18 validation cases, the system achieved high accuracy for binary classification (0.91) and for detecting specific stabilization techniques: prehospital rigid c-collar (0.95), hospital semi-rigid c-collar (0.93), and manual in-line stabilization (0.97). The precision scores were 0.89 for binary classification of any stabilization method, 0.71 for prehospital rigid c-collar, 0.89 for hospital semi-rigid c-collar, and 0.04 for manual in-line. Recall, F1, and MCC scores aligned with these findings, with the highest values observed for detecting the hospital semi-rigid c-collar among the stabilization techniques. Adding simulation videos improved manual in-line stabilization detection, with accuracy 0.62, precision 0.88, recall 0.58, F1 score 0.70, and MCC 0.27. Conclusion The 2-stage computer vision system showed excellent performance for detecting c-spine stabilization, with limitations for manual in-line stabilization due to its rarity. Simulation data improved manual in-line detection, highlighting potential benefits of a more balanced dataset. The computer vision system may serve as a prototype for automated monitoring of trauma resuscitation using the camera infrastructure in the resuscitation room.

Conventional methods in semi-supervised learning (SSL) often face challenges related to limited data utilization, mainly due to their reliance on threshold-based techniques for selecting high-confidence unlabeled data during training. Various efforts (e.g., FreeMatch) have been made to enhance data utilization by tweaking the thresholds, yet none have managed to use 100% of the available data. To overcome this limitation and improve SSL performance, we introduce \algo, a novel algorithm that fully utilizes unlabeled data to boost semi-supervised learning. \algo integrates a self-supervised learning strategy, i.e., Masked Autoencoder (MAE), that uses all available data to enforce the visual representation learning. This enables the SSL algorithm to leverage all available data, including samples typically filtered out by traditional methods. In addition, we propose a synthetic data training approach to further increase data utilization and improve generalization. These innovations lead \algo to achieve state-of-the-art results on challenging datasets. For instance, on CIFAR-100 with 2 labels per class, STL-10 with 4 labels per class, and Euro-SAT with 2 labels per class, \algo achieves low error rates of 18.71%, 9.47%, and 3.07%, respectively. The code will be made publicly available.

Objectives Human monitoring of personal protective equipment (PPE) adherence among healthcare providers has several limitations, including the need for additional personnel during staff shortages and decreased vigilance during prolonged tasks. To address these challenges, we developed an automated computer vision system for monitoring PPE adherence in healthcare settings. We assessed the system performance against human observers detecting nonadherence in a video surveillance experiment. Materials and Methods The automated system was trained to detect 15 classes of eyewear, masks, gloves, and gowns using an object detector and tracker. To assess how the system performs compared to human observers in detecting nonadherence, we designed a video surveillance experiment under 2 conditions: variations in video durations (20, 40, and 60 seconds) and the number of individuals in the videos (3 versus 6). Twelve nurses participated as human observers. Performance was assessed based on the number of detections of nonadherence. Results Human observers detected fewer instances of nonadherence than the system (parameter estimate -0.3, 95% CI -0.4 to -0.2, P < .001). Human observers detected more nonadherence during longer video durations (parameter estimate 0.7, 95% CI 0.4-1.0, P < .001). The system achieved a sensitivity of 0.86, specificity of 1, and Matthew's correlation coefficient of 0.82 for detecting PPE nonadherence. Discussion An automated system simultaneously tracks multiple objects and individuals. The system performance is also independent of observation duration, an improvement over human monitoring. Conclusion The automated system presents a potential solution for scalable monitoring of hospital-wide infection control practices and improving PPE usage in healthcare settings.

Trauma is a leading cause of mortality worldwide, with about 20% of these deaths being preventable. Most of these preventable deaths result from errors during the initial resuscitation of injured patients. Decision support has been evaluated as an approach to support teams during this phase to reduce errors. Existing systems require manual data entry and monitoring, which makes tasks challenging to accomplish in a time-critical setting. This paper identified the specific challenges of achieving effective decision support in trauma resuscitation based on computer vision techniques, including complex backgrounds, crowded scenes, fine-grained activities, and a scarcity of labeled data. To address the first three challenges, the proposed system involved an actor tracker that identifies individuals, allowing the system to focus on actor-specific features. Video Masked Autoencoder (Video-MAE) was used to overcome the issue of insufficient labeled data. This approach enables self-supervised learning using unlabeled video content, improving feature representation for medical activities. For more reliable performance, an ensemble fusion method was introduced. This technique combines predictions from consecutive video clips and different actors. Our method outperformed existing approaches in identifying fine-grained activities, providing a solution for activity recognition in trauma resuscitation and similar complex domains.

Process data constructed from event logs provides valuable insights into procedural dynamics over time. The confidential information in process data, together with the data's intricate nature, makes the datasets not sharable and challenging to collect. Consequently, research is limited using process data and analytics in the process mining domain. In this study, we introduced a synthetic process data generation task to address the limitation of sharable process data. We introduced a generative adversarial network, called ProcessGAN, to generate process data with activity sequences and corresponding timestamps. ProcessGAN consists of a transformer-based network as the generator, and a time-aware self-attention network as the discriminator. It can generate privacy-preserving process data from random noise. ProcessGAN considers the duration of the process and time intervals between activities to generate realistic activity sequences with timestamps. We evaluated ProcessGAN on five real-world datasets, two that are public and three collected in medical domains that are private. To evaluate the synthetic data, in addition to statistical metrics, we trained a supervised model to score the synthetic processes. We also used process mining to discover workflows for synthetic medical processes and had domain experts evaluate the clinical applicability of the synthetic workflows. ProcessGAN outperformed the existing generative models in generating complex processes with valid parallel pathways. The synthetic process data generated by ProcessGAN better represented the long-range dependencies between activities, a feature relevant to complicated medical and other processes. The timestamps generated by the ProcessGAN model showed similar distributions with the authentic timestamps. In addition, we trained a transformer-based network to generate synthetic contexts (e.g., patient demographics) that were associated with the synthetic processes. The synthetic contexts generated by our model outperformed the baseline models, with the distributions similar to the authentic contexts. We conclude that ProcessGAN can generate sharable synthetic process data indistinguishable from authentic data. Our source code is available in https://github.com/raaachli/ProcessGAN.