The application of Artificial Intelligence (AI) for object detection has seen significant advancements, yet detecting physically small and rare objects remains a challenge due to the limited availability of labeled real-world data. To overcome this challenge, we generated synthetic data using Unreal Engine 5.0 and UnrealGT to augment the training datasets for object detection models. Our approach integrates both synthetic and real-world data, with hand-labeled annotations created using the CVAT tool, allowing models such as YOLOv8, YOLOv11, YoloNAS, and Detectron2 with Slicing Aided Hyper Inference (SAHI) to be trained under various environmental conditions. We explored weather, lighting, and proximity variations to ensure the robustness of the models used and validate our approach during comparative analyzes. The results demonstrate the effectiveness of synthetic data in improving model performance, particularly for rare objects in complex environments. This methodology offers promising applications in surveillance, security, and other real-world use cases where rare object detection is critical.
In response to the growing need for socially responsible computer scientists and data scientists, our team is developing a comprehensive data science certificate program specifically tailored for non-computing majors, with a focus on data science for social justice. This program aims to broaden participation in data science and create non-traditional pathways for diverse student populations. Each course in the program is designed to be accessible to non-computing majors, equipping them with the skills to analyze and address social justice issues through data science. Process Oriented Guided Inquiry Learning (POGIL) is employed as an instructional strategy promoting active learning, and real datasets related to social justice are utilized for hands-on activities and assignments, enhancing practical learning experiences. The courses are taught in a synchronous hybrid format, across multiple universities, accommodating both live online and in-person students. This tutorial will equip educators with the tools to incorporate data science for social justice in their courses. Attendees will have access to materials developed for these courses, enabling them to integrate similar content into their own curricula. A key focus is on recent challenges and opportunities created by generative AI. The presenters will share their experiences, course materials, and strategies for introducing computer science through a social justice lens. Participants will share ideas and strategies, which will be collated and made available in a shared repository. This initiative aims to enable educators to train future generations in data science while addressing social justice issues.
Accurate, timely, and comprehensive documentation of medical interventions under battlefield conditions is critical but challenging due to operational constraints and the limitations of manual record-keeping. This paper introduces a multi-modal artificial intelligence (AI) system designed to automate the detection, verification, and documentation of tourniquet applications by integrating synchronized video and audio data streams. The visual pipeline utilizes fine-tuned YOLO11 models for real-time object detection and pose estimation, achieving precise identification and anatomical localization of tourniquets. Concurrently, an audio processing pipeline employs a Whisper speech recognition model, tailored to reliably capture verbal confirmations from medical personnel, even amid significant operational noise. A lightweight fusion algorithm harmonizes the output of these independent modalities, generating structured, time-stamped records that detail device type, anatomical positioning, and procedural context. This multimodal documentation facilitates immediate operational decision making and supports retrospective analyses. In addition, the system features an intuitive graphical user interface (GUI) that offers real-time visualization of detections and transcriptions, improving situational awareness, and minimizing cognitive load for medical personnel. The current implementation operates effectively on standard laptop hardware, with architectural considerations explicitly oriented towards future deployment on resource-constrained edge devices, such as the Raspberry Pi 5 and NVIDIA Jetson Nano. The presented results demonstrate the practicality of deploying lightweight, scalable AI-driven documentation systems in austere, tactical environments, establishing a solid foundation for extending real-time monitoring capabilities across a broader spectrum of battlefield trauma interventions.
Background and Objectives: Stratifying thyroid nodules according to malignancy risk is a crucial step in early diagnosis and patient care. Recently, deep learning techniques have emerged as powerful tools for medical diagnostics, particularly with convolutional neural networks (CNNs) applied to medical image classification. This study aimed to develop a new hybrid CNN model for classifying thyroid nodules using the TN5000 ultrasound image dataset. Materials and Methods: The TN5000 dataset includes 5000 ultrasound images, with 3572 malignant and 1428 benign nodules. To address the issue of class imbalance, the researchers applied an R-based anomaly data augmentation method and a GAN-based technique (G-RAN) to generate synthetic benign images, resulting in a balanced dataset for training. The model architecture was built on a pre-trained EfficientNet-B3 backbone, further enhanced with squeeze-and-excitation (SE) blocks and residual refinement modules to improve feature extraction. The task was to classify malignant nodules (labeled 1) and benign nodules (labeled 0). Results: The proposed hybrid CNN achieved strong performance, with an accuracy of 89.73%, sensitivity of 90.01%, precision of 88.23%, and an F1-score of 88.85%. The total training time was 42 min. Conclusions: The findings demonstrate that the proposed hybrid CNN model is a promising tool for thyroid nodule classification on ultrasound images. Its high diagnostic accuracy suggests that it could serve as a reliable decision-support system for clinicians, improving consistency in diagnosis and reducing human error. Future work will focus on clinical validation, explainability of the model’s decision-making process, and strategies for integration into routine hospital workflows.
Accessing and querying NASA’s Planetary Data System (PDS) can present a steep learning curve due to the complex metadata structures and large volume of filterable parameters. To address this, we developed an LLM-powered assistant to help users generate accurate filter selections for planetary datasets, with an initial focus on Mars Reconnaissance Orbiter (MRO) data. Our approach leverages Retrieval-Augmented Generation (RAG) with hybrid methods, including BM25 and dense embeddings, to interpret natural language queries and recommend relevant filters. We also introduce a similarity search function that enables users to retrieve scientifically relevant datasets based on semantically related queries or previous selections, improving the refinement and contextualization of results. In addition, we developed an image analysis module that allows users to input planetary images and returns visually and contextually similar images by using a vision encoder and a pre-indexed embedding database. This functionality is currently limited to the Mars Science Laboratory (MSL) mission, specifically imagery related to the Curiosity rover. This module also extracts available details about the input image, enhancing user understanding and supporting visual-based data discovery. For PDS3 datasets (legacy format), we implemented a script that achieves over 80% accuracy and precision in predicting appropriate filters. For PDS-4 (the modern metadata structure), we are testing a new AI assistant that leverages the PDS API client to guide users through the well-organized dropdown filters, potentially removing the need for direct URL prediction due to the improved schema. A unified GUI is also in development to integrate all assistant modules and further enhance accessibility. This system enhances usability across both metadata structures, significantly lowering the barrier for scientific data discovery and increasing the efficiency of user interaction with planetary archives.
The surge in data science courses highlights an educational shift towards incorporating data skills as a fundamental component of a well-rounded academic curriculum. This growth indicates a recognition of the critical role that data plays in shaping our understanding of complex issues in today's data-driven society. It also underscores the need for data science education to be accessible, representative, and tailored to a wide array of learners and professionals. The work to date to establish a data science foundational knowledge framework represents a pivotal step in formalizing and enhancing data science education. Yet, these frameworks have been largely designed for those pursuing data science careers. The field is in need of a data science framework that focuses on the essential introductory knowledge and skills for non-CS students to build a solid foundation in data science. This paper provides a description of the literature review process and experiences the research team has drawn from to develop such a framework. It also positions this framework for future research in studying effectiveness and alignment to the K-12 space.
While students are often passionate about their chosen fields, they often have limited awareness of the profound impact of AI technologies on their professions. In order to advance efforts in building subject-relevant AI literacy among undergraduate students studying Computer Science and non-Computer Science (Criminal Justice and Forensic Science) it is imperative to engage in rigorous efforts to develop and study curricular infusion of Artificial Intelligence topics. Using a Design-Based Research model, the project team and the external evaluators studied the first iteration of the module development and implementation. Using data collected through surveys, focus groups, critical review, and reflection exercises the external evaluation team produced findings that informed the project team in revising and improving their materials and approach for the second iteration. These efforts can help educators and the AI module developers tailor their AI curriculum to address these specific areas, ensuring that students develop a more accurate understanding of applications of AI in their future career field.
Efficient and effective fire detection has proven critical and if not achieved it can pose significant ecological and economic challenges. By introducing early exits into fire video processing using MSDNet, our approach enables quick identification of fires and smoke, ensuring a prompt response to potential fire incidents. Emphasizing efficiency, our method is tailored for resource-constrained edge devices, providing a practical solution for fire-prone regions and enhancing overall fire detection and prevention efforts. Investigating different model sizes yielded accuracy ranging from 86% to 94%, with smaller models outperforming larger models. The adoption of MSDNet allowed for the achievement of an F1-Score of 0.2. This preliminary work has shown the value of small models in the robust detection of fires and introducing early exits can further performance.
Artificial Intelligence (AI) models for object detection face challenges when tasked with identifying rare objects due to the scarcity of data for these items. This scarcity often leads to mediocre performance, as the model lacks sufficient training examples for these specific objects. Acquiring real-world data for such rare objects can also be a challenging and time-consuming process. To address this issue, we developed a method aimed at improving object detection for rare objects. Our method involves the generation of synthetic data, which is subsequently utilized to train AI models. By incorporating synthetic data into the training process, we augment the available dataset, enabling AI models to better recognize and classify rare objects. The synthetic data production methods of this work can be used to build AI applications in areas such as physical security and surveillance, and self-driving vehicles which need to detect rare objects on roads to navigate safely.
This paper explores innovative approaches to ad-dressing data gaps in time series datasets, specifically focusing on monthly mean water levels along the US East and Gulf of Mexico coasts. We employ both pragmatic and machine learning (ML) techniques to backfill missing data, enhancing the reliability of historical water level records that are essential for climate and coastal hazard studies. Our pragmatic approach includes a comprehensive framework for distance calculation, correlation analysis, and relative intensity measurement, providing a robust baseline for data imputation. In parallel, we develop advanced ML models, including ensemble methods such as Voting and Stacking Regressors, which significantly outperform traditional techniques in hindcasting missing values. These models leverage feature engineering and data augmentation, incorporating cli-matic indices and temporal features to capture complex patterns in the data. The results demonstrate that the combined approach both fills data gaps effectively and also offers a versatile solution for improving the quality of environmental time series datasets, contributing to better understanding and forecasting of coastal water level variations.
Reports of progress in research into Artificial Intelligence (AI) and its applications are accumulating very rapidly. Specifically, Machine Learning (ML) applications based on large data sets have moved to the forefront of innovations in the field. New ML models have led to the adoption of AI in different disciplines. The development of the most recent large language models has created so much interest that it might mean a revolution in using of AI. Some make a stronger claim that it is a turning point in human civilization's history, and we have started the AI age after replacing the obsolete in many aspects of Information Age applications. One of the immediate challenges is how to use AI and ML responsibly with proper protection for humans and human society. In this paper, we report on our efforts in introducing trustworthiness of ML in college curricula and what factors influence AI-supported decision making. The main goal is to allow students to gain an understanding not only of concepts but also of the limitations of AI. This will help in their participation in our society of the AI Age. The process of AI democratization needs to be established to control the growth of ML use and understand the human dangers of various types of data-driven modeling approaches in AI.
Unmanned aerial vehicle (UAV) flight in confined spaces is an emerging area of interest because solutions can create opportunities for developing drone-based inspections for societal benefits, military and other applications. This article describes new work in the area of UAV flight in confined spaces. To develop the UAVs' ability to operate in small space environments, the task of identifying an opening to the confined space and determine its center was solved, with the goal being to autonomously minimize the distance between the UAV's position and the gate's center. There were three main approaches for gate detection, each involving placing an object in the four corners of the gateway. The first approach to gate detection was grayscale and HSV filtering of LED lights. Using a combination of both HSV and grayscale filtering caused glare from the LED lights to skew center detection of each light. The second strategy was to train YOLOv5 using over 200 annotated images of LED lights, but it still produced an issue of identifying false positives which caused the center to skew. The third detection strategy utilized april tags instead of LED lights. The specific patterns solve the issue of false positives entirely, for each april tag has a unique recognizable pattern. The new issue was visibility as a result of lighting. However, the positives of size manipulation and accuracy overshadowed this. The maximum resolutions and distances at which the April tags could be detected were tested with the picamera V1. Based on algorithms for the RaspberryPi, the UAV was able to adjust it's flight with the center of the april tag gateway achieving the project goal.
Physics-guided machine learning (PGML) offers a new approach to stability modeling during machining that leverages experimental data generated during the machining process while incorporating decades of theoretical process modeling efforts. This approach addresses specific limitations of machine learning models and physics-based models individually. Data-driven machine learning models are typically black box models that do not provide deep insight into the underlying physics and do not reflect physical constraints for the modeled system, sometimes yielding solutions that violate physical laws or operational constraints. In addition, acquiring the large amounts of manufacturing data needed for machine learning modeling can be costly. On the other hand, many physical processes are not completely understood by domain experts and have a high degree of uncertainty. Physics-based models must make simplifying assumptions that can compromise prediction accuracy. This research explores whether data generated by an uncertain physics-based milling stability model that is used to train a physics-guided machine learning stability model, and then updated with measured data, domain knowledge, and theory-based knowledge provides a useful approximation to the unknown true stability model for a specific set of factory operating conditions. Four novel strategies for updating the machine learning model with experimental data are explored. These updating strategies differ in their assumptions about and implementation of the type of physics-based knowledge included in the PGML model. Using a simulation experiment, these strategies achieve useful approximations of the underlying true stability model while reducing the number of experimental measurements required for model update.
The capabilities of Language Models (LMs) have continued to increase in recent years, as have their computational requirements. Widely available APIs have also become available. These APIs present new challenges for ease of gradient based fine-tuning by users, resulting in the use models which may be larger than necessary and more expensive, therefore reducing accessibility. In this paper, we present a new methodology for increasing performance of single-shot LMs by chaining multiple smaller LMs. Additionally, as the derived representation is in plain-text it is readily human interpretable. We show that optimizing the context which leads to this derived representation results in improved performance and reduced cost.
Deep-learning based language models (LMs) have significantly enhanced services relating to language generation and classification. Our focus in this paper is on the Multiple Mini Interview (MMI) which is commonly used internationally by medical schools to screen applicants based on their ability to answer short questions in a considerate, professional manner. In this paper we establish the ability of LMs, specifically GPT-3, to generate MMI questions, simulate responses, and rate answers. We compare these simulated questions with their human generated counterparts and, after identifying the optimal hyperparameters, find that 92% of generated responses are capable of fooling humans. We also find that after identifying the optimal hyperparameters for question-answering, LMs are capable producing high quality simulated MMI responses, with an average human rating of 3.5 out of 5. Finally, GPT-3 is shown to have some agreement with human ratings, although it tends to overestimate the quality of the response. Conditional text generation by LMs alone seems to be able to significantly support MMI preparation.
With the recent successes of research in Artificial Intelligence (AI), specifically Machine Learning (ML), infusion of AI and ML concepts and use of tools can help increase the responsible adoption of AI and ML in different disciplines. We report on the design of a workshop on ML, results from the evaluation of the workshop, and suggests topics and pedagogical approaches that may be adopted for disseminating the core concepts of ML which are among the most prevalent data-driven modeling approaches in AI. The workshop individually targeted a diverse range of participants including college and university faculty members from different disciplines at 2-and-4 year institutions. The materials of the workshop were made publicly available. Future workshops and educational modules building on this work will be able to infuse knowledge of the frontiers of AI and problems & benefits.
This paper describes a novel supporting tool for the case-based learning (CBL). Recent advances in deep-learning based language models (LMs) have enabled highly dynamic interactivity in dialog services and story generation. We leverage the progress in modelling language to develop a technologically augmented CBL pedagogy which we analyze with a standardized assessment. Our assessment shows reasonable case interactivity, low rates of factual inaccuracy, and no inappropriate machine-sourced responses. We also compare our assessment results across the case categories of Ethics, Chemistry, Biology, and Medicine, but find no statistically significant differences. In summary, we develop a framework for analyzing the ability of LMs to augment CBL, apply this framework to the GPT-3 LM, and discuss some of the challenges and potential solutions to ensuring proper usage in the classroom environment.
Accurate and objective monitoring of a fracture's healing process is both essential to patient quality of care, as well as determination of the chances of nonunion and postoperative intervention. In recent years, due to industrialization, injury rates in developing countries, notably road traffic injuries (RTIs), have drastically increased. This has led to many fracture patients in ill- equipped countries, such as Kenya with only 60 orthopedic surgeons for a population of 36.9 million, not having any rigorous rehabilitation protocol or quality postoperative care. This work focuses on the development of a telemetric gait analysis insole that works in conjunction with a mobile application and convolutional neural network. This technique automates the tedious process of tracking postoperative fracture rehabilitation by analyzing ground reaction forces (GRFs) of patients which correlate well with weight-bearing ability, fracture healing, and delayed union. 4 force-sensitive resistors (FSRs) are placed in the insole under the primary areas for force measurement. An Arduino microcontroller compiles the data and sends it to a Python program via a Bluetooth module. The Python program performs peak analysis on the data to determine the average peak Vertical Ground Reaction Force (VGRF) of the strides to measure if a patient is properly healing. As a further step, we employ a 1D-CNN to differentiate between healed and healing patients to automatically find which patients have nonunions. With these methodologies we are able to automatically diagnose rural patients with nonunions based on only ground reaction force measurements at minimal costs and without an on-site physician.