In recent years, Artificial Intelligence (AI) has seen rapid advances in performance and impact, disrupting major industries, including finance and healthcare. Machine learning open-source software (MLOSS) platforms such as GitHub and Hugging Face have contributed significantly to this advancement, enabling AI developers to share, reuse, and collaborate on AI development. While these platforms accelerate AI development, the MLOSS assets they host also contain vulnerabilities that can impact applications that leverage them. To map the MLOSS landscape and understand the vulnerabilities contained within MLOSS on platforms such as GitHub and Hugging Face, we have developed an MLOSS Collection Pipeline. Our pipeline has collected 373,634 models from Hugging Face and 39,115 repositories from GitHub and identified 6,751,739 vulnerabilities. The results of our pipeline offer several promising directions for future research, including vulnerability linking analysis and cross-platform vulnerability propagation identification.

The increasing adoption of open-source AI models has introduced critical security risks, as vulnerabilities in AI model source code propagate through the software supply chain. This study presents a reinforcement learning (RL) based framework for automatically optimizing engineered prompts to reduce security vulnerabilities in existing AI model source code. This approach serves as a security-focused intervention applied during human interaction with potentially insecure AI source code. Our RL agent operates within a structured state space and utilizes an action space consisting of 75 prompt engineering actions to refine LLM prompts for AI model source code. The reward function guides the model toward secure and executable code. Our evaluation demonstrates that the RL agent reduces high-severity vulnerabilities by 40.27%, medium-severity vulnerabilities by 64.91%, and low-severity vulnerabilities by 68.91% in existing repositories that users may leverage. This method can help developers utilizing OSS AI model source code ensure the security of their software supply chain. To our knowledge, this is the first RL-based vulnerability mitigation specifically designed for AI model source code, contributing to ongoing efforts in human-machine intelligence for security analytics.

Cybersecurity is a present and growing concern that needs to be addressed with both behavioral and design-oriented research. Public cloud providers such as Amazon Web Services and federal funding agencies such as the National Science Foundation have invested billions of dollars into developing high-performance computing resources accessible to users through configurable virtual machine (VM) images. This approach offers users the flexibility of changing and updating their environment for their computational needs. Despite the substantial benefits, users often introduce thousands of vulnerabilities by installing open-source software packages and misconfiguring file systems. Given the scale of vulnerabilities, security personnel struggle to identify and prioritize vulnerable assets for remediation. In this research, we designed a novel unsupervised deep learning-based Multi-View Combinatorial-Attentive Autoencoder (MV-CAAE) to capture multi-dimensional vulnerability data and automatically identify groups of similar vulnerable compute instances to help facilitate the development of targeted remediation strategies. We rigorously evaluated the proposed MV-CAAE against state-of-the-art methods in three technical clustering experiments. Experiment results indicate that the MV-CAAE achieves V-measure scores (metric of cluster quality) 8 percent-48 percent higher than benchmark methods. We demonstrated the practical value through a comprehensive case study by clustering vulnerable VMs and gathering qualitative feedback from experienced security professionals through semi-structured interviews. The results indicated that clustering vulnerable assets can help prioritize vulnerable instances for remediation and enhance decision-making tasks. The present design-research work also contributes to our theoretical knowledge of cyber-defense.

Emerging Artificial Intelligence (AI) applications are bringing with them both the potential for significant societal benefit and harm. Additionally, vulnerabilities within AI source code can make them susceptible to attacks ranging from stealing private data to stealing trained model parameters. Recently, with the adoption of open-source software (OSS) practices, the AI development community has introduced the potential to worsen the number of vulnerabilities present in emerging AI applications, building new applications on top of previous applications, naturally inheriting any vulnerabilities. With the AI OSS community growing rapidly to a scale that requires automated means of analysis for vulnerability management, we compare three categories of unsupervised graph embedding methods capable of generating repository embeddings that can be used to rank existing applications based on their functional similarity for AI developers. The resulting embeddings can be used to suggest alternatives to AI developers for potentially insecure AI repositories.

Artificial Intelligence (AI) has rapidly proliferated as a critical disruptive technology in the 21st century. Hugging Face hosts pre-trained models, facilitating the sharing and use of open-source code. Hugging Face has been used by 22,000+ organizations, including Intel and Microsoft, with 2.6+ billion model downloads. While Hugging Face democratizes access to AI models, these models may contain unknown security vulnerabilities. In this research, we automatically collect models from Hugging Face, link them to their underlying code bases on GitHub, and perform a large-scale vulnerability assessment of these repositories. Through our approaches, we collected about 110,000 models from Hugging Face and over 29,000 GitHub repositories. Our vulnerability assessment revealed a larger percentage (35.98 % ) of high-severity vulnerabilities compared to low-severity vulnerabilities (6.79%). This trend in severity levels contradicts the results of severities detected in repositories forked from root repositories and searched repositories. Given that many of the vulnerabilities reside in fundamental AI repositories such as Transformers, the results of this vulnerability assessment have significant implications for supply chain software security and AI risk management more broadly.

Social Coding Repositories (SCRs) such as GitHub host open-source code that is significant to the global economy. However, open-source code is especially vulnerable, with most vulnerabilities being introduced due to human error. An important mitigation strategy is preventing the introduction of vulnerabilities. One of the ways this can be achieved is through targeted developer security training, which necessitates the identification of high-risk actors and predicting the introduction of vulnerabilities. In this study, we propose a novel framework for predicting the introduction of vulnerabilities in SCRs by users, with a novel dynamic graph representation learning model, security continuous propagation, and evolution (seCoPE). The proposed seCoPE framework addresses the limitations of existing methods by incorporating the relative influence of nodes on the propagation of information to generate high-quality nodal embeddings. We systematically evaluate seCoPE against prevailing Recurrent Neural Network (RNN) based and Attention-based models on a vulnerability introduction dataset. The proposed framework has important implications for cybersecurity providers, firms, and software developers.

The rapid pace of the development of artificial intelligence (AI) solutions is enabled by leveraging foundational tools and frameworks that allow AI developers to focus on application logic and rapid prototyping. However, the security vulnerabilities present in foundation repositories might cause irreparable damage due to the AI solutions built using these libraries being deployed in production environments. Our research leverages source code hosted on the prevailing social coding platform GitHub to identify vulnerabilities in foundational repositories commonly used for modern AI development (Linux, BERT, PyTorch, and Transformers), as well as the AI repositories that utilize foundation repositories as dependencies. Using an unsupervised graph embedding approach, we generate graph embeddings that capture vulnerability information and the relationships between repositories. Based on these embeddings, we performed clustering as our downstream task to group similarly vulnerable repositories. Our research identifies patterns and similarities between repositories and will help develop effective mitigation of vulnerabilities present in groups of repositories based on foundational AI repositories. We also discuss the implications of identifying such clusters of vulnerable repositories.

Major technology companies (e.g., Google, Microsoft, etc.) have begun to host hundreds of repositories on GitHub. Although these repositories provide a valuable resource to enhance the speed of software collaborations and development, they often contain vulnerabilities that can quickly spread as users import the open-source software into their projects. This research aims to identify the propagation patterns of vulnerabilities introduced by technology organizations across the GitHub community to identify influential repositories for targeted remediation. In this paper, we report our ongoing work to leverage the social structure of GitHub data to construct a graph representing the relationship amongst repositories. We applied prevailing unsupervised graph embedding methods to generate embeddings of each repository based on its network attributes and vulnerability information. The embeddings generated from this study will be used in a downstream clustering task to identify how vulnerabilities propagate between repositories. The results produced from this analysis will enable the development of effective vulnerability mitigation strategies that can address vulnerabilities in major organization repositories as well as the repositories to which the vulnerabilities have spread to.

The accelerated growth of computing technologies has provided interdisciplinary teams a platform for producing innovative research at an unprecedented speed. Advanced scientific cyberinfrastructures, in particular, provide data storage, applications, software, and other resources to facilitate the development of critical scientific discoveries. Users of these environments often rely on custom developed virtual machine (VM) images that are comprised of a diverse array of open source applications. These can include vulnerabilities undetectable by conventional vulnerability scanners. This research aims to identify the installed applications, their vulnerabilities, and how they vary across images in scientific cyberinfrastructure. We propose a novel unsupervised graph embedding framework that captures relationships between applications, as well as vulnerabilities identified on corresponding GitHub repositories. This embedding is used to cluster images with similar applications and vulnerabilities. We evaluate cluster quality using Silhouette, Calinski-Harabasz, and Davies-Bouldin indices, and application vulnerabilities through inspection of selected clusters. Results reveal that images pertaining to genomics research in our research testbed are at greater risk of high-severity shell spawning and data validation vulnerabilities.

The scientific cyberinfrastructure community heavily relies on public internet-based systems (e.g., GitHub) to share resources and collaborate. GitHub is one of the most powerful and popular systems for open source collaboration that allows users to share and work on projects in a public space for accelerated development and deployment. Monitoring GitHub for exposed vulnerabilities can save financial cost and prevent misuse and attacks of cyberinfrastructure. Vulnerability scanners that can interface with GitHub directly can be leveraged to conduct such monitoring. This research aims to proactively identify vulnerable communities within scientific cyberinfrastructure. We use social network analysis to construct graphs representing the relationships amongst users and repositories. We leverage prevailing unsupervised graph embedding algorithms to generate graph embeddings that capture the network attributes and nodal features of our repository and user graphs. This enables the clustering of public cyberinfrastructure repositories and users that have similar network attributes and vulnerabilities. Results of this research find that major scientific cyberinfrastructures have vulnerabilities pertaining to secret leakage and insecure coding practices for high-impact genomics research. These results can help organizations address their vulnerable repositories and users in a targeted manner.

The frequency and costs of cyber-attacks are increasing each year. By the end of 2019, the total cost of data breaches is expected to reach $2.1 trillion through the ever-growing online presence of enterprises and their consumers. The tools to perform these attacks and the breached data can often be purchased within the Dark-net. Many of the threat actors within this realm use its various platforms to broker, discuss, and strategize these cyber-threat assets. To combat these attacks, researchers are developing Cyber-Threat Intelligence (CTI) tools to proactively monitor the ever-growing online hacker community. This paper will detail the creation and use of a CTI tool that leverages a social network to identify cyber-threats across major Dark-net data sources. Through this network, emerging threats can be quickly identified so proactive or reactive security measures can be implemented.

Cyber-attacks cost the global economy over $450 billion annually. To combat this issue, researchers and practitioners put enormous efforts into developing Cyber Threat Intelligence, or the process of identifying emerging threats and key hackers. However, the reliance on internal network data to has resulted in inherently reactive intelligence. CTI experts have urged the importance of proactively studying the large, ever-evolving online hacker community. Despite their CTI value, collecting data from hacker community platforms is a non-trivial task. In this paper, we summarize our efforts in systematically identifying and automatically collecting a large-scale of hacker forums, carding shops, Internet-Relay-Chat, and Dark Net Marketplaces. We also present our efforts to provide this data to the larger CTI community via the AZSecure Hacker Assets Portal (www.azsecure-hap.com). With our methodology, we collected 102 platforms for a total of 43,981,647 records. To the best of our knowledge, this compilation of hacker community data is the largest such collection in academia.