In this paper, we systematically present and discuss practical applications of Large Language Models (LLMs) in software security, concretely in code vulnerability detection, fuzz testing, and exploit generation. Measurements of various research outcomes are analyzed to answer questions about the performance of LLMs in those fields, including a comparison with tools that follow traditional approaches. In addition, the drawbacks and a future outlook, along with a delineation of technical challenges, are provided. Challenges include the cost- and time-intensive training of LLMs, the limited context-length understanding of program code, the high false positive rate due to hallucinations, and keeping the data up-to-date so that definitions of newly detected vulnerabilities are covered.
Edge intelligent applications like VR/AR and language model based chatbots have become widespread with the rapid expansion of IoT and mobile devices. However, constrained edge devices often cannot serve the increasingly large and complex deep learning (DL) models. To mitigate these challenges, researchers have proposed optimizing and offloading partitions of DL models among user devices, edge servers, and the cloud. In this setting, users can take advantage of different services to support their intelligent applications. For example, edge resources offer low response latency. In contrast, cloud platforms provide low monetary cost computation resources for computation-intensive workloads. However, communication between DL model partitions can introduce transmission bottlenecks and pose risks of data leakage. Recent research aims to balance accuracy, computation delay, transmission delay, and privacy concerns. They address these issues with model compression, model distillation, transmission compression, and model architecture adaptations, including internal classifiers. This survey contextualizes the state-of-the-art model offloading methods and model adaptation techniques by studying their implication to a multi-objective optimization comprising inference latency, data privacy, and resource monetary cost.
Efficiently optimizing multi-model inference pipelines for fast, accurate, and cost-effective inference is a critical challenge in machine learning production systems. Our work introduces IPA, an innovative online deep learning inference pipeline adaptation system. IPA dynamically configures batch sizes, replication, and model variants to simultaneously optimize accuracy, minimize costs, and meet user-defined latency requirements. Unlike existing approaches, IPA captures the correlation between configuration changes across multiple pipeline stages, enabling holistic end-to-end optimization of inference pipelines rather than optimizing each stage individually. Key to IPA's innovation is its joint optimization of accuracy and cost in a multi-stage pipeline setting. Although previous work has focused predominantly on cost-aware optimizations, IPA leverages accuracy-aware adaptations, combining autoscaling and model-switching techniques to achieve precise trade-offs between accuracy, cost, and latency. This approach is particularly advantageous in scenarios where heterogeneity in model variants can be exploited to enhance overall pipeline performance dynamically. Through extensive evaluations in a Kubernetes implementation with five real-world inference pipelines (Video Monitoring, Audio Question Answering, Audio Sentiment Analysis, Summarisation Question Answering, and Natural Language Processing), we demonstrate that IPA improves normalized accuracy by up to 35% with a negligible cost increase of less than 5%. The system's effectiveness in achieving granular trade-offs makes it a valuable contribution to the optimization of complex multistages ML inference pipelines.
Some recent services use a sequencer to simplify ordering operations on sharded data. The sequencer assigns each operation a multi-sequence number which explicitly orders the operation on each shard it accesses. Existing sequencers have two shortcomings. First, failures can result in some multi-sequence numbers never being assigned, exposing a non-contiguous multi-sequence, which requires complex scaffolding to handle. Second, existing implementations use single-machine sequencers, limiting service throughput to the ordering throughput of one machine.We make two contributions. First, we posit that sequencers should expose our new contiguous multi-sequence abstraction. Contiguity guarantees every sequence number is assigned an operation, simplifying the abstraction. Second, we design and implement MASON , the first system to expose the contiguous multi-sequence abstraction and the first to provide a scalable multi-sequence. MASON is thus an ideal building block for consistent, scalable services. Our evaluation shows MASON unlocks scalable throughput for two strongly-consistent services built on it.
The ever-increasing variety of microcontrollers aggravatesthe challenge of porting embedded software to new devicesthrough much manual work, whereas code generators can beused only in special cases. Moreover, only little technical documentation for these devices is available in machine-readableformats that could facilitate automating porting efforts. Instead, the bulk of documentation comes as print-orientedPDFs. We hence identify a strong need for a processor toaccess the PDFs and extract their data with a high quality toimprove the code generation for embedded software.In this paper, we design and implement a modular processor for extracting detailed datasets from PDF files containing technical documentation using deterministic table processing for thousands of microcontrollers. Namely, we systematically extract device identifiers, interrupt tables, package and pinouts, pin functions, and register maps. In our evaluation, we compare the documentation from STMicro againstexisting machine-readable sources. Our results show thatour processor matches 96.5 % of almost 6 million referencedata points, and we further discuss identified issues in bothsources. Hence, our tool yields very accurate data with onlylimited manual effort and can enable and enhance a significant amount of existing and new code generation use cases inthe embedded software domain that are currently limited by alack of machine-readable data sources.
