We develop a novel approach for Linear combination of unitaries (LCU) type decomposition for structured sparse matrices. Such matrices frequently arise during numerical solution of partial differential equations which are ubiquitous in science and engineering. LCU is a versatile quantum algorithmic primitive that plays an important role in context of both variational quantum algorithms (VQA) and fully fault-tolerant ones, and has been applied to a diverse range of problems. Conventionally, Pauli basis is used for LCU decomposition, which however in worst case can result in number of LCU terms that scale quadratically with respect to the matrix size. We show that by using an alternate basis one can better exploit the sparsity and underlying structure of matrix leading to number of tensor product terms which scale only logarithmically with respect to the matrix size. We develop numerical and semi-analytical approaches for computing sigma basis decomposition for an arbitrary matrix. Given this new basis is comprised of non-unitary operators, we employ the concept of unitary completion to design efficient quantum circuits for evaluation of the expectation values of operators composed of tensor product of elements from sigma basis which can be used for cost function evaluation in VQAs. We also develop an approach for block encoding of arbitrary operator given its decomposition in sigma basis which could be used in variety of fully fault-tolerant algorithms. We compare our approach with other related concepts in the literature including unitary dilation and provide numerical illustrations on several PDE examples.
Recent studies suggest that gradient-based methods applied to relaxed box-constrained Quadratic Unconstrained Binary Optimization (QUBO) formulations can outperform classical heuristics in some large-scale regimes, often relying on heavy parallelization. However, these methods still underperform heuristics in other settings. In this work, we clarify this apparent discrepancy through a detailed analysis of the relaxed non-convex QUBO local maxima for both the Maximum Independent Set (MIS) and Maximum Cut (MaxCut) problems, and by introducing a new quadratic objective for MaxCut. Motivated by this analysis, we propose a mutation-based differentiable global reset algorithm, combined with local search to escape local maxima. We term our approach mQO, standing for mutation-based Quadratic combinatorial Optimization. The proposed strategy dramatically improves the performance of gradient-based solvers without heavy reliance on GPU parallelized initializations, indicating that stalling, rather than model capacity or compute, is the dominant bottleneck. As a result, on large-scale graphs, mQO achieves superior performance against state-of-the-art heuristics, commercial integer programming solvers, and recent GPU methods.
Generative artificial intelligence (AI) and natural language processing (NLP) applications, particularly those utilizing large language models (LLMs) with Retrieval-Augmented Generation (RAG), fine-tuning and their variations, are receiving widespread attention. However, despite their promise, the dependability of LLM applications remains uncertain. This study introduces a new dimension to improving LLM performance through systematic query classification. A taxonomy-based approach was developed to categorize queries by their likelihood of eliciting accurate responses, from those likely to succeed to those prone to failure. Bloom's Taxonomy (BT) provided the conceptual direction for assessing and classifying queries using a curated dataset of question-answer pairs mapped to BT levels to evaluate RAG-LLM performance across categories. Building on insights from BT-based analysis, an innovative 'Taxonomical Query Classifier' (TQC) framework for LLMs is introduced: Finder (F), Evaluator (Ev), Explainer (Ex), and Generator (G)-collectively termed FEEG. FEEG-TQC classifies queries to improve predictability, accuracy, and enhance human-AI interaction. It differentiates queries based on intent, whether they (F) seek factual information, (Ev) evaluate content, (Ex) explain concepts, or (G) generate new content. By developing the concept of query category variations, this research provides a systematic approach to managing query quality and predictive estimations of LLM response accuracy. The findings contribute to the development of more effective human-AI interaction opportunities for LLM-based generative AI systems, emphasizing query classification as an important factor in improving generative AI systems.
The success and wide adoption of generative AI (GenAI), particularly large language models (LLMs), have attracted the attention of cybercriminals seeking to abuse models, steal sensitive data, or disrupt services. Moreover, providing security to LLM-based systems is a significant challenge, as both traditional threats to software applications and threats targeting LLMs and their integration must be mitigated. In this survey, we shed light on the security and privacy concerns of such LLM-based systems by performing a systematic review and comprehensive categorization of threats and defensive strategies considering the entire software and LLM life cycles. We analyze real-world scenarios with distinct characteristics of LLM usage, spanning from development to operation. In addition, threats are classified according to their severity level and the scenarios to which they pertain, facilitating the identification of the most relevant threats. Recommended defense strategies are systematically categorized and mapped to the corresponding life cycle phase and possible attack strategies they mitigate. This work paves the way for consumers and vendors to understand and efficiently mitigate risks during the integration of LLMs in their respective solutions or organizations. It also enables the research community to benefit from the discussion of open challenges and edge cases that may hinder the secure and privacy-preserving adoption of LLM-based systems.
Large language models (LLMs) have significantly transformed natural language understanding and generation, but they raise privacy concerns due to potential exposure of sensitive information. Studies have highlighted the risk of information leakage, where adversaries can extract sensitive information embedded in the prompts. In this work, we introduce a novel private prediction framework for generating high-quality synthetic text with strong privacy guarantees. Our approach leverages the Differential Privacy (DP) framework to ensure worst-case theoretical bounds on information leakage without requiring any fine-tuning of the underlying models. The proposed method performs inference on private records and aggregates the resulting per-token output distributions. This enables the generation of longer and coherent synthetic text while maintaining privacy guarantees. Additionally, we propose a simple blending operation that combines private and public inference to further enhance utility. Empirical evaluations demonstrate that our approach outperforms previous state-of-the-art methods on in-context-learning (ICL) tasks, making it a promising direction for privacy-preserving text generation while maintaining high utility.
