In high-dimensional data processing,the K-nearest neighbor(KNN)graph is a critical data structure widely used in tasks such as clustering,graph neural networks,and recommendation systems.However,with the increasing use of pretrained embedding models in unstructured data modeling and retrieval,embedding model fine-tuning has become a key step in enhancing the semantic representation capability of embeddings.Such fine-tuning often leads to systematic changes in the vector representations of all data points,which invalidates the original neighborhood relationships in the KNN graph.Existing research primarily focuses on building KNN graphs for static data,lacking efficient solutions for adapting to updated embeddings after fine-tuning.To address this gap,this study proposes FastAdjust,an efficient KNN graph update method tailored for embedding model fine-tuning scenarios.Leveraging the observation that fine-tuning usually causes only minor changes to individual embeddings,incremental adjustments to the original KNN graph are performed by FastAdjust through a local update strategy,significantly improving update efficiency while maintaining graph quality.Specifically,FastAdjust first employs a clustering structure based on product quantization to efficiently and accurately locate a subset of candidate neighbors for each data point,thus narrowing the search space.Secondly,based on data density and the magnitude of embedding variation,FastAdjust leverages their correlation with changes in the KNNs to adaptively allocate update resources according to the degree of neighbor relationship changes,thus improving overall update efficiency.Experimental results on real-world datasets demonstrate that FastAdjust efficiently and accurately adapts KNN graphs to embedding updates with significantly reduced computational cost,showing strong practical value and scalability.
High-speed train Electric Multiple Units (EMU) ground maintenance platforms generate critical telemetry data from sub systems such as pneumatics, braking, and traction. While real-time monitoring is essential for safety, many existing ground stations rely on monolithic Relational Database Management Systems (RDBMS) and lack the infrastructure for heavy big data stacks. This paper proposes a lightweight, non-intrusive real-time post-processing architecture based on Python and standard RDBMS mechanisms which requires no modification to the original data ingestion pipeline. The system is evaluated using telemetry schemas from EMU braking systems. Results show a saturation throughput of 1600 records per second with end-to-end latency of 20 ms on total data volumes of 106 records, which is sufficient to support the monitoring of over 200 concurrent subsystem units at standard sampling rates. A comparative analysis with the industry-standard tool Debezium reveals similar performance limits under identical hardware constraints. This study validates that, for specific legacy industrial scenarios, an RDBMS-native approach offers a superior trade-off between performance, cost, and maintainability.
With the widespread application of multi-source,heterogeneous,and multi-modal data in scenarios such as large models and data lakes,there has been a significant growth in vector-based data retrieval and storage management.By mapping heterogeneous data into high-dimensional vector representations and leveraging vector indices,vector databases facilitate the unified management of diverse data types and enable high-quality similarity search,establishing them as a crucial foundation for applications like generative retrieval and AI-native databases.However,existing vector databases face significant bottlenecks in terms of storage and indexing efficiency,index construction complexity,and retrieval accuracy.Specifically,massive high-dimensional vectors lead to increased storage overhead and maintenance costs for indices.Furthermore,vector index structures are often bloated,resulting in substantial memory consumption.Moreover,the degradation of retrieval accuracy caused by distortion from compression techniques remains an unresolved challenge.This study proposes a framework based on weight residual vector quantization(WRVQ).This method achieves efficient compression and storage with very low distortion by decoupling the quantization direction from the residual magnitude.It stores the residual direction as a unit vector and appends a weight marker.For indexing,a three-layer inverted index structure tailored to the characteristics of WRVQ is designed,comprising an exact match layer,a fuzzy match layer,and a search layer.This structure organically integrates asymmetric distance computation(ADC)with nearest neighbor search techniques to realize approximate nearest neighbor(ANN)search that balances both high accuracy and high efficiency.Experimental results on large-scale datasets demonstrate that,compared to traditional low-dimensional embedding models and existing quantization methods,WRVQ achieves significant improvements across key metrics,including quantization loss,storage compression ratio,and retrieval recall.Furthermore,it exhibits considerable advantages in both index construction and query performance.
Text2SQL has evolved into a significant tool for data analysis and database management by reducing the technical barriers for non-expert users to interact with relational databases.The introduction of large language model(LLM),represented by GPT,further improves the performance of Text2SQL systems.However,existing Text2SQL techniques are difficult to apply directly to the spatial database domain because spatial data involves complex geometric relationships,diverse query types,and the demand for high-precision semantic understanding.To address these issues and lower the threshold for interaction between non-experts and spatial databases,a natural language query(NLQ)transformation method for spatial databases is proposed.The method consists of two core phases:(1)natural language understanding;(2)executable language generation.In phase(1),an entity information extraction algorithm is employed to extract key query entities,and a spatial data query corpus is constructed based on large language models to determine the query type.In phase(2),a structured language model(SLM)is selected according to the query type,and the entities are then mapped into the structured language model to generate the final executable language for spatial databases.Experimental results on multiple real-world datasets demonstrate that the proposed method enables efficient transformation from natural language queries to executable languages of spatial databases.
The increasing frequency of fire incidents in dense urban areas has highlighted the limitations of traditional sensor-based detection systems, such as delayed response and high false alarm rates. To address these challenges, this study proposes an enhanced fire detection algorithm based on You Only Look Once Version 5 (YOLOv5), incorporating a Squeeze-and-Excitation (SE) attention mechanism. The SE module was selected for its effective channel-wise feature recalibration capability and its minimal computational overhead, making it ideal for deployment on edge devices in real-time scenarios. Experimental results demonstrate that the proposed method achieves a precision of 0.87, recall of 0.86, and mAP@0.5 of 0.91, with a mAP@0.5:0.95 of 0.42, indicating robust performance under stricter evaluation criteria. The model maintains high computational efficiency, exhibiting strong potential for practical, real-time fire detection applications.
Rapid gains in Large Language Model (LLM)-based tools are transforming software engineering, from auto-completing function stubs to drafting architectural RFCs. However, current use often depends on ad hoc prompting, resulting in brittle code snippets, inconsistent style guides, and unpredictable test coverage. To enable scalable and repeatable automation, systematic prompt engineering is essential for generating high-quality software artifacts (such as unit tests, refactor patches, and Application Programming Interface (API) documentation) from the same underlying model. To address this need, we propose the Preference-Driven Refinement (PDR) method for prompt engineering, designed to support automated software engineering workflows. PDR introduces an iterative loop where developers specify preferences (e.g., naming conventions, performance constraints, or security rules) after each generation. These preferences—typically captured by editing prompt phrasing or including curated examples—are encoded into subsequent prompts, enabling the model to produce outputs that adhere to project-specific standards and practices. This refinement loop creates a more automated, policy-aware interface between developers and generative models, supporting on-boarding, code review, and other software lifecycle tasks. We present empirical evaluations demonstrating how PDR leverages in-context learning and synthetic example generation to systematically improve prompt quality. Our results show that PDR reduces trial-and-error iterations and yields higher-quality outputs, though with modest increases in refinement time. These findings highlight how structured prompt refinement can help automate manual tasks in software engineering, thereby enhancing consistency, efficiency, and developer experience in AI-assisted development environments.
This study systematically examines the critical challenges confronting global agriculture, including resource scarcity, environmental degradation, and the increasing demand for food production driven by population growth. The research aims to investigate the application of intelligent sensing technologies, encompassing the Internet of Things (IoT), Artificial Intelligence (AI), and big data analytics, in addressing these challenges. The study's primary objective is to design and evaluate an integrated smart agricultural platform that leverages these technologies to enhance productivity, optimize resource efficiency, and support sustainable rural development. Through a synthesis of 50 peer-reviewed studies published between 2015 and 2023, selected based on relevance, methodological rigor, and impact, this review demonstrates that IoT-based systems improve water-use efficiency by up to 30%, while AI-driven crop monitoring reduces pesticide usage by 25%. These findings propose a conceptual framework for the deployment of intelligent sensing technologies, advancing precision agriculture and sustainable food systems.
In certain designs and applications of practical lattice-based cryptography,the use of a specialized variant of LWE problems,where the public matrix is sampled from a non-uniform distribution,is required to establish the securities of corresponding cryptographic schemes.Recently,the formal definition of LWE problems with semi-uniform seeds was introduced in some work,in which the hardness of Euclidean,ideal,and module lattice-based LWE problems with semi-uniform seeds was proved through reduction roadmaps similar to those employed in the hardness proofs of entropic LWE problems.However,known reduction introduces significant losses in the Gaussian parameters of errors and dimensions.Moreover,additional non-standard assumptions are required to demonstrate the hardness of LWE problems with semi-uniform seeds over rings.In this study,a tighter reduction is proposed for LWE problems with semi-uniform seeds by incorporating modified techniques from the hardness proofs of Hint-LWE problems.The proposed reduction is largely unaffected by the algebraic structures of the underlying problems and can be uniformly applied to Euclidean,ideal,and module lattice-based LWE problems with semi-uniform seeds.The hardness of these LWE problems can be established based on standard LWE assumptions without the need for any additional non-standard assumptions.Furthermore,the dimension of the corresponding LWE problems remains unchanged,and the reduction introduces only minimal losses in Gaussian parameters of errors.
Different types of natural resource data are stored in different databases, which results in them having a low degree of correlation. Combining the data of different natural resources to build a knowledge graph for knowledge reasoning would likely increase the correlation of these data, enabling information mining and assisting in the management of natural resources. We construct a natural resource knowledge graph that interrelates different databases and develop a new knowledge reasoning scheme using remote sensing, vegetation distribution, and soil erosion spatial distribution data from Hunan Province, China. The pro-posed scheme includes an input layer, unified generation layer, feature extraction layer, knowledge reason-ing layer and output layer for analyzing the influence of altitude, soil type and vegetation type on the spa-tial distribution of soil erosion. The experimental results show that the proposed knowledge reasoning scheme performs well according to the MRR and Hits@N evaluation metrics. Our research provides a scien-tific basis for predicting the spatial distribution of soil erosion and preventing soil erosion in practical sce-narios.
Event logs record key activity data for process mining, yet quality defects such as attribute and case errors undermine economic benefits by corrupting critical operational insights. Identical timestamp errors occur when multiple events of a process instance mistakenly share the same timestamp—distort execution sequences, misrepresent concurrency, and compromise analysis accuracy. These errors manifest as multi-layered problems, with case-level sequence errors being particularly impactful. Current solutions inadequately correct consecutive event ordering errors due to incomplete utilization of log sequence information. We propose LLM-based Event Sequence Reconstruction (LESR), a novel framework integrating probabilistic ranking with fine-tuned Large Language Model (LLM) to resolve sequence faults. Its two-phase LLM architecture first generates candidates then selects optimally. Evaluated on 5 real and synthetic logs from e-commerce, healthcare, and public administration domains, LESR significantly outperforms baselines in repairing sequence errors caused by identical timestamps.
This research is dedicated to the process and the best practices of the formalization of artificial intelligence (AI) tools usage in software engineering in the end of 2024. The growing experience of first pioneers in this direction in 2022–2023 and visible demand on the market defined the actual business need: study the new intents and initiatives in AI usage in software development with the focus on the corresponding best practices. AI usage in software development has a lot of threats and risks (from labor and economical aspects to the information security) and the main goal of this research is to gather the opinions and visions of senior software engineers and managers into solid industry approach, that might help in future management of practical usage of AI in IT business and production processes. Pan-European research covers the experience of 27 teams in IT companies, system integrators and banks with strong in-house software development.
Practitioners are often dependent on Stack Overflow code during software development, where poor quality is occasionally reported. Research tends to focus on ranking content, identifying defects and predicting future content, but less attention is dedicated to identifying the most suitable techniques for modelling/prediction. Contextualizing the Stack Overflow code quality problem as regression-based, we examined the variables that predict Stack Overflow (Java) code quality, and the regression approach that provides the best predictive power. We observed answer count (β = 0.138), code length (β = 0.382), code spaces (β = 0.099) and lines of code (β = 1.959) as the strongest predictors of code quality on Stack Overflow. Six regression approaches were considered in our evaluation, where Gradient Boosting Machine (GBM) achieved superior performance (RMSE = 2.77, R2 = 0.99, MAE = 0.79) compared to other methods including eXtreme Gradient Boosting (XGBoost) (RMSE = 3.12, R2 = 0.97, MAE = 2.36), and Classification and Regression Trees (CART) (RMSE = 3.45, R2 = 0.96, MAE = 1.77). In fact, even when evaluated against Deep Neural Networks (DeepNN), GBM’s superior performance is maintained. Follow-up evaluations using two independent datasets on Electrical Grid Stability and USA Cancer Mortality confirm GBM’s superior performance, supporting claims for generalizability of our findings. Outcomes here point to the value of the GBM ensemble learning mechanism and need for continued modelling techniques’ experimentation.
To enhance the utilization efficiency of library self-service equipment and minimize manual intervention, this study proposes a systematic framework for operational status monitoring and automated alarm issuance. Leveraging technical methodologies, the system continuously tracks the operational status of network-connected self-service facilities, including online connectivity status, printer consumable levels, and operational application software functionality. The monitoring center serves as the central control hub, conducting scheduled network device diagnostics to ensure seamless normal operation. A Quick Response (QR)-code-based repair reporting mechanism is established, enabling readers to efficiently report equipment issues via code scanning, with repair requests directly transmitted to librarian workstations. Collected monitoring data and user-submitted repair information undergo systematic screening and processing to generate actionable alerts, which are delivered to designated terminals as pop-up notifications for immediate attention. By integrating proactive equipment monitoring, user repair reporting, and automated alerting, the proposed system swiftly addresses operational disruptions, thereby improving equipment availability, reducing staff workload, and elevating the intelligent service capabilities of libraries. This approach demonstrates a practical solution to optimize self-service facility management, enhancing user experience and advancing institutional service efficiency through technology-driven interventions.
In-process feedback is essential in providing useful information to stakeholders to improve software quality in an agile development environment, where development decisions are often made quickly with access to limited measurement data and timely feedback is needed for progress monitoring and for quality assurance, all under fluid, rapidly changing market conditions. This study adapts the original Orthogonal Defect Classification (ODC), initially developed and deployed in large commercial software systems following the traditional waterfall process, to aODC, or our adapted ODC for agile development, and demonstrates its ability to provide valuable in-process feedback for a semiconductor software using data that is normally available from the agile development process in a small company. To assess the impact of aODC, 1) we first define our defect and quality metrics, including total defect count, in-field defects discovered by customers, defect distribution, product reliability, and reliability growth; 2) we then quantify the baseline using these metrics for the early versions of this software system prior to the deployment of aODC; and 3) lastly, we quantify the quality improvement using the same metrics after aODC deployment. The comparison results show: 1) a more than 50% reduction in total defects and a 16% reduction of defects found by customers; 2) a significantly higher share of defects discovered in the early part of the process by the developers, at 38%, up from 24.7% in the baseline, and a significantly lower share of defects discovered later by the testers, at 46.5%, down from 66.3% in the baseline; and 3) a higher reliability, with a success rate of 0.914 compared to 0.884 in the baseline, and a more significant reliability growth, quantified by the purification level of 0.99 as compared to 0.91 in the baseline. These results demonstrate that aODC, our adapted ODC to the agile development environment, offers valuable early in-process feedback leading to quantifiable quality improvement.
Time-series data are widely used in fields such as industrial manufacturing,meteorology,electric power,and vehicles,which has spurred the development of time-series database management systems.More and more database systems are migrating to the cloud,and the architecture of end-cloud collaboration is becoming more common,leading to increasingly large data scales to be processed.In scenarios such as end-cloud collaboration and massive time series,a large number of short time series are generated due to short synchronization cycles and frequent data flushing,among other reasons,presenting new challenges to database systems.Efficient data management and compression methods can significantly improve storage performance,enabling database systems to handle the storage of massive time series.Apache TsFile is a columnar storage file format specifically designed for time series scenarios,playing an important role in database management systems such as Apache IoTDB.This study elaborates on the group compression and merging methods used in Apache TsFile to address scenarios with a large number of short time series,especially in application scenarios with a vast number of time series such as the Industrial Internet of Things.This group compression method fully considers the data characteristics in the short time series scenario.Through device grouping,it improves metadata utilization,reduces file index size,decreases short time series,and significantly improves compression effectiveness.After validation with real-world datasets,the proposed grouping method shows significant improvements in compression effect,reading,writing,file merging,and other aspects,enabling better management of TsFiles in short time series scenarios.
Databases are important foundational components in computer services.However,performance anomalies may occur during their operation,affecting business service quality.How to diagnose performance anomalies in databases has become a hot issue in industry and academia.Recently,a series of automated database anomaly diagnosis methods have been successively proposed.They analyze the runtime status of the database and determine the overall database anomaly types.However,with the continuous expansion of data scale,distributed databases are becoming an increasingly popular solution in the industry.In a distributed database,which is composed of multiple nodes,existing anomaly diagnosis methods struggle to effectively locate node anomalies,fail to identify compound anomalies across multiple nodes,and are unable to perceive the complex performance influence relationships between nodes,lacking effective diagnostic capabilities.To address these challenges,this study proposes a distributed database diagnosis method for compound anomalies,named DistDiagnosis.It models the anomalous state of distributed databases using a Compound Anomaly Graph,which not only represents anomalies at each node but also effectively captures the correlations between nodes.DistDiagnosis introduces a node correlation-aware root cause anomaly ranking method,effectively locating root cause anomalies according to the influence of nodes on the database.In this study,anomaly testing cases for various scenarios are constructed on OceanBase,a domestically developed distributed database.Experimental results show that DistDiagnosis outperforms other advanced baselines,achieving the AC@1,AC@3,and AC@5 values of 0.97,0.98,and 0.98.Compared to the second-best method,DistDiagnosis improves accuracy by up to 5.20%,5.45%,and 4.46%in each diagnostic scenario.
Single-master multi-slave is the mainstream architecture of cloud-native databases.In the cluster,slave nodes can share the read-only requests of the master node,while write requests are handled by the master node.Based on this,to further meet the demands of large-scale transaction expansion,some cloud databases attempt to implement multi-write transaction expansion.One possible approach to multi-write expansion is to introduce shared cache among computing nodes to support cross-node data access.For shared-cache database systems,the overhead of cross-node remote access is significantly higher than that of local access.Therefore,the design of cache protocol is a crucial factor that affects system performance and scalability.This study proposes two innovative improvements to the coherence protocol and implements PG-RAC,a shared-cache database,which supports multi-write transactions based on PostgreSQL.On one hand,PG-RAC proposes a new distributed chained routing strategy,which disperses routing information among computing nodes.Compared to the routing strategy that utilizes single-node directory management,it reduces the average transaction latency by approximately 20%.On the other hand,this study also enhances the duplicate page invalidation mechanism by separating invalidation operations from the transaction path,reducing the latency of the critical path in the transaction.Based on this,PG-RAC takes advantage of the characteristics of multi-version concurrency control(MVCC)and further proposes to delay the invalidation point of duplicate pages,which effectively improves cache utilization.TPC-C experimental results show that for a cluster with 4 compute nodes,the throughput is nearly 2 times that of PostgreSQL and 1.5 times that of the distributed database Citus.
Cloud-native databases,with advantages such as out-of-the-box functionality,elastic scalability,and pay-as-you-go,are currently a research hotspot in academia and industry.Currently,cloud-native databases only support"single writer and multiple readers",that is,read-write transactions are concentrated on a single read-write node,and read-only transactions are distributed to multiple read-only nodes.This limitation restricts the system's ability to process read-write transactions,making it difficult to meet the demands of write-intensive businesses.To this end,this study proposes the D3C(deterministic concurrency control cloud-native database)architecture.It breaks through the limitation of"single writer and multiple readers"and supports concurrency execution of read-write transactions on multiple read-write nodes by designing a cloud-native database transaction processing mechanism based on deterministic concurrency control.D3C splits transactions into sub-transactions and independently executes them on each node according to a predefined global order,ensuring serializability for transaction execution on multiple read-write nodes.Additionally,this study introduces mechanisms like asynchronous batch data persistence mechanisms based on multi-version to ensure transaction processing performance and proposes a consistency point-based fault recovery mechanism to achieve high availability.Experimental results show that D3C can achieve 5.1 times the performance of the"single writer and multiple readers"architecture in write-intensive scenarios while meeting the key requirements of cloud-native databases.
One of the most important features of multi-tenant databases in cloud environments is scalability.However,most elastic scaling techniques struggle to make effective scaling decisions for dynamically changing loads.If load changes can be predicted in advance,resource supply can be accurately adjusted.Given this,this study proposes a load-prediction-based elastic scaling method for multi-tenant databases.It includes a combined load prediction model and an elastic scaling strategy.The load prediction model combines the advantages of convolutional neural networks,long short-term memory networks and gated recurrent units.It can accurately forecast memory requirements of database clusters.Based on the prediction results,the elastic scaling strategy adjusts the number of virtual machines to ensure that resource supply remains within a reasonable range.Compared to existing methods,the combined load prediction model can reduce prediction errors by 8.7%to 21.8%and improve prediction fitting degree by 4.6%.Furthermore,this study improves the Bayesian optimization algorithm for hyperparameter tuning of the combined prediction model.The improved hyperparameter tuning model reduces errors by above 20%and improves fitting degree by 1.04%,which proves that it can well address the poor performance of Bayesian optimization in combined domains of discrete and continuous solutions.Compared to the most widely used scaling strategy in Kubernetes,the proposed elastic scaling method reduces response time by 8.12%and latency by 9.56%.It can avoid the latency and the waste of resources to a large extent.
The Regional Metrology Laboratory is the lone government institution that provides calibration services in Region IV-A or in CALABARZON. RML's objective is to guarantee the safety and fair trade of both local and international markets by calibrating measuring devices and instruments used by industry and the general public. RML is also ISO/IEC 17025 certified, demonstrating that its calibration services meet the international standard for testing and calibration laboratories. However, the complexity of the preparation of calibration certificates with the increasing number of clienteles becomes the bottleneck and cause of delay in the release of certificates. RML ++ is a system which is a management software solution that optimizes the processing of documentary requirements, from the receipt of customers' measuring devices and instruments to the issuance of calibration certificates. The system was developed using the Model-Driven Architecture (MDA) that is built on the principle of abstraction, modelling reuse, and patterns to help understand standard referencing and all the mathematical calculations of the three (3) calibration certificates namely the Single Range Electronic Balance, Thermohygrometer, and Test Weights. RML ++ was able to reduce the processing time of the calibration certificates from encoding to releasing with a systematic process of the mathematical calculation of uncertainty and selection of standards.
