Denial constraints (DCs) formalize integrity rules that keep data consistent across profiling and cleaning tasks. However, discovering DCs remains computationally expensive due to the exponential size of the predicate search space and the cost of maintaining large intermediate data structures. Existing software-based DC discovery algorithms rely on evidence set materialization, candidate enumeration, and minimality checks, resulting in superlinear runtime growth and unpredictable performance, which limits their scalability in practice. We present DCArray, an FPGA-based accelerator that performs DC discovery directly in hardware pipelines. DCArray encodes DC predicates as boolean patterns and evaluates them using highly parallel logical units, each operating within a single clock cycle. Boolean patterns are encoded in a hardware-friendly prefix-tree design, eliminating the need for intermediate evidence sets and reducing memory overhead and data movement. DCArray implements minimality checks at the circuit level, achieving predictable runtime independent of dataset distribution. Across real and synthetic datasets, DCArray achieves speedups of up to 1560x over DCFinder and up to 10x over ECP on 1M tuples, while efficiently utilizing HBM/PCIe. At the 10M-tuple scale, end-to-end performance is primarily limited by sustained host-device I/O bandwidth, making DCArray a practical alternative for production data-quality pipelines.

Compressed bitmap indexes are a means of executing queries efficiently while reducing memory usage. However, most existing bitmap formats are designed for CPUs and exhibit irregular control flow and memory access patterns that limit performance on GPUs. This paper presents HiPES, a GPU-native, chunk-based compressed bitmap scheme. Experimental results show that HiPES can achieve an 8 × increase in compression throughput relative to GPU-WAH in dense workloads, while maintaining competitive compression ratios. In sparse workloads, the efficiency degrades due to internal fragmentation, associated with the fixed-size chunks layout. When the chunk payload bit-width (w) is aligned with shared memory, the throughput is maximized. Additionally, for values of w around 16, we reach a balance in terms of efficiency in which both metadata overhead and internal fragmentation are minimized. This work establishes a basis for GPU-resident computational frameworks, allowing for the elimination of the overhead associated with decompression in high-scale filtering, Graph Neural Networks (GNN) pre-processing, and in-situ scientific analytics.

Denial constraints (DCs) are at the heart of maintaining data consistency. Automatically discovering DCs from the data is computationally expensive due to the large search space. In this ongoing work, we propose a hardware-accelerated design for the automatic discovery of DCs, implemented on FPGAs. Our design eliminates the need for intermediate data structures, thereby reducing memory requirements and boosting performance. In preliminary experiments on a real-world dataset, our hardware-accelerated design outperformed state-of-the-art software solutions by factors ranging from 7× to 110×. An earlier version of our hardware design earned 3rd place in the prestigious ACM SIGMOD Research Competition.

Publicly available datasets are subject to new versions, with each new version potentially reflecting changes to the data. These changes may involve adding or removing attributes, changing data types, and modifying values or their semantics. Integrating these datasets into a database poses a significant challenge: how to keep track of the evolving database schema while incorporating different versions of the data sources? This paper presents a statistical methodology to validate the integration of 12 years of open-access datasets from Brazil's School Census, with a new version of the datasets released annually by the Brazilian Ministry of Education (MEC). We employ various statistical tests to find matching attributes between datasets from a specific year and their potential equivalents in datasets from later years. The results show that by using the Kolmogorov-Smirnov test we can successfully match columns from different dataset versions in about 90

A visualização de violações de regras de qualidade de dados possui grande utilidade na limpeza de dados. Uma operação amplamente utilizada para essa visualização é a projeção das combinações de tuplas que violam as regras. No entanto, essa operação é custosa quando consideramos os formalismos estado-da-arte em limpeza de dados, como as restrições de negação. No pior caso, todas as combinações de pares de tuplas da tabela violam a regra, resultando em uma complexidade quadrática em relação ao número de registros. Este artigo apresenta e avalia experimentalmente diversas técnicas para a implementação eficiente da projeção de violações de restrições de negação.

An array database is a software that uses non-linear data structures to store and process multidimensional data, including images and time series. As multi-dimensional data applications are generally data-intensive, array databases can benefit from multi-processing systems to improve performance. However, when dealing with Non-Uniform Memory Access (NUMA) machines, the movement of massive amounts of data across NUMA nodes may result in significant performance degradation. This paper presents a mechanism for scheduling array database threads based on data movement patterns and performance monitoring information. Our scheduling mechanism uses non-cooperative game theory to determine the optimal thread placement. Threads act as decision-makers selecting the best NUMA node based on each node’s remote memory access cost. We implemented and tested our mechanism on two array databases (Savime and SciDB), demonstrating improved NUMA-affinity. With Savime, we observed a maximum speedup of 1.64 × and a consistent reduction of up to 2.46 × in remote data access during subarray operations. With SciDB, we observed a speedup of up to 1.38 × and a reduction of 1.71 × in remote data access.

This paper introduces an approach for discovering denial constraints (DCs) to identify faults in transmission lines. However, the considerable volume of data in the studied scenario makes traditional DC discovery impractical due to lengthy execution times. We propose an alternative DC discovery approach that uses streaming windows to address this issue. Our experiments demonstrate that the DCs identified in pre-fault windows differ significantly from those in post-fault windows. This valuable insight enables us to detect faults autonomously, eliminating the need for human intervention (i.e., an unsupervised method). The experimental evaluation featuring diverse fault events reveals that our approach achieves fault detection with remarkable 100% accuracy.

Array database management systems (Array databases) are specialized software to streamline multi-dimensional data processing. Due to the data-hungry nature of multi-dimensional data applications (e.g., images and time series), array databases must ideally provide linear speedup when using a multi-processing system. However, when dealing with non-uniform memory access (NUMA) machines, array databases may require massive data movement for processing across the NUMA nodes resulting in severe performance impact. This paper investigates the performance impact of five well-known thread pinning strategies running array filtering operations in two different NUMA architectures. To identify the maximum potential performance improvement, we perform an in-width analysis evaluating all possible thread pinning combinations. Our experiments showed execution metrics of two array databases, namely SAVIME and SciDB. We observe a maximum speedup by 2.25× in SAVIME with a reduction in remote memory access by 5× . For SciDB, we observed a speedup of up to 5.83× and a reduction on the remote memory access by 4.1× . Our main finding is that well-known static thread pinning strategies only yield 48

Distributed database systems store and manipulate data on multiple machines. In these systems, the processing cost of query operations is mainly impacted by the data access latency between machines over the network. With recent technology advances in programmable network devices, the network switches provide new opportunities for dynamically managing the network topology, enabling the data processing on these devices with the same network throughput. In this paper, we explore the programmable network switches in query processing, evaluating the processing performance of a cost model in executing the hash join operation. We assume the storage of the hash table built from outer relation and the materialization of the join probing are made in switches using advanced matching techniques similar to package inspections enabled by Ternary Content-Addressable Memories (TCAM) or SRAM via hashing. Our results show that processing the hash join operation using network switches achieved the best results compared to traditional servers, with an average time reduction of 91.82% (Query-10 from TPC-H) and 96.52% (Query-11 from TPC-H).

O processamento paralelo é uma solução para melhorar o desempenho de consultas de banco de dados, reduzindo o tempo de resposta, e aumentando a vazão no processamento de consultas. Com a evolução de hardware surgiram novas tecnologias para o paralelismo. Uma delas é o uso de GPUs (Graphics Processing Units) para processamento de propósito geral. A GPU é uma unidade de processamento massivamente paralela com um número maior de núcleos executando em uma frequência menor comparado a CPU (Central Processing Unit). Neste contexto, este artigo apresenta um estudo comparativo do processamento de uma operação de consulta na GPU e CPU.

Transmission lines are crucial components of the electric power system and are exposed to several conditions that can disrupt the transmission of electrical power. In this scenario, a protection system must detect and classify a fault, for example, to enable the quick repair and restoration of a faulty line. This paper presents a method for fault type classification with two main characteristics: (1) independence of the sampling rate of the protection system; and (2) capacity to classify failures for different transmission lines than the one used to train the algorithm (generalization competence). Initially, the first post-fault cycle of the three-phase current signal is used to get the groundDetection feature, which aims to indicate the action or not of the ground in the failure. Next, maximum and minimum values are obtained from a pre-fault cycle of the current waveform to normalize the post-fault cycle by the MinMax technique, separately for each phase and individually for each example. Lastly, we get energy-based attributes together with maximum and minimum values from these normalized cycles for each phase to create the feature vector along with the groundDetection attribute. The extracted features are then used as input to the Random Forest algorithm to predict the fault type. The results demonstrate average accuracy higher than 99% for diversified simulated events for both characteristics previously mentioned. Our method also manifested the capacity to classify real fault events even being trained with synthetic examples with an accuracy of 96%.

Array Database Management Systems (Array databases) support query processing over multi-dimensional data. Data storage is implemented with non-linear structures to mitigate the shortcomings of the relational model when dealing with raw binary data, such as images, time series, and others. Due to data-hungry nature of multi-dimensional data applications, array databases must ideally provide a linear speedup when using a multi-processing system. When dealing with Non-Uniform Memory Access (NUMA) machines, array databases may require massive data movement across the nodes resulting in a severe performance impact, depending on the user operation. In this paper, we analyze the performance impact of the NUMA architecture in the SAVIME and SciDB array databases running five different well-known static thread pinning strategies. Our experiments showed a maximum speedup of these different strategies by 2.49x for SAVIME and up to 1.40x for SciDB. We also observed that these static strategies only yield 48% from the potential speedup (and 26% of the energy reduction), opening a new research topic.

Exploratory data analysis is the primary technique used by data scientists to extract knowledge from new data sets. This type of workload is composed of trial-and-error hypothesis-driven queries with a human in the loop. To keep up with the data scientist’s productivity, the system must be capable of answering queries in interactive times. Given that these queries are highly selective multidimensional queries, multidimensional indexes are necessary to ensure low latency. However, creating the appropriate indexes is not a given due to the highly exploratory and interactive nature of such human-in-the-loop scenarios.In this paper, we identify four main objectives that are desirable for exploratory data analysis workloads: (1) low overhead over the initial queries, (2) low query variance (i.e., high robustness), (3) predictable index convergence, and (4) low total workload time. Given that not all of them can be achieved at the same time, we present three novel incremental multidimensional indexing techniques that represent three sample points on a Pareto front for this multi-objective optimization problem. (a) The Adaptive KD-Tree is designed to achieve the lowest total workload time at the expense of a higher indexing penalty for the initial queries, lack of robustness, and unpredictable convergence. (b) The Progressive KD-Tree has predictable convergence and a user-defined indexing cost for the initial queries. However, total workload time can be higher than with Adaptive KD-Trees, and per-query time still varies. (c) The Greedy Progressive KD-Tree aims at full robustness at the expense of only improving the per-query cost after full index convergence.Our extensive experimental evaluation using both synthetic and real-life data sets and workloads shows that (a) the Adaptive KD-Tree reduces total workload time by up to a factor 2 compared to the state-of-the-art, (b) the Progressive KD-Tree achieves predictable convergence with up to one order of magnitude lower initial query cost, and (c) the Greedy Progressive KDTree exhibits the lowest query variance up to three orders of magnitude lower than the state-of-the-art.

Existing approaches for image-based Automatic Meter Reading (AMR) have been evaluated on images captured in well-controlled scenarios. However, real-world meter reading presents unconstrained scenarios that are way more challenging due to dirt, various lighting conditions, scale variations, in-plane and out-of-plane rotations, among other factors. In this work, we present an end-to-end approach for AMR focusing on unconstrained scenarios. Our main contribution is the insertion of a new stage in the AMR pipeline, called corner detection and counter classification, which enables the counter region to be rectified -- as well as the rejection of illegible/faulty meters -- prior to the recognition stage. We also introduce a publicly available dataset, called Copel-AMR, that contains 12,500 meter images acquired in the field by the service company's employees themselves, including 2,500 images of faulty meters or cases where the reading is illegible due to occlusions. Experimental evaluation demonstrates that the proposed system, which has three networks operating in a cascaded mode, outperforms all baselines in terms of recognition rate while still being quite efficient. Moreover, as very few reading errors are tolerated in real-world applications, we show that our AMR system achieves impressive recognition rates (i.e., > 99%) when rejecting readings made with lower confidence values.

This work makes contributions that reach central problems in connection with data dependencies. The first problem regards the discovery of dependencies of high expressive power. We introduce an efficient algorithm for the discovery of denial constraints: a type of dependency that has enough expressive power to generalize other important types of dependencies and to express complex business rules. The second problem concerns the application of dependencies for improving data consistency. We present a modification for traditional dependency discovery approaches that enables the dependency discovery algorithms to return reliable results even if they run on data containing some inconsistent records. Also, we present a system for detecting violations of dependencies efficiently. Our extensive experimental evaluation shows that our system is up to three orders-of-magnitude faster than state-of-the-art solutions, especially for larger datasets and massive numbers of dependency violations. The last contribution in this work regards the application of dependencies in query optimization. We present a system for the automatic discovery and selection of functional dependencies. Our experimental evaluation shows that our system selects relevant functional dependencies that help reducing the overall query response time for various types of query workloads.

Dependências de dados são fundamentais em importantes áreas do gerenciamento de dados, tais como qualidade, integração e análise de dados. Esta tese apresenta contribuições relevantes para importantes problemas relacionados à tais dependências. O primeiro é relacionado à detecção de dependências. Estudamos a detecção de restrições de negação, pois elas generalizam outros tipos de dependências, e conseguem expressar complexas regras de qualidade de dados. Apresentamos um algoritmo para a descoberta de restrições de negação e o avaliamos em uma variedade de cenários. Em comparação com soluções do estado da arte, nosso algoritmo melhora significativamente a eficiência da detecção em termos de tempo de execução. O segundo problema diz respeito à aplicação de dependências na melhoria da consistência de dados. Mostramos que é possível extrair evidências de conjuntos de dados para descobrir restrições que se mantêm aproximadamente e que identificam, com boa precisão e recuperação, inconsistências no conjunto de dados de entrada. Apresentamos, ainda, um sistema para detecção de erros baseados em restrições de negação que apresenta execuções até três ordens de magnitude mais rápidas do que as de soluções do estado da arte, especialmente para conjuntos de dados maiores e restrições complexas. Por fim, nossa última contribuição é sobre a aplicação de dependências na otimização de consultas. Apresentamos um sistema para a detecção e seleção automática de dependências funcionais baseado em representações extraídas das cargas de trabalho. Nossos experimentos mostram que a aplicação das dependências selecionadas pode reduzir o tempo de resposta geral de diversas consultas. As contribuições acima foram publicadas em veículos de renome nacional (SBBD) e internacional (PVLDB, CIKM e DEXA), e possibilitaram cooperação nacional com universidades federais (UFPR e UTFPR), bem como internacional com institutos de pesquisa (HPI-Alemanha e SnT-Luxemburgo).

O reconhecimento facial já faz parte na vida de muitos de nós. Grande parte dos smartphones atuais efetua o desbloqueio do aparelho utilizando a face como forma de identificar o dono do aparelho e proporcionar acesso aos dados. No entanto, ele também vem ganhando espaço para outros objetivos, principalmente em soluções corporativas como controle de acessos, validação de documentos e para compras online. A fim de aumentarmos a acurácia de nosso score biométrico, foi desenvolvido um modelo de cálculo de risco que leva em consideração o comportamento do consumidor, baseado em seu histórico de transações, visando a diminuição de fraudes

SQL-on-Hadoop engines such as Hive provide a declarative interface for processing large-scale data over computing frameworks such as Hadoop. The underlying frameworks contain a large number of configuration parameters that can significantly impact performance, but which are hard to tune. The problem of automatic parameter tuning has become a lively research area and several sophisticated tuning advisors have been proposed for Hadoop. In this paper, we conduct an experimental study to explore the impact of Hadoop parameter tuning on Hive. We reveal that the performance of Hive queries does not necessarily improve when using Hadoop-focused tuning advisors out-of-the-box, at least when following the current approach of applying the same tuning setup uniformly for evaluating the entire query. After extending the Hive query processing engine, we propose an alternative tuning approach and experimentally show how current Hadoop tuning advisors can now provide good and robust performance for Hive queries, as well as improved cluster resource utilization. We share our observations with the community and hope to create an awareness for this problem as well as to initiate new research on automatic parameter tuning for SQL-on-Hadoop systems.

A Software-defined wide-area networking (SD-WAN) é uma tecnologia que permite (re)programar dispositivos de rede via software. Nesse caso, os dispositivos de rede possuem uma flexibilidade maior na administração, permitindo a programabilidade da rede. Este artigo apresenta uma avaliação do processamento distribuído de operadores hash join de banco de dados em dispositivos de rede, a partir de uma análise baseada em modelo de custo.

O custo do processamento de uma consulta em sistemas de banco de dados distribuídos liga-se diretamente ao custo da transferência de dados na rede. A Software-Defined Wide Area Network (SD-WAN) e uma tecnologia que permite (re)programar dispositivos de rede via software. Sua programabilidade proporciona novas possibilidades para gerenciar topologias de forma dinâmica, possibilitando ainda o processamento de dados nesses dispositivos. Este artigo avalia o processamento distribuído da operação hash join em switches de rede, a partir de um modelo de custo. Os resultados mostram que o processamento dessas operações em switches de rede alcança desempenho comparável ao processamento tradicional em servidores com um trafego de dados similar.