The large availability of tabular Open Data sources with hundreds of attributes and relations makes the query development a difficult task, where analytic queries are common. When writing such queries, often called SPJG (Select-Project-Join-GroupBy), it is necessary to understand a data model and to write JOIN operations. The most common approach is to use business intelligence frameworks, or recent solutions based on keywords or examples. However, they require the utilization of specific applications and there is a lack of support for web-based APIs. We present a solution that eases the task of query development for tabular Open Data analytics through an API, using a simplified query representation where it is not allowed to specify the data relations, and consequently neither the joins over them, called Relation-Free Query. We define a single virtual schema that captures the database structure, which allows the use of relation-free queries in existent DBMS's. The concrete queries are exposed by a RESTful API, which is then translated into a database query language using known query generation solutions. The API is available as a microservice. We present a case study to describe solution, using a real world scenario to query in an integrated database of several Brazilian open databases with hundreds of attributes.

While several public institutions provide its data openly, the effort required to access, integrate and query this data is too high, reducing the amount of possible dataset users. The Blended Integrated Open Data (BIOD) project has as objective to ease the access to public Open Data. It integrates and makes available more than 300Gb of data, containing billions of records from different Open Data Sets, allowing to query over them, and thus to retrieve related information from originally disconnected data sets. This paper presents the set of open data available, how to access it and how produce new compatible data to improve the existing data set.

Desde 2012, com o início da vigência da Lei de Acesso a Informação, a quantidade de dados disponíveis cresceu de tal forma que a dificuldade passou a ser não o acesso em si, mas o tratamento e interpretação das informações disponibilizadas. Nesse contexto o SIMTransparência foi desenvolvido, com o objetivo de proporcionar uma visualização alternativa ao Portal da Transparência do Governo Federal, exibindo dados de várias bases em uma interface intuitiva e dinâmica, capaz de auxiliar o gestor e cidadão comum a melhor compreender e fiscalizar as despesas do governo federal. Esse artigo descreve as tecnologias envolvidas no desenvolvimento do SIMTransparência e formas de replicar sua base de dados.

BACKGROUND:Stochastic mapping is frequently used in comparative biology to simulate character evolution, enabling the probabilistic computation of statistics such as number of state transitions along a tree and distribution of states in its internal nodes. Common implementations rely on Continuous-time Markov Chain simulations whose parameters are difficult to adjust and subjected to inherent inaccuracy. Thus, researchers must run a large number of simulations in order to obtain adequate estimates. Although execution time tends to be relatively small when simulations are performed on a single tree assumed to be the "true" topology, it may become an issue if analyses are conducted on several trees, such as the ones that make up posterior distributions obtained via Bayesian phylogenetic inference. Working with such distributions is preferable to working with a single tree, for they allow the integration of phylogenetic uncertainty into parameter estimation. In such cases, detailed character mapping becomes less important than parameter integration across topologies. Here, we present an R-based implementation (SFREEMAP) of an analytical approach to obtain accurate, per-branch expectations of numbers of state transitions and dwelling times. We also introduce an intuitive way of visualizing the results by integrating over the posterior distribution and summarizing the parameters onto a target reference topology (such as a consensus or MAP tree) provided by the user.RESULTS:We benchmarked SFREEMAP's performance against make.simmap, a popular R-based implementation of stochastic mapping. SFREEMAP confirmed theoretical expectations outperforming make.simmap in every experiment and reducing computation time of relatively modest datasets from hours to minutes. We have also demonstrated that SFREEMAP returns estimates which were not only similar to the ones obtained by averaging across make.simmap mappings, but also more accurate, according to simulated data. We illustrate our visualization strategy using previously published data on the evolution of coloniality in scleractinian corals.CONCLUSION:SFREEMAP is an accurate and fast alternative to ancestral state reconstruction via simulation-based stochastic mapping.

The Brazilian government is maintaining several digital inclusion projects, providing computers and Internet connection to developing regions around the country. However, these projects can only succeed if they are constantly assessed; namely, the projects infrastructure deployment must be closely monitored and evaluated. In this paper, we introduce a system called SIMMC, which is currently monitoring and evaluating more than 4,500 computing devices from Brazilian digital inclusion projects. This system is innovative because, in addition to being used by the government for managing and expanding its projects, the collected data is also publicly available on a web page, allowing the citizens to follow the projects' deployment. We describe the SIMMC architecture, reporting some techniques used to optimize its data analysis processes, and describe how the information acquired and presented by the system has been used to enable public administration overhaul and improve efficiency on the project management, as well as its strategic use for security, theft, and defrauding.

The Brazilian government is maintaining several digital inclusion projects, providing computers and Internet connection to developing regions around the country. However, these projects can only succeed if they are constantly assessed; namely, the projects infrastructure deployment must be closely monitored and evaluated. In this paper, we introduce a system called SIMMC, which is currently monitoring and evaluating more than 4,500 computing devices from Brazilian digital inclusion projects. This system is innovative because, in addition to being used by the government for managing and expanding its projects, the collected data is also publicly available on a web page, allowing the citizens to follow the projects’ deployment. We describe the SIMMC architecture, reporting some techniques used to optimize its data analysis processes, and describe how the information acquired and presented by the system has been used to enable public administration overhaul and improve efficiency on the project management, as well as its strategic use for security, theft, and defrauding.

In this paper we focus on the aggregate query model implemented over NoSQL document-stores for read-mostly data bases. We discuss that the aggregate query model can be a good fit for read-mostly databases if the following design requirements are met: on-line time range queries, aggregates with predefined filters, frequent schema evolution and no ad-hoc. In our model, we present a composite object schema implementation over NoSQL document-stores, in which data associations are nested in a document under the same search key. We present the design choices to obtain a model adapted to our needs. Our schema is inspired by the star schema of Data Warehouses to reduce accessing data associations in many different documents and computing aggregates within the same composite. We present performance results of our empirical study over a 300 million records database that serves in production for the Ministry of Communications of Brazil. Results show the performance gains and penalties of our star composite schema when compared to the traditional multidimensional schema.

This paper briefly presents a model for monitoring a large, heterogeneous and geographically scattered computer park. The data collection is performed by a software agent. The collected data are sent to the central server over the Internet, and stored by the storage system. An on-line portal makes up the visualization system, featuring charts, reports, and other tools for assessing the state of the park. This system is currently monitoring circa 150,000 machines.