Buildings consume over 40% of the total energy in modern societies, and improving their energy efficiency can significantly reduce our energy footprint. In this article, we present WattScale, a data-driven approach to identify the least energy-efficient buildings from a large population of buildings in a city or a region. Unlike previous methods such as least-squares that use point estimates, WattScale uses Bayesian inference to capture the stochasticity in the daily energy usage by estimating the distribution of parameters that affect a building. Further, it compares them with similar homes in a given population. WattScale also incorporates a fault detection algorithm to identify the underlying causes of energy inefficiency. We validate our approach using ground truth data from different geographical locations, which showcases its applicability in various settings. WattScale has two execution modes—(i) individual and (ii) region-based, which we highlight using two case studies. For the individual execution mode, we present results from a city containing >10,000 buildings and show that more than half of the buildings are inefficient in one way or another indicating a significant potential from energy improvement measures. Additionally, we provide probable cause of inefficiency and find that 41%, 23.73%, and 0.51% homes have poor building envelope, heating, and cooling system faults, respectively. For the region-based execution mode, we show that WattScale can be extended to millions of homes in the U.S. due to the recent availability of representative energy datasets.
Short-term rainfall forecasting plays a critical role in meteorology, hydrology, and other related areas. Currently, data-driven approaches have made considerable progress in rainfall forecasting. However, these approaches suffer from the following major drawbacks. First, they do not accommodate a reasonable and effective model to mathematically represent the complete changes of a rainfall block. Second, scale division rainfall forecasting is overlooked in existing literature. Third, generalization is not well validated in these approaches. To address these issues, we propose a novel Regional Scale Division Forecasting model using attention mechanism and long short term memory network (RSDF-AM-LSTM) for short-term and scale-division rainfall forecasting. The forecasting model can take full account of the regional characteristics of the rainfall blocks. It is established by employing a method similar to image evolution. The approach also provides a reasonable and effective model to formulate the complete changes of rainfall blocks, including newborn, splitting, strengthening, weakening, and merging. A deep learning algorithm based on AM-LSTM is proposed to forecast the change of rainfall block. AM-LSTM describes temporal and spatial association among four parallel LSTMs through AM. The rainfall data used in our experiments are obtained from 3,200 meteorological stations in and around China. The experimental results show that RSDF-AM-LSTM outperforms two popular atmospheric models and other traditional machine learning approaches.
Measures of public opinion on economic matters are vital in creating accurate official statistics needed to design appropriate policy interventions and shape private investment decisions. This is traditionally done using public opinion polls and representative surveys. However, this process is time and money intensive and currently suffers from reduced public participation. As a result, official statistics are usually delayed and are not frequently available at the resolution needed for better policy interventions. Hence, researchers have looked into anonymized digital data to continuously sense information about public behaviors, especially those related to consumer sentiment index and unemployment insurance claims. However, past studies relied on linear models with simplistic assumptions and thus provided limited extrapolatory power and no insights as to why these predictive methods work. Worryingly, the strong correlations reported in these studies disappeared when the original models were tested with newer social media data. We propose a novel interpretable machine learning model, called Group Additive Gaussian Processes, to provide accurate and near real-time estimates of economic indicators about public behaviors using social media data, along with insights into the model behavior. Our model exploits the underlying structure in data and encodes interpretability in the modeling framework. It is based on Gaussian Process regression, which provides a robust non-parametric Bayesian learning framework that produces calibrated uncertainty measures along with its predictions. A key challenge in the learning task is learning from limited training data. We demonstrate how our model not only learns but also generalizes well in these scenarios. Through extensive evaluation we show how our model performs on two important indicators of economic health-consumer confidence index and unemployment insurance claims data. Further, we demonstrate how our model can reduce the need to conduct surveys by producing highly accurate and frequent estimates in between the surveying periods.
Air pollution is one of the major concerns in global urbanization. Data science can help to understand the dynamics of air pollution and build reliable statistical models to forecast air pollution levels. To achieve these goals, one needs to learn the statistical models which can capture the dynamics from the historical data and predict air pollution in the future. Furthermore, the large size and heterogeneity of today’s big urban data pose significant challenges on the scalability and flexibility of the statistical models. In this work, we present a scalable belief updating framework that is able to produce reliable predictions, using over millions of historical hourly air pollutant and meteorology records. We also present a non-parametric approach to learn the statistical model which reveals interesting periodical dynamics and correlations of the dataset. Based on the scalable belief update framework and the non-parametric model learning approach, we propose an iterative update algorithm to accelerate Gaussian process, which is notorious for its prohibitive computation with large input data. Finally, we demonstrate how to integrate information from heterogeneous data by regarding the beliefs produced by other models as the informative prior. Numerical examples and experimental results are presented to validate the proposed method.
Rapidly developing location acquisition technologies provide a powerful tool for understanding and predicting human mobility in cities, which is very significant for urban planning, traffic regulation, and emergency management. However, with the existing methodologies, it is still difficult to accurately predict millions of peoples’ mobility in a large urban area such as Tokyo, Shanghai, and Hong Kong, especially when collected data used for model training are often limited to a small portion of the total population. Obviously, human activities in city are closely linked with point-of-interest (POI) information, which can reflect the semantic meaning of human mobility. This motivates us to fuse human mobility data and city POI data to improve the prediction performance with limited training data, but current fusion technologies can hardly handle these two heterogeneous data. Therefore, we propose a unique POI-embedding mechanism, that aggregates the regional POIs by categories to generate an artificial POI-image for each urban grid and enriches each trajectory snippet to a four-dimensional tensor in an analogous manner to a short video. Then, we design a deep learning architecture combining CNN with LSTM to simultaneously capture both the spatiotemporal and geographical information from the enriched trajectories. Furthermore, transfer learning is employed to transfer mobility knowledge from one city to another, so that we can fully utilize other cities’ data to train a stronger model for the target city with only limited data available. Finally, we achieve satisfactory performance of human mobility prediction at the citywide level using a limited amount of trajectories as training data, which has been validated over five urban areas of different types and scales.
Urban systems are characterized by complexity and dynamicity. Data-driven simulations represent a promising approach in understanding and predicting complex dynamic processes in the presence of shifting demands of urban systems. Yet, today’s silo-based, de-coupled simulation engines fail to provide an end-to-end view of the complex urban system, preventing informed decision-making. In this article, we present DataStorm to support integration of existing simulation, analysis and visualization components into integrated workflows. DataStorm provides a flow engine, DataStorm-FE , for coordinating data and decision flows among multiple actors (each representing a model, analytic operation, or a decision criterion) and enables ensemble planning and optimization across cloud resources. DataStorm provides native support for simulation ensemble creation through parameter space sampling to decide which simulations to run, as well as distributed instantiation and parallel execution of simulation instances on cluster resources. Recognizing that simulation ensembles are inherently sparse relative to the potential parameter space, we also present a density-boosting partition-stitch sampling scheme to increase the effective density of the simulation ensemble through a sub-space partitioning scheme, complemented with an efficient stitching mechanism that leverages partial and imperfect knowledge from partial dynamical systems to effectively obtain a global view of the complex urban process being simulated.
We propose PASS , a O ( n ) algorithm for data reduction that is specifically aimed at preserving the semantics of time series data visualization in the form of line chart. Visualization of large trend line data is a challenge and current sampling approaches do produce reduction but result in loss of semantics and anomalous behavior. We have evaluated PASS using seven large and well-vetted datasets (Taxi, Temperature, DEBS challenge 2012-2014 dataset, New York Stock Exchange data, and Integrated Surface Data) and found that it has several benefits when compared to existing state-of-the-art time series data reduction techniques. First, it can preserve the semantics of the trend. Second, the visualization quality using the reduced data from PASS is very close to the original visualization. Third, the anomalous behavior is preserved and can be well observed from the visualizations created using the reduced data. We have conducted two user surveys collecting 3,000+ users’ responses for visual preference as well as perceptual effectiveness and found that the users prefer PASS over other techniques for different datasets. We also compare PASS using visualization metrics where it outperforms other techniques in five out of the seven datasets.
During the past 15 years, automatic text scaling has become one of the key tools of the Text as Data community in political science. Prominent text-scaling algorithms, however, rely on the assumption that latent positions can be captured just by leveraging the information about word frequencies in documents under study. We challenge this traditional view and present a new, semantically aware text-scaling algorithm, SemScale , which combines recent developments in the area of computational linguistics with unsupervised graph-based clustering. We conduct an extensive quantitative analysis over a collection of speeches from the European Parliament in five different languages and from two different legislative terms, and we show that a scaling approach relying on semantic document representations is often better at capturing known underlying political dimensions than the established frequency-based (i.e., symbolic) scaling method. We further validate our findings through a series of experiments focused on text preprocessing and feature selection, document representation, scaling of party manifestos, and a supervised extension of our algorithm. To catalyze further research on this new branch of text-scaling methods, we release a Python implementation of SemScale with all included datasets and evaluation procedures.
The core challenge of steganography is always how to improve the hidden capacity and the concealment. Most current generation-based linguistic steganography methods only consider the probability distribution between text characters, and the emotion and topic of the generated steganographic text are uncontrollable. Especially for long texts, generating several sentences related to a topic and displaying overall coherence and discourse-relatedness can ensure better concealment. In this article, we address the problem of generating coherent multi-sentence texts for better concealment, and a topic-aware neural linguistic steganography method that can generate a steganographic paragraph with a specific topic is present. We achieve a topic-controllable steganographic long text generation by encoding the related entities and their relationships from Knowledge Graphs. Experimental results illustrate that the proposed method can guarantee both the quality of the generated steganographic text and its relevance to a specific topic. The proposed model can be widely used in covert communication, privacy protection, and many other areas of information security.
The world indicators released by the World Bank or other organizations usually give the basic public knowledge about the world. However, separate and static index lacks the complex interplay among different indicators and thus cannot help us have an overall understanding of the world. To this end, we study the world indicators from a different angle. Firstly, we discover that there exist correlations between indicators either from a static view or from a dynamic view. Moreover, taking the trade and diplomatic relationships into consideration, we construct a multi-relational network to depict the interactions between different countries, and propose a Multiple Relations to Vector (MR2vec) model to study world indicators from a network perspective. The experimental results show the changes of world indicators are predictable with the proposed model, and our proposed MR2vec has wide adaptability in predicting multi-relation networks.
There are numerous real-world problems where a user must make decisions under uncertainty. For the problem of influence maximization on a social network, for example, the user must select a set of K influencers who will jointly have a large influence on many users. With the lack of prior knowledge about the diffusion process or even topological information, this problem becomes quite challenging. This problem can be cast as a combinatorial bandit problem, where the user can repeatedly choose a candidate set of K out of N arms at each time, with an aim to achieve an efficient trade-off between exploration and exploitation. In this work, we present the first combinatorial bandit algorithm for which the only feedback is a non-linear reward of the selected K arms. No other feedback is needed. In the context of influence maximization, this means no feedback in the form of which nodes or edges were activated needs to be available, just the amount of influence. The novel algorithm we propose, CMAB-SM, is based on a divide-and-conquer strategy. It is computationally and storage efficient. Over a time horizon T , the proposed algorithm achieves a regret bound of Õ( K 1/2 N 1/3 T 2/3 ). This bound is sub-linear in all of the parameters: T , N , and K . We empirically demonstrate our algorithm’s performance using the applications of influence maximization and product cross-selling. For influence maximization, we provide experiments on real-world social networks, showing that the proposed CMAB algorithm outperforms bandit-specific and social-influence-domain-specific algorithms in terms of empirical run-time and expected influence. For product cross-selling, we also demonstrate that the proposed CMAB algorithm outperforms considered baselines on synthetic data.
Class imbalance can adversely affect the performance of machine learning for prediction and classification. One approach to address the class imbalance problem is synthetic minority oversampling. Oversampling approaches can be broadly categorized as either being structural or statistical in nature. Structural approaches generally have the advantage of identifying and oversampling those minority data points that best facilitate class separation, while statistical approaches model the underlying distribution from which the minority samples can be drawn. In this article, we formulate a distance-based approach that generates samples by both modeling the underlying minority class distribution and by geometrically considering those borderline samples entangled in the majority class. We demonstrate the efficacy of our approach operating on the Class-Imbalance data set from UCI by comparing its mean accuracy, AUC and F 1 -score performance against both statistical and structural synthetic minority oversampling methods.
Due to its nature of learning from dynamic interactions and planning for long-run performance, Reinforcement Learning (RL) has attracted much attention in Interactive Recommender Systems (IRSs). However, most of the existing RL-based IRSs usually face large discrete action space problem, which severely limits their efficiency. Moreover, data sparsity is another problem that most IRSs are confronted with. The utilization of recommendation-related textual knowledge can tackle this problem to some extent, but existing RL-based recommendation methods either neglect to combine textual information or are not suitable for incorporating it. To address these two problems, in this article, we propose a T ext-based deep R einforcement learning framework using self-supervised G raph representation for I nteractive R ecommendation (TRGIR). Specifically, we leverage textual information to map items and users into a same feature space by a self-supervised embedding method based on the graph convolutional network, which greatly alleviates data sparsity problem. Moreover, we design an effective method to construct an action candidate set, which reduces the scale of the action space directly. Two types of representative reinforcement learning algorithms have been applied to implement TRGIR. Since the action space of IRS is discrete, it is natural to implement TRGIR with Deep Q-learning Network (DQN). In the TRGIR implementation with Deep Deterministic Policy Gradient (DDPG), denoted as TRGIR-DDPG, we design a policy vector, which can represent user’s preferences, to generate discrete actions from the candidate set. Through extensive experiments on three public datasets, we demonstrate that TRGIR-DDPG achieves state-of-the-art performance over several baselines in a time-efficient manner.
Deep learning has achieved great success in various areas and its success is closely linked to the availability of massive data. But in general, a large dataset could include sensitive data and therefore the model should have the capability to avoid privacy leakage. To achieve this aim, many works apply the famous privacy framework named differential privacy into deep learning to preserve privacy. In this article, we propose a novel perturbed iterative gradient descent optimization (PIGDO) algorithm and prove that this algorithm satisfies the differential privacy. Besides, we propose a modified moments accountant (MMA) method to conduct the privacy analysis and obtain a tighter bound of privacy loss compared with the original moments accountant method. A number of experiments demonstrate that our optimization algorithm can not only improve the model accuracy and training speed, but also achieve better privacy guarantees over the state-of-the-art algorithm while reaching the equivalent accuracy. We provide codes for all of our experiments in https://github.com/CGCL-codes/DPDLIGDO.git .
Tensor network as an effective computing framework for efficient processing and analysis of high-dimensional data has been successfully applied in many fields. However, the performance of traditional tensor networks still cannot match the strong fitting ability of neural networks, so some data processing algorithms based on tensor networks cannot achieve the same excellent performance as deep learning models. To further improve the learning ability of tensor network, we propose a quantized tensor neural network in this article (QTNN), which integrates the advantages of neural networks and tensor networks, namely, the powerful learning ability of neural networks and the simplicity of tensor networks. The QTNN model can be further regarded as a generalized multilayer nonlinear tensor network, which can efficiently extract low-dimensional features of the data while maintaining the original structure information. In addition, to more effectively represent the local information of data, we introduce multiple convolution layers in QTNN to extract the local features. We also develop a high-order back-propagation algorithm for training the parameters of QTNN. We conducted classification experiments on multiple representative datasets to further evaluate the performance of proposed models, and the experimental results show that QTNN is simpler and more efficient while compared to the classic deep learning models.
The first age of Big Data started roughly ten years ago. It has had an enormous impact in many fields of science. It underlies the rapid development of data-driven applications and gives rise to many innovative data processing systems. Ten years on, Big Data is entering a new generation. In particular, data is being used at a much larger, global scale. Furthermore, there is a trend of multiple data owners coming together to perform collaborative data analytics, and many datadriven business decisions are made based on statistical analytics from multi-source, multimodal, and worldwide data. The new generation of Big Data opens the door for innovative data-driven applications that are not possible even in the early age of Big Data. However, the new scale, both in terms of the data and the number of participants, brings significant challenges ranging from secure data sharing to federated data analytics. At the same time, emerging technologies such as 5G, AI, and blockchains demand high-performance, scalable, and secure data management. It is therefore crucial to have new theories, algorithms, and systems, for future applications that make various trade-offs between security, performance, and data quality, in this new age of Big Data. This special issue aims to publish work on a variety of data science technologies including statistical theory, data management, data mining, and machine learning, which realize the potentials of next-generation Big Data. This special issue received five high-quality submissions, and three of them were accepted. The topics of the accepted articles are briefly introduced below. The article titled “Hierarchical Satellite System Graph for Approximate Nearest Neighbor Search on Big Data” presents a hierarchical method to build the Monotonic Search Networks to solve the approximate nearest neighbor search problem. The proposed index and search algorithms can be deployed in a distributed manner, effectively decreasing the search steps and reducing the computational cost over large-scale data. The article titled “Quantized Tensor Neural Network” introduces a quantized tensor neural network to improve the learning ability of the tensor network. The proposed method effectively integrates the high-order convolution operations into the non-linear tensor network to learn local features of high-dimensional data. A high-order error backpropagation algorithm is further developed to optimize the parameters in tensor networks. The proposed method can learn hidden features efficiently with fewer parameters on image classification tasks. The article titled “Differentially Private Deep Learning with Iterative Gradient Descent Optimization” studies the problem of privacy preservation with deep learning. It proposes a novel perturbed iterative gradient descent optimization algorithm which satisfies the differential privacy and achieves better model accuracy. A modified moments accountant method is introduced to get the tighter bound of privacy loss compared with the existing privacy accounting methods.
The failure of obtaining employment could lead to serious psychosocial outcomes such as depression and substance abuse, especially for college students who may be less cognitively and emotionally mature. In addition to academic performance, employers’ unconscious biases are a potential obstacle to graduating students in becoming employed. Thus, it is necessary to understand the nature of such unconscious biases to assist students at an early stage with personalized intervention. In this paper, we analyze the existing bias in college graduate employment through a large-scale education dataset and develop a framework called SUMMER (bia S -aware grad U ate e M ploy ME nt p R ediction) to predict students’ employment status and employment preference while considering biases. The framework consists of four major components. Firstly, we resolve the heterogeneity of student courses by embedding academic performance into a unified space. Next, we apply a Wasserstein generative adversarial network with gradient penalty (WGAN-GP) to overcome the label imbalance problem of employment data. Thirdly, we adopt a temporal convolutional network to comprehensively capture sequential information of academic performance across semesters. Finally, we design a bias-based regularization to smooth the job market biases. We conduct extensive experiments on a large-scale educational dataset and the results demonstrate the effectiveness of our prediction framework.
Approximate nearest neighbor search is a classical problem in data science, which is widely applied in many fields. With the rapid growth of data in the real world, it becomes more and more important to speed up the nearest neighbor search process. Satellite System Graph (SSG) is one of the state-of-the-art methods to solve the problem. However, with the further increase of the data scale of problems, SSG still needs a considerable amount of time to finish the search due to the limitation of step length and start point locations. To solve the problem, we propose Hierarchical Satellite System Graph (HSSG) and present its index algorithm and search algorithm. The index process can be distributed deployed due to the good parallelism of our designed hierarchical structure. The theoretical analysis reveals that HSSG decreases the search steps and reduces the computational cost and reduces the search time by searching on the hierarchical structure with a similar indexing time compared with SSG, hence reaches a better search efficiency. The experiments on multiple datasets present that HSSG reduces the distance computations, accelerates the search process, and increases the search precision in the real tasks, especially under the tasks with large scale and crowded distributions, which presents a good application prospect of HSSG.
Economic-wise, a common goal for companies conducting marketing is to maximize the return revenue/profit by utilizing the various effective marketing strategies. Consumer behavior is crucially important in economy and targeted marketing, in which behavioral economics can provide valuable insights to identify the biases and profit from customers. Finding credible and reliable information on products’ profitability is, however, quite difficult since most products tend to peak at certain times w.r.t. seasonal sales cycles in a year. On-Shelf Availability (OSA) plays a key factor for performance evaluation. Besides, staying ahead of hot product trends means we can increase marketing efforts without selling out the inventory. To fulfill this gap, in this paper, we first propose a general profit-oriented framework to address the problem of revenue maximization based on economic behavior, and compute the O n-shelf P opular and most P rofitable P roducts (OPPPs) for the targeted marketing. To tackle the revenue maximization problem, we model the k-satisfiable product concept and propose an algorithmic framework for searching OPPP and its variants. Extensive experiments are conducted on several real-world datasets to evaluate the effectiveness and efficiency of the proposed algorithm.
With the dramatic improvements in both the capability to collect personal data and the capability to analyze large amounts of data, increasingly sophisticated and personal insights are being drawn. These insights are valuable for clinical applications but also open up possibilities for identification and abuse of personal information. In this article, we survey recent research on classical methods of privacy-preserving data mining. Looking at dominant techniques and recent innovations to them, we examine the applicability of these methods to the privacy-preserving analysis of clinical data. We also discuss promising directions for future research in this area.
