We consider a causal inference problem frequently encountered in online advertising systems, where a publisher (e.g., Instagram, TikTok) interacts repeatedly with human users and advertisers by sporadically displaying to each user an advertisement selected through an auction. Each treatment corresponds to a parameter value of the advertising mechanism (e.g., auction reserve-price), and we want to estimate through experiments the corresponding long-term treatment effect (e.g., annual advertising revenue). In our setting, the treatment affects not only the instantaneous revenue from showing an ad, but also changes each user's interaction-trajectory, and each advertiser's bidding policy – as the latter is constrained by a finite budget. In particular, each a treatment may even affect the size of the population, since users interact longer with a tolerable advertising mechanism. We drop the classical i.i.d. assumption and model the experiment measurements (e.g., advertising revenue) as a stopped random walk, and use a budget-splitting experimental design, the Anscombe Theorem, a Wald-like equation, and a Central Limit Theorem to construct confidence intervals for the long-term treatment effect.

Many classification performance metrics exist, each suited to a specific application. However, these metrics often differ in scale and can exhibit varying sensitivity to class imbalance rates in the test set. As a result, it is difficult to use the nominal values of these metrics to evaluate, compare and monitor classification performances, especially when imbalance rates vary. To address this problem, we introduce the outperformance standardization (OPS) function, a universal standardization method for confusion-matrix-based classification performance (CMBCP) metrics. It maps any given metric to a common scale of [0,1], while providing a clear and consistent interpretation. Specifically, the resulting OPS value (o-value) represents the percentile rank of the observed classification performance within a reference distribution of possible performances. This unified framework enables meaningful comparison and monitoring of classification performance across test sets with differing imbalance rates. We illustrate how o-values can be applied to a variety of commonly used classification performance metrics and demonstrate the utility and robustness of our method through experiments on real-world datasets spanning multiple classification applications.

The AIMD algorithm, which underpins the Transmission Control Protocol (TCP) for transporting data packets in communication networks, is perhaps the most successful control algorithm ever deployed. Recently, its use has been extended beyond communication networks, and successful applications of the AIMD algorithm have been reported in transportation, energy, and mathematical biology. A very recent development in the use of AIMD is its application in solving large-scale optimization and distributed control problems without the need for inter-agent communication. In this context, an interesting problem arises when multiple AIMD networks that are coupled in some sense (usually through a nonlinearity). The purpose of this note is to prove that such systems in certain settings inherit the ergodic properties of individual AIMD networks. This result has important consequences for the convergence of the aforementioned optimization algorithms. The arguments in the paper also correct conceptual and technical errors in [1].

The design and operation of Bittensor is a decentralized and anonymous system where actors are incentivized by rewards to provide utilities. To ensure that it is a fair game, utilities obtained by copying other participants should be identified and punished. Our first contribution is to apply a commitment scheme to address this free-rider problem. Under appropriate conditions, we show theoretically and empirically that a commitment scheme dissuades copying by reducing the rewards to the copier. In particular, this dissuasive power is a function of the duration between the commit- and reveal-steps. Our second contribution is to propose the liquid alpha solution to amplify the effect of the commitment scheme.

Shapley value attribution (SVA) is an increasingly popular explainable AI (XAI) method, which quantifies the contribution of each feature to the model's output. However, recent work has shown that most existing methods to implement SVAs have some drawbacks, resulting in biased or unreliable explanations that fail to correctly capture the true intrinsic relationships between features and model outputs. Moreover, the mechanism and consequences of these drawbacks have not been discussed systematically. In this paper, we propose a novel error theoretical analysis framework, in which the explanation errors of SVAs are decomposed into two components: observation bias and structural bias. We further clarify the underlying causes of these two biases and demonstrate that there is a trade-off between them. Based on this error analysis framework, we develop two novel concepts: over-informative and underinformative explanations. We demonstrate how these concepts can be effectively used to understand potential errors of existing SVA methods. In particular, for the widely deployed assumption-based SVAs, we find that they can easily be under-informative due to the distribution drift caused by distributional assumptions. We propose a measurement tool to quantify such a distribution drift. Finally, our experiments illustrate how different existing SVA methods can be over- or under-informative. Our work sheds light on how errors incur in the estimation of SVAs and encourages new less error-prone methods.

Shapley value attribution (SVA) is an increasingly popular Explainable AI (XAI) approach that has been widely used in many recent applied studies to gain new insights into the underlying information systems. However, most existing SVA methods are error-prone, providing biased or unreliable explanations that fail to correctly capture the informational dependencies between features and model outputs. These explanation errors can be decomposed into two components: 1) observation bias which stems from data sparsity and leads to over-informativeness; and 2) structural bias which stems from distributional assumptions and leads to under-informativeness. To alleviate these biases, in this paper, we propose a series of refinement methods that combine out-of-distribution (OOD) detection and importance sampling. In essence, our methods aim to rectify the distribution drift caused by distributional assumptions. We apply our refinement methods to two popular SVAs: the marginal SVA and the surrogate model-based SVA. Our extensive experiments show that the proposed methods significantly enhance the informativeness of both local and global Shapley value-based explanations.

Battery degradation in electric vehicles is detrimental and keeping track of the State of Health (SOH) of the battery is essential. Current battery SOH estimation techniques (such as monitoring voltage decrease with increasing cycles) have several drawbacks, such as having monotonically decreasing predictions in non-monotonic systems and susceptibility to noise effects. There is a simpler and more reliable technique by monitoring the charging times of the battery in every cycle. This method is proven to be accurate and to have low susceptibility to noise effects. Results show nearly identical results in scenarios with and without noise.

We introduce a new color-based fiducial marker system-RainbowTag (RT)-for detection and identification that is suitable for autonomous navigation due to robustness to varying lighting conditions, motion blur, partial occlusion and folding. This system uses cameras already present on the vehicles to complement spatial information estimated from other sensors (e.g., Global Positioning System, inertial measurement, radar). RT is composed of a fiducial marker design and its adapted detection algorithm. Numerous real-world experiments demonstrate that markers can be reliably detected in various lighting conditions, in the presence of large motion blur, and even when folded or partially occluded. In all test conditions, RT outperforms the fiducial markers Aruco and ChromaTag. Compared to other blur-resistant fiducials that are circularly symmetric [1], [2], RT has the advantage that it encodes orientation information. Our detection algorithm is powered by a novel color segmentation approach that carefully orchestrates information from the hue constant IPT, the perceptually uniform CIELAB, and the Bradford LMS cone response color spaces.

We consider a risk-aware multi-armed bandit framework with the goal of avoiding catastrophic risk. Such a framework has multiple applications in financial risk management. We introduce a new conditional value-at-risk (CVaR) estimation procedure combining extreme value theory with automated threshold selection by ordered goodness-of-fit tests, and we apply this procedure to a pure exploration best-arm identification problem under a fixed budget. We empirically compare our results with the commonly used sample average estimator of the CVaR, and we show a significant performance improvement when the underlying arm distributions are heavy-tailed.

Transformer-based language models can generate fluent text and be efficiently adapted across various natural language generation tasks. However, language models that are pretrained on large unlabeled web text corpora have been shown to suffer from degenerating toxic content and social bias behaviors, consequently hindering their safe deployment. Various detoxification methods have been proposed to mitigate language model toxicity; however, these methods struggle to detoxify language models when conditioned on prompts that contain specific social identities related to gender, race, or religion. In this study, we propose Reinforce-Detoxify, a reinforcement learning-based method for mitigating toxicity in language models. We address the challenge of safety in language models and propose a new reward model that can detect toxic content and mitigate unintended bias towards social identities in toxicity prediction. The experiments demonstrate that the Reinforce-Detoxify method for language model detoxification outperforms existing detoxification approaches in automatic evaluation metrics, indicating that our approach in language model detoxification is less prone to unintended bias toward social identities in generated content.

The unprecedented growth of demand for charging electric vehicles (EVs) calls for novel expansion solutions to today’s charging networks. Riding on the wave of the proliferation of sharing economy, Airbnb-like charger sharing markets open the opportunity to expand the existing charging networks without requiring costly and time-consuming infrastructure investments, yet the successful design of such markets relies on innovations at the interface between game theory, mechanism design, and large scale optimization. In this paper, we propose a price-based iterative double auction for charger sharing markets where charger owners rent out their under-utilized chargers to the charge-needing EV drivers. Charger owners and EV drivers form a two-sided market which is cleared by a price-based double auction. Chargers’ locations, availabilities, and unit time service costs as well as drivers’ time and location preferences are considered in the allocation and scheduling process. The goal is to compute social welfare maximizing schedules which benefit both charger owners and EV drivers and, in turn, ensure the continuous growth of the market. We prove that the proposed double auction is budget balanced and individually rational. In addition, results from our computational study show that the proposed auction achieves on average 94% efficiency compared with that of the optimal solutions and is suitable for a larger day-ahead charger sharing market setting in terms of running time.

As the harms of online toxic language become more apparent, countering online toxic behavior is an essential application of natural language processing. The first step in managing toxic language risk is identification, but algorithmic approaches have themselves demonstrated bias. Texts containing some demographic identity terms such as gay or Black are more likely to be labeled as toxic in existing toxic language detection datasets. In many machine learning models introduced for toxic language detection, non-toxic comments containing minority and marginalized community-specific identity terms were given unreasonably high toxicity scores. To address the challenge of bias in toxic language detection, we propose a two-step training approach. A pretrained language model with a multitask learning objective will mitigate biases in the toxicity classifier prediction. Experiments demonstrate that jointly training the pretrained language model with a multitask objective can effectively mitigate the impacts of unintended biases and is more robust to model bias towards commonly-attacked identity groups presented in datasets without significantly hurting the model's generalizability.

As online communities have grown, so has the ability to exchange ideas, which includes an increase in the spread of toxic language, including racism, sexual harassment, and other negative behaviors that are not tolerated in polite society. Hence, toxic language detection within online conversations has become an essential application of natural language processing. In recent years, machine learning approaches for toxic language detection have primarily focused on many researchers in academics and industries. However, in many of these machine learning models, non-toxic comments containing specific identity terms, such as gay, Black, Muslim, and Jewish, were given unreasonably high toxicity scores. In this research, we propose a new approach based on the domain adaptation language model and multi-task deep neural network to identify and mitigate this form of unintended model bias in online conversations. We use six toxic language detection and identification tasks to train the model to detect toxic contents and mitigate unintended bias in model prediction. We evaluate our model and compare it with other state-of-the-art deep learning models using specific performance metrics to measure the model bias. In detailed experiments, we show our approach can identify the toxic language in conversations with considerably more robustness to model bias towards commonly-attacked identity groups presented in online conversations in social media.

The conditional value-at-risk (CVaR) is a useful risk measure in fields such as machine learning, finance, insurance, energy, etc. When measuring very extreme risk, the commonly used CVaR estimation method of sample averaging does not work well due to limited data above the value-at-risk (VaR), the quantile corresponding to the CVaR level. To mitigate this problem, the CVaR can be estimated by extrapolating above a lower threshold than the VaR using a generalized Pareto distribution (GPD), which is often referred to as the peaks-over-threshold (POT) approach. This method often requires a very high threshold to fit well, leading to high variance in estimation, and can induce significant bias if the threshold is chosen too low. In this paper, we address this bias-variance tradeoff by deriving a new expression for the GPD approximation error of the CVaR, a bias term induced by the choice of threshold, as well as a bias correction method for the estimated GPD parameters. This leads to the derivation of a new CVaR estimator that is asymptotically unbiased and less sensitive to lower thresholds being used. An asymptotic confidence interval for the estimator is also constructed. In a practical setting, we show through experiments that our estimator provides a significant performance improvement compared with competing CVaR estimators in finite samples from heavy-tailed distributions.

In a federated setting, agents coordinate with a central agent or a server to solve an optimization problem in which agents do not share their information with each other. Wirth and his co-authors, in a recent paper, describe how the basic additive-increase multiplicative-decrease (AIMD) algorithm can be modified in a straightforward manner to solve a class of optimization problems for federated settings for a single shared resource with no inter-agent communication. The AIMD algorithm is one of the most successful distributed resource allocation algorithms currently deployed in practice. It is best known as the backbone of the Internet and is also widely explored in other application areas. We extend the single-resource algorithm to multiple heterogeneous shared resources that emerge in smart cities, sharing economy, and many other applications. Our main results show the convergence of the average allocations to the optimal values. We model the system as a non-homogeneous Markov chain with place-dependent probabilities. Furthermore, simulation results are presented to demonstrate the efficacy of the algorithms and to highlight the main features of our analysis.

Apparel landmarks are the functional key-points on the apparels that can be used for a more discriminative visual analysis of the apparel images. Such a framework can facilitate apparel alignment in displaying apparel images on the websites for recommendation systems, apparel image retrieval, apparel style transfer, trend analysis or help build a system to ensure dress code in a particular environment. However, challenges such as background clutter, human poses, scales apparel variation and lighting can render such a task difficult. We present a conceptually simple, flexible, and general framework for apparels' landmark detection that can also be simultaneously used for apparel detection. In addition to the position of the landmarks in the apparels, we also classify the landmarks as visible or occluded in the same framework. We perform all these tasks in parallel using multi-task learning. Our proposed convolutional neural network is end-to-end differentiable and simple to train, since all these tasks are performed on the same architecture without any additional parameters to learn. The fashion landmark detection task is similar to joint localization and detection problems like human pose estimation, hence our approach extends stacked hourglass architecture, originally proposed to solve human pose estimation. Over the past few years, many modifications have been proposed to improve this architecture. We also compare the performances of some of these different variations of stacked hourglass architectures. These architectures leverage both global and local features captured by the deep convolutional neural networks to better localize the apparel in the image as well as the landmarks in those apparels.

Reinforcement learning has become an important scheduling solution with many successes in markets with dynamic pricing options, e.g., electric vehicle charging in a deregulated electricity market. However, the highly-uncertain requests and partially-unknown individual preferences remain major challenges to effective demand responses in the user-centric environment. For charging stations who aim to maximize the long-term revenue in this fast-growing market, an accurate estimate of user's sensitivity, or acceptance, of the prices they offered to the potential customers is the key to the success of dynamic pricing. While most existing pricing schemes assume users will consistently follow stable patterns that are observable or inferrable by the charging service provider, it remains crucial to consider how users may be influenced by historic prices they have observed and react strategically to decide optimal charging demands that can maximize their utilities. To overcome this limitation, this paper presents a new framework based on reinforcement mechanism design to determine the optimal charging price in a mechanism design setting, which can optimize the long-term revenue of charging stations as well as the social welfare of users with private utility functions. Specifically, the strategic interaction between the station and users is modelled as a discrete finite Markov decision process, a Q-learning-based dynamic pricing mechanism is proposed to explore how price affects users' demands over a sequence of time. The experiments demonstrate that our pricing mechanism outperforms the predetermined time-of-use pricing in maximizing the long-term revenue of the charging station.

Developing a machine that can hold an engaging conversation with a human is one of the main challenges in designing a dialogue system in the field of natural language processing. Responses generated by neural conversational models with log-likelihood training methods tend to lack informativeness and diversity. We address the limitation of log-likelihood training in dialogue generation models, and we present the Reinforce Transformer decoder model, our new approach for training the Transformer decoder based conversational model, which incorporates proximal policy optimization techniques from re-inforcement learning with the Transformer decoder architecture. We specifically examine the use of our proposed model for multi-turn dialogue response generation in a real word human to a human dataset. To verify the effectiveness of our proposed framework, we evaluate our model on the Reddit dialogues data, which is a real word human to a human dataset. Experiments show that our proposed response generating model in a dialogue achieves significant improvement over recurrent sequence-to-sequence models and also the state of the art Transformer based dialogue generation models based on diversity and relevance evaluation metrics.

In several social choice problems, agents collectively make decisions over the allocation of multiple divisible and heterogeneous resources with capacity constraints to maximize utilitarian social welfare. The agents are constrained through computational or communication resources or privacy considerations. In this paper, we analyze the convergence of a recently proposed distributed solution that allocates such resources to agents with minimal communication. It is based on the randomized additive-increase and multiplicative-decrease (AIMD) algorithm. The agents are not required to exchange information with each other, but little with a central agent that keeps track of the aggregate resource allocated at a time. We formulate the time-averaged allocations over finite window size and model the system as a Markov chain with place-dependent probabilities. Furthermore, we show that the time-averaged allocations vector converges to a unique invariant measure, and also, the ergodic property holds.

Investing into a new product or service is a high-risk, high-return activity. This is best symbolized by the observation that the return over investment distribution of startups is a power law. Introduction of new products or services to the market might fail to generate profit even though there is a demand. Early adopters are also penalized, as they often pay a high price for something which will end up being cheaper, and might lose their warranty if the firm goes bankrupt. Innovation is slowed down. We propose to equally redistribute part of the generated profit at the end of a predefined time period to previous customers using Ethereum smart contract. Because customers are aware of the amount they would get back, their behaviors will change. The return over investment distribution and therefore the risk and return balance of the firms will also be affected. We formally define both a classic market and a market that is using our proposed system, and present an architecture to deploy such system. A preliminary numerical simulation is provided.