Designing service systems requires selecting among alternative configurations – choosing the best chatbot variant, the optimal routing policy, or the most effective quality control procedure. In many service systems, the primary evidence of performance quality is textual – customer support transcripts, complaint narratives, compliance review reports – rather than the scalar measurements assumed by classical optimization methods. Large language models (LLMs) can read such textual evidence and produce standardized quality scores, but these automated judges exhibit systematic biases that vary across alternatives and evaluation instances. Human expert review remains accurate but costly. We study how to identify the best service configuration with high confidence while minimizing expensive human audits, given that automated evaluation is cheap but biased. We formalize this as a sequential decision problem where a biased proxy score is observed for every evaluation, and a verified outcome can be acquired selectively at additional cost. We prove that LLM-only selection fails under arm-dependent bias, and that naive selective-audit estimators can be asymptotically biased. We develop an estimator combining proxy scores with inverse-propensity-weighted residuals and construct anytime-valid confidence sequences. Our algorithm, PP-LUCB, jointly decides which alternatives to evaluate and whether to request human audits, concentrating reviews where the LLM judge is least reliable. We prove correctness and establish instance-dependent cost bounds showing near-optimal efficiency. On a customer support ticket classification task, our algorithm correctly identifies the best model in 40/40 trials while achieving 90% audit cost reduction.

We study fixed-confidence best-arm identification (BAI) where a cheap but potentially biased proxy (e.g., LLM judge) is available for every sample, while an expensive ground-truth label can only be acquired selectively when using a human for auditing. Unlike classical multi-fidelity BAI, the proxy is biased (arm- and context-dependent) and ground truth is selectively observed. Consequently, standard multi-fidelity methods can mis-select the best arm, and uniform auditing, though accurate, wastes scarce resources and is inefficient. We prove that without bias correction and propensity adjustment, mis-selection probability may not vanish (even with unlimited proxy data). We then develop an estimator for the mean of each arm that combines proxy scores with inverse-propensity-weighted residuals and form anytime-valid confidence sequences for that estimator. Based on the estimator and confidence sequence, we propose an algorithm that adaptively selects and audits arms. The algorithm concentrates audits on unreliable contexts and close arms and we prove that a plug-in Neyman rule achieves near-oracle audit efficiency. Numerical experiments confirm the theoretical guarantees and demonstrate the superior empirical performance of the proposed algorithm.

Online platforms often expand their seller base to offer greater product variety and serve heterogeneous consumer preferences. However, a larger seller base can also intensify price competition among sellers and reduce platform revenue. Building on the literature on assortment reduction, we study whether platforms can mitigate price competition through a partitioned display policy, under which sellers are divided into distinct partitions, and each partition is matched to a portion of incoming customer traffic. We develop a Stackelberg game in which the platform chooses the display policy, and sellers subsequently set prices in response to the competitors they face within their assigned partition. The framework covers single-unit, finite-inventory, and infinite-inventory sellers. We characterize how the platform’s optimal display policy depends on demand and inventory. In finite-inventory settings, when demand is sufficiently high, full display is optimal because the demand-side benefit of showing the entire assortment dominates the price-competition benefit of partitioning. Under low or moderate demand, however, partitioned display can improve platform revenue by softening price competition. In addition, we develop an algorithm that effectively solves seller partitions, traffic allocation, and equilibrium prices. We also incorporate fairness constraints on seller outcomes and customer welfare, and show that partitioned display can remain revenue-improving under moderate fairness requirements. Finally, using Airbnb transaction data to calibrate a counterfactual marketplace environment, we illustrate the magnitude and drivers of the revenue–fairness trade-off under partitioned display.

Traditionally, firms have offered coupons to customer groups at predetermined discount rates. However, advancements in machine learning and the availability of abundant customer data now enable platforms to provide real-time customized coupons to individuals. In this study, we partner with Meituan, a leading shopping platform, to develop a real-time, end-to-end coupon allocation system that is fast and effective in stimulating demand while adhering to marketing budgets when faced with uncertain traffic from a diverse customer base. Leveraging comprehensive customer and product features, we estimate Conversion Rates (CVR) under various coupon values and employ isotonic regression to ensure the monotonicity of predicted CVRs with respect to coupon value. Using calibrated CVR predictions as input, we propose a Lagrangian Dual-based algorithm that efficiently determines optimal coupon values for each arriving customer within 50 milliseconds. We theoretically and numerically investigate the model performance under parameter misspecifications and apply a control loop to adapt to real-time updated information, thereby better adhering to the marketing budget. Finally, we demonstrate through large-scale field experiments and observational data that our proposed coupon allocation algorithm outperforms traditional approaches in terms of both higher conversion rates and increased revenue. As of May 2024, Meituan has implemented our framework to distribute coupons to over 100 million users across more than 110 major cities in China, resulting in an additional CNY 8 million in annual profit. We demonstrate how to integrate a machine learning prediction model for estimating customer CVR, a Lagrangian Dual-based coupon value optimizer, and a control system to achieve real-time coupon delivery while dynamically adapting to random customer arrival patterns.

Demand forecasting for new fashion products is particularly challenging due to the lack of historical data. Despite using the textual information companies have, only a few recent studies have investigated how to incorporate products' images as an additional source of information. In this paper, we investigate new methods to utilize both textual descriptions and visual representations of products' impact on the forecast of new products' sales in the absence of historical input features. Different models are examined to study the use of textual features, image features, and a combination of both for forecasting approaches. Specifically, we propose a novel hybrid model that uses a combined structure of Autoencoder, pre-trained transformer, and gradient boosting-based model Catboost. The data used in this study is derived from genuine data provided by a European fashion company. The model demonstrating the highest performance with purely textual information is selected as our baseline model. Our proposed model utilizes the concept of product similarity to conduct image-based instance selection. The model significantly improves prediction accuracy in certain fashion categories and improves the overall prediction accuracy by more than 8.5% in terms of both root-mean-square error (RMSE) and median absolute percentage error (MdAPE). The robustness test on another open fashion dataset, VISUELLE, demonstrates that our proposed models consistently improve demand forecasting accuracy in similar contexts.

Host-generated property images as a visual channel reveal substantial information about properties. Selecting proper images to display can lead to higher demand and increased rental revenue. In this paper, we define, estimate, and optimize the impacts of Airbnb photos on customers’ renting decisions. We apply ResNet-50, a convolutional neural network model, to build two separate, supervised learning models to evaluate the image quality and room types posted by Airbnb hosts. Then, we characterize the overall impacts of photo layout by the room type featured in the photo, photo quality, and order of display on the listings’ web pages. To address two estimation challenges in the Airbnb setting, namely, censored demand and changing consideration sets, we propose a novel pairwise comparison model that utilizes customers’ booking sequence data to consistently estimate the impact of photo layout on customers’ renting decisions. Our estimation results suggest that the cover image has a significantly larger impact than noncover photos and a high-quality bedroom cover image leads to the largest increase in demand. Furthermore, we build a nonlinear integer programming optimization problem and develop an algorithm to determine the optimal photo layout. Our counterfactual analysis suggests that a listing’s unilateral adoption of optimal photo layout leads to 11.0% more bookings on average. Moreover, depending on the neighborhood and market size, when listings simultaneously switch to the optimal photo layout, they get booked for two to five additional days in a year on average, which boosts revenue by $500 to $1,100. This paper was accepted by Swaminathan, Jayashankar, operations management. Funding: This research was partially sponsored by the MIT Data Science Lab and also benefited from generous support provided by Zalando. Supplemental Material: The online companion and data are available at https://doi.org/10.1287/mnsc.2022.4616 .

Download This Paper Open PDF in Browser Add Paper to My Library Share: Permalink Using these links will ensure access to this page indefinitely Copy URL Copy DOI

We present our work with a global online fashion retailer, Zalando, as an example of how a global retailer can utilize massive amount of data to optimize price discount decisions over a large number of products in multiple countries on a weekly basis. Given demand forecasts under a collection of discrete prices, Zalando’s objective is to set discount levels to maximize total profit over the entire selling horizon while taking into account both local and global business constraints. Local constraints refer to single product level requirements, where Zalando needs to balance sales across different countries and over different weeks while adhering to a first-come-first serve policy. That is, as long as product inventory exists, a customer is served independent of the customer’s origin country or time of arrival. Global constraints refer to specific targets set by management for different product categories and each country. We address these challenges by applying a three-step process. In the first step, we cluster products into groups that behave similarly and solve the aggregated problem in a way that allows us to decouple the problem into a problem for each product category. Each product category includes thousands of individual products (SKUs) and the various markets where products are sold, each of which with its own target sales and margins. In the second step, we decompose this problem using Lagrangian relaxation into a problem for each product (SKU) and provide an efficient way to identify the Lagrange multipliers. Finally, in the last step, we optimize decisions for individual products and also address local business constraints. For this new approach, which was implemented as part of Zalando's price discount decision process, we provide results from offline tests and field experiments to demonstrate its benefit.

For the target price policy of agricultural products, it is not only the guarantee of the free operation of the market price of agricultural products, but also the key for the government to play a leading role in it, and then actively arouse the enthusiasm of farmers, and promote the market risk to be continuously reduced. Farmers as the main body of the target price policy, in the view of the farmers, the policy will directly affect the enthusiasm of farmers to grow. Taking soybean crops as an example, the relevant departments found through the investigation of farmers. Because of the lack of organization, farmers’ ability to regulate the market is severely restricted. Therefore, in order to ensure the future of soybean and other farmers in China, it has not formed a profound understanding of the price policy. In the following articles, the effective reform of crop price policy will start with the operation and mechanism of the target price policy. The problems and reasons in the implementation of the target price policy and the policy suggestions for the implementation of the target price policy are analyzed.