Online shoppers face many difficulties finding the right garment size that fits them. There has been a large body of work that try to tackle this problem from a customer perspective. Most often the approaches rely solely on purchases and returns to offer a personalized size-recommendation. In this work we investigate the potential effect of additional fashion characteristics such as shape and fit attributes of an article to better predict a potential size issue even before a purchase has been made. We frame the problem as a Multi-Task-Learning (MTL) problem and extend prior work to predict size issues from garments visual information. In the experiments, we explore different MTL architectures and a variety of datasets to further harness the power of these models. To the best of our knowledge, the proposed approach is the first to combine size, fit and shape attributes of fashion articles to address the challenge of size issue prediction in online shopping. Our findings indicate that there is a positive effect of additional attributes and that we can improve the size-issue prediction in comparison to the state-of-the-art models.

Size recommendation is a task of predicting the best-fitting fashion article size for a customer, based on their purchase history and other signals. Size recommendations assume that for every article there is a size that fits the customer’s body. However, fit recommendations require more fine-grained information to be able to predict the fitting experience and to steer customers to appropriate articles. This requires an understanding of the shape and expected fit of an article, which are not always readily available. In this study, we show how customer reviews can be leveraged to identify topics corresponding to customer concerns and to common issues of the item purchase and post-purchase experience. We empirically evaluate three typologically different topic extraction methods (Latent Dirichlet Allocation, text embedding clustering and zero-shot text classification) in the highly complex domain of real-world fashion reviews and analyse their pros and cons. We demonstrate that coarse-grained topic labels correspond to item’s fit issues, such as “too big” and “too small” and show how fine-grained topics signal other specific fit issues of an article (e.g. “short leg length” for jeans or “loose sleeve area” for upper garments).

In recent years of online fashion, the availability of large-scale datasets has fueled the success of data-driven algorithmic products for supporting customers in their journey on fashion e-commerce platforms. Very often, these datasets are collected in an implicit manner, are subjective, and do not have expert annotated labels. The use of inconsistent and noisy data to train machine learning models could potentially harm their performance and generalization capabilities. In this paper, we explore uncertainty quantification metrics within the context of online size and fit recommender systems and show how they could be used to deal with noisy instances and subjective labels. We further propose an uncertainty-aware loss function based on Monte-Carlo dropout uncertainty estimation technique. Through experiments on real data at scale within the challenging domain of size and fit recommendation, we benchmark multiple uncertainty metrics and demonstrate the effectiveness of the proposed approach for training in the presence of noise.

One of the major challenges facing e-commerce fashion platform is that of recommending to customers the right size and fit for fashion apparel. In this work we study this topic in depth and demonstrate its various complexities focusing in particular on the challenging cold-start problem that arises when no order history is available for a specific customer. We demonstrate the multifaceted value of data obtained by involving the customer in the loop and show how it allows for an effective cold-start recommender system. We highlight our findings via detailed experiments performed on hundreds of thousands of customers and items in real world e-commerce scenarios. In addition, results and discussions are provided investigating the trade-off between the recommender’s effectiveness and the customer’s experience with the goal of introducing accurate solutions with low user cognitive load.

One of the major challenges facing e-commerce fashion platform is that of recommending to customers the right size and fit for fashion apparel. In this work we study this topic in depth and demonstrate its various complexities focusing in particular on the challenging cold-start problem that arises when no order history is available for a specific customer. We demonstrate the multifaceted value of data obtained by involving the customer in the loop and show how it allows for an effective cold-start recommender system. We highlight our findings via detailed experiments performed on hundreds of thousands of customers and items in real world e-commerce scenarios. In addition, results and discussions are provided investigating the trade-off between the recommender’s efficiency and the customer’s experience with the goal of introducing accurate solutions with low user cognitive load.

We present a detailed analysis of the class of regression decision tree algorithms which employ a regulized piecewise-linear node-splitting criterion and have regularized linear models at the leaves. From a theoretic standpoint, based on Rademacher complexity framework, we present new high-probability upper bounds for the generalization error for the proposed classes of regularized regression decision tree algorithms, including LASSO-type, and $\ell_{2}$ regularization for linear models at the leaves. Theoretical result are further extended by considering a general type of variable selection procedure. Furthermore, in our work we demonstrate that the class of piecewise-linear regression trees is not only numerically stable but can be made tractable via an algorithmic implementation, presented herein, as well as with the help of modern GPU technology. Empirically, we present results on multiple datasets which highlight the strengths and potential pitfalls, of the proposed tree algorithms compared to baselines which grow trees based on piecewise constant models.

The availability of multi-modal datasets that pair images and textual descriptions of their content has been a crucial driver in progress of various text-image tasks such as automatic captioning and text-to-image retrieval. In this paper, we present FEIDEGGER, a new multi-modal corpus that focuses specifically on the domain of fashion items and their visual descriptions in German. We argue that such narrow-domain multi-modality presents a unique set of challenges such as fine-grained image distinctions and domain-specific language, and release this dataset to the research community to enable study of these challenges. This paper illustrates our crowdsourcing strategy to acquire the textual descriptions, gives an overview over the FEIDEGGER dataset, and discusses possible use cases.

We address the problem of selecting groups of jointly informative, continuous, features in the context of classification and propose several novel criteria for performing this selection. The proposed class of methods is based on combining a Gaussian modeling of the feature responses with derived bounds on and approximations to their mutual information with the class label. Furthermore, specific algorithmic implementations of these criteria are presented which reduce the computational complexity of the proposed feature selection algorithms by up to two-orders of magnitude. Consequently we show that feature selection based on the joint mutual information of features and class label is in fact tractable; this runs contrary to prior works that largely depend on marginal quantities. An empirical evaluation using several types of classifiers on multiple data sets show that this class of methods outperforms state-of-the-art baselines, both in terms of speed and classification accuracy.

We propose several novel criteria for the selection of groups of jointly informative continuous features in the context of classication. Our approach is based on combining a Gaussian modeling of the feature responses, with derived upper bounds on their mutual information with the class label and their joint entropy. We further propose specic algorithmic implementations of these criteria which reduce the computational complexity of the algorithms by up to two-orders of magnitude, making these strategies tractable in practice. Experiments on multiple computer-vision data-bases, and using several types of classiers, show that this class of methods outperforms state-of-the-art baselines, both in terms of speed and classication accuracy.

We propose a novel approach to eciently select informative samples for large-scale learning. Instead of directly feeding a learning algorithm with a very large amount of samples, as it is usually done to reach state-of-the-art performance, we have developed a \distillation procedure to recursively reduce the size of an initial training set using a criterion that ensures the maximization of the information content of the selected sub-set. We demonstrate the performance of this procedure for two dierent computer vision problems. First, we show that distillation can be used to improve the traditional bootstrapping approach to object detection. Second, we apply distillation to a classication problem with articial distortions. We show that in both cases, using the result of a distillation process instead of a random sub-set taken uniformly in the original sample set improves performance signicantly.

We consider the problem of automatic macroaction discovery in imitation learning, which we cast as one of change-point detection. Unlike prior work in change-point detection, the present work leverages discriminative learning algorithms. Our main contribution is a novel supervised learning algorithm which extends the classical Boosting framework by combining it with dynamic programming. The resulting process alternatively improves the performance of individual strong predictors and the estimated changepoints in the training sequence. Empirical evaluation is presented for the proposed method on tasks where change-points arise naturally as part of a classification problem. Finally we show the applicability of the algorithm to macro-action discovery in imitation learning and demonstrate it allows us to solve complex image-based goal-planning problems with thousands of features.

We propose to train an ensemble with the help of a reservoir in which the learning algorithm can store a limited number of samples. This novel approach lies in the area between offline and online ensemble approaches and can be seen either as a restriction of the former or an enhancement of the latter. We identify some basic strategies that can be used to populate this reservoir and present our main contribution, dubbed Greedy Edge Expectation Maximization (GEEM), that maintains the reservoir content in the case of Boosting by viewing the samples through their projections into the weak classifier response space. We propose an efficient algorithmic implementation which makes it tractable in practice, and demonstrate its efficiency experimentally on several compute-vision data-sets, on which it outperforms both online and offline methods in a memory constrained setting.

We present a novel approach to automatic macroaction discovery and its application to a complex goal-planning task. The problem of macro-action discovery is framed as one of multiple change point detection and is addressed with the help of the Dynamic Programming Boosting algorithm. The procedure is then employed to solve a complex goal-planning problem which entails an avatar navigating a 3D environment. By using DPBoost to decompose the problem into a number of simpler ones, we are able to successfully address both the complexity and partial observability of the environment.

It has been demonstrated repeatedly that combiningmultiple types of image features improves the performance of learning-based classification and regression. However, no tools exist to facilitate the creation of large pools of feature extractors by extended teams of contributors. The MASH project aims at creating such tools. It is organized around the development of a collaborative web platform where participants can contribute feature extractors, browse a repository of existing ones, run image classification and goal-planning experiments, and participate in public large-scale experiments and contests. The tools provided on the platform facilitate the analysis of experimental results. In particular, they rank the feature extractors according to their efficiency, and help to identify the failure mode of the prediction system.

The standard strategy for efficient object detection consists of building a cascade composed of several binary classifiers. The detection process takes the form of a lazy evaluation of the conjunction of the responses of these classifiers, and concentrates the computation on difficult parts of the image which cannot be trivially rejected. We introduce a novel algorithm to construct jointly the classifiers of such a cascade, which interprets the response of a classifier as the probability of a positive prediction, and the overall response of the cascade as the probability that all the predictions are positive. From this noisy-AND model, we derive a consistent loss and a Boosting procedure to optimize that global probability on the training set. Such a joint learning allows the individual predictors to focus on a more restricted modeling problem, and improves the performance compared to a standard cascade. We demonstrate the efficiency of this approach on face and pedestrian detection with standard data-sets and comparisons with reference baselines.

In this paper, we describe the components of a novel algorithm for the detection of grasping points from monocular images of previously unseen objects. A basic building block of our approach is the use of a newly devised descriptor, capable of representing grasping point shape and appearance by the use of histograms of oriented gradients in a semi-local manner. Combined with boosting our method learns discriminative grasp point models for new objects from a set of annotated real-world images. The method has been extensively evaluated on challenging images of real scenes, exhibiting largely varying characteristics concerning illumination conditions, scene complexity, and viewpoint. Our experiments show that the method, despite these variations, works in a stable manner and that its performance compares favorably to the state-of-the-art.

There are a number of supervised machine learning methods such as classiers pretrained using restricted Boltzmann machines and convolutional networks that work very well for handwritten character recogni- tion. However, they require a large amount of labeled training data to achieve good performance which unlike unlabeled data is often expensive to obtain. In this paper a number of novel semi-supervised learn- ing methods for handwritten character recognition are presented based on the previous algorithms. These methods are oriented towards learning from as little labeled data as possible and for this goal they use unlabeled data and active learning. The proposed techniques are of varying complexity and involve sim- ple K-means clustering, feature mapping with self organizing maps, dimensionality reduction with deep auto-encoders, and sub-sampling techniques. The presented algorithms outperform both a generic semi- supervised active learning algorithm and two well known supervised algorithms.

There are a number of supervised machine learning methods such as classiffers pretrained using restricted Boltzmann machines and convolutional networks that work very well for handwritten character recognition. However, they require a large amount of labeled training data to achieve good performance which unlike unlabeled data is often expensive to obtain. In this paper a number of novel semi-supervised learning methods for handwritten character recognition are presented based on the previous algorithms. These methods are oriented towards learning from as little labeled data as possible and for this goal they use unlabeled data and active learning. The proposed techniques are of varying complexity and involve simple K-means clustering, feature mapping with self organizing maps, dimensionality reduction with deep auto-encoders, and sub-sampling techniques. The presented algorithms outperform both a generic semisupervised active learning algorithm and two well known supervised algorithms.