Learning multimodal entity representations and their ensembles, with applications in a data-driven advisory framework for video game players

We investigate the impact of combining multiple representations derived from heterogeneous data sources on the performance of machine learning (ML) models. In particular, we experimentally compare the approach in which independent models are trained on data representations from different sources with the one in which a single model is trained on joined data representations. As a case study, we discuss various entity representation learning methods and their applications in our data-driven advisory framework for video game players, called SENSEI. We show how to use the discussed methods to learn representations of cards and decks for two popular collectible card games (CCGs), namely Clash Royale (CR) and Hearthstone: Heroes of Warcraft (HS). Then, we follow our approach to create ML models which constitute the back-end for several out of SENSEI’s end-user functionalities. When learning representations, we consider techniques inspired by the NLP domain, as they allow us to create embeddings which capture various aspects of similarity between entities. We put them together with representations composed of manually engineered features and standard bags-of-cards. On top of that, we propose a new end2end deep learning architecture with an attention mechanism aimed at reflecting meaningful inter-entity interactions.