Soft-Rank Diversity: Diversity Regularisation for Multi-Hop Knowledge Graph Reasoning in Conversational Recommendation

Dialogue-based Conversational Recommender Systems (CRS) are trained to reproduce item sequences chosen by human annotators, who disproportionately mention popular titles. The model learns to mimic this bias, leaving most of the catalog unrecommended, a problem known as low catalog coverage. We propose Soft-Rank Diversity (SRD), a fully differentiable training loss that penalises concentration of probability mass on clusters of similar items, encouraging the model to spread recommendations across the catalog. We target knowledge-graph-based CRS because knowledge graph (KG) embeddings place items that share real-world attributes close together, so SRD can detect and penalise real redundancy among candidates. SRD attaches to the item-scoring step of multi-hop KG reasoning, where it operates on a focused subset of related candidates, and admits an efficient O(MdB) factorisation, where M is the number of candidate items, d is the embedding dimension, and B is the batch size. The closest alternative training loss, the Determinantal Point Process (DPP) likelihood, incurs cubic cost and, as we show, degrades both accuracy and coverage. On CR-Walker with the ReDial and TG-ReDial benchmarks, SRD improves catalog coverage with competitive recall on ReDial and an accepted accuracy-coverage trade-off on TG-ReDial, while requiring nearly half of training time compared to DPP.