Empathy is a complex psychological concept consisting of affective and cognitive aspects. It plays a crucial role in human communication. Dialogue systems are designed to interact with people in natural language. Integrating empathy into dialogue systems is valuable to enhance user experience, although it is a challenging task. Existing studies in Empathetic Conversational Response Generation (ECRG) have made notable advancements. However, their focus has mainly been on specific facets of empathy, such as mimicking the user’s emotions and integrating commonsense knowledge to enhance cognitive empathy. This leaves opportunities for further exploration in areas such as enhancing self-other awareness. This paper will conduct a comprehensive review of empathy theories to bridge this gap, offering a theoretical understanding of empathy in various dimensions. Additionally, it critically reviews existing studies on ECRG, including algorithms, methods to address affective empathy and cognitive empathy, datasets, and evaluation methods. By highlighting the technical challenges and opportunities, this review provides guidance for researchers on AI research for modeling empathy in dialogue systems by investigating empathetic response generation for conversational AI modeling.