The rapid adoption of generative artificial intelligence (GenAI) in higher education environments is transforming how learners access guidance and feedback. Yet, students' AI self-efficacy (AISE) in using AI tools may critically influence how they engage with these technologies. This study developed a GenAI agent integrated with a structured knowledge graph (KG) to provide personalized, real-time feedback in an online Preschool Health and Hygiene course. Ninety-eight university students were grouped by AISE levels (high, medium, low) to explore differences in AI feedback adoption, critical thinking, and cognitive load during two learning tasks. The results indicated that high-AISE learners leveraged knowledge graph-integrated generative AI feedback more strategically, showing greater self-monitoring skills, improved self-evaluations, and lower cognitive load on complex tasks. It was also found that the high-AISE group showed higher self-evaluation ratings than the low-AISE group only on their second revised tasks, while no significant difference was found between the two groups in the first initial and revised tasks, or in the second initial task. Qualitative findings suggest evolving learner-agent relationships, shifting from reliance on AI authority toward reflective collaboration. In particular, the high-AISE group was more inclined to perceive GenAI as a collaborative learning partner, demonstrating higher-order abilities in information evaluation and integration. These insights highlight the need to scaffold AI literacy and tailor GenAI-supported learning environments to students' AISE levels, informing future designs of adaptive higher education systems. The findings also imply the importance of conducting further research to support students in improving their AI self-efficacy in the future.