The KANDY Benchmark: Incremental Neuro-Symbolic Learning and Reasoning with Kandinsky Patterns
CoRR(2024)
摘要
Artificial intelligence is continuously seeking novel challenges and
benchmarks to effectively measure performance and to advance the
state-of-the-art. In this paper we introduce KANDY, a benchmarking framework
that can be used to generate a variety of learning and reasoning tasks inspired
by Kandinsky patterns. By creating curricula of binary classification tasks
with increasing complexity and with sparse supervisions, KANDY can be used to
implement benchmarks for continual and semi-supervised learning, with a
specific focus on symbol compositionality. Classification rules are also
provided in the ground truth to enable analysis of interpretable solutions.
Together with the benchmark generation pipeline, we release two curricula, an
easier and a harder one, that we propose as new challenges for the research
community. With a thorough experimental evaluation, we show how both
state-of-the-art neural models and purely symbolic approaches struggle with
solving most of the tasks, thus calling for the application of advanced
neuro-symbolic methods trained over time.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要