ToolRerank: Adaptive and Hierarchy-Aware Reranking for Tool Retrieval

Tool learning aims to extend the capabilities of large language models (LLMs)with external tools. A major challenge in tool learning is how to support alarge number of tools, including unseen tools. To address this challenge,previous studies have proposed retrieving suitable tools for the LLM based onthe user query. However, previously proposed methods do not consider thedifferences between seen and unseen tools, nor do they take the hierarchy ofthe tool library into account, which may lead to suboptimal performance fortool retrieval. Therefore, to address the aforementioned issues, we proposeToolRerank, an adaptive and hierarchy-aware reranking method for tool retrievalto further refine the retrieval results. Specifically, our proposed ToolRerankincludes Adaptive Truncation, which truncates the retrieval results related toseen and unseen tools at different positions, and Hierarchy-Aware Reranking,which makes retrieval results more concentrated for single-tool queries andmore diverse for multi-tool queries. Experimental results show that ToolRerankcan improve the quality of the retrieval results, leading to better executionresults generated by the LLM.