Reward Finetuning for Faster and More Accurate Unsupervised Object Discovery
NeurIPS(2023)
摘要
Recent advances in machine learning have shown that Reinforcement Learning
from Human Feedback (RLHF) can improve machine learning models and align them
with human preferences. Although very successful for Large Language Models
(LLMs), these advancements have not had a comparable impact in research for
autonomous vehicles -- where alignment with human expectations can be
imperative. In this paper, we propose to adapt similar RL-based methods to
unsupervised object discovery, i.e. learning to detect objects from LiDAR
points without any training labels. Instead of labels, we use simple heuristics
to mimic human feedback. More explicitly, we combine multiple heuristics into a
simple reward function that positively correlates its score with bounding box
accuracy, \ie, boxes containing objects are scored higher than those without.
We start from the detector's own predictions to explore the space and reinforce
boxes with high rewards through gradient updates. Empirically, we demonstrate
that our approach is not only more accurate, but also orders of magnitudes
faster to train compared to prior works on object discovery.
更多查看译文
关键词
more accurate unsupervised object
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要