Integrating Quantitative and Qualitative Reasoning for Value Alignment.

Agents that focus only on achieving their own goals may cause significant harm to society. As a result, when deciding which actions to perform, agents have to consider societal values and how their actions impact these values-the 'value alignment problem'. There is therefore a need to integrate quantitative machine reasoning with an ability to reason about qualitative human values. In this paper, we present a novel framework for value-based reasoning that aims to bridge the gap between these two modes of reasoning. In particular, our framework extends the theory of grading to model how societal values can trade off with each other or with the agent's goals. Furthermore, our framework introduces the use of hyper-real numbers to represent both quantitative and qualitative aspects of reasoning and help address the value alignment problem.