Curiosity towards exploring new objects in one’s environment is a key driver of intelligent agents. We explore the problem of mapping in environments which are non-stationary, and where areas may exhibit different change patterns. This is an important challenge for potential “domestic” robots, which would have to perform tasks in houses. We propose a computational approach to building a curious agent which constructs a map of its environment and estimates how often a specific location in this map is changing. Using these estimates, the agent will actively try to reach rapidly changing areas more often than static areas. As a result, the behavior becomes more focused towards learning about significant changes in the environment. We present experiments in a robotic navigation framework implemented using ROS. The results show the utility of this framework for map acquisition and maintenance.