Just visual context or part of the gesture? The role of arm orientation in bent pointing interpretation

Pointing gestures can take on different shapes. For example, people often point with a bent wrist at a referent that is occluded by another object. We hypothesized that while the extrapolation of the index finger is the most important visual cue in such bent pointing gestures, arm orientation is affecting interpretations as well. We tested two competing hypotheses. First, the arm could be processed as a less reliable but additional direction cue also indicating the referent. Consequently, the index finger extrapolation would be biased towards the arm direction (assimilation effect). Second, the arm could be perceived as visual context of the index finger, leading to an interpretation that is repulsed from the arm direction (contrast effect). To differentiate between both, we conducted two experiments in which arm and finger orientation of a virtual pointer were independently manipulated. Participants were asked to determine the pointed-at location. As expected, participants based their interpretations on the extrapolation of the index finger. In line with the second hypothesis, the more the arm was oriented upwards, the lower the point was interpreted and vice versa. Thus, interpretation pattern indicated a contrast effect. Unexpectedly, gestures with aligned arm and index finger deviated from the general contrast effect and were interpreted linearly compared to bent gestures. In sum, the experiments show that interpretations of bent pointing gestures are not only based on the direction of the index finger but also depend on the arm orientation and its relationship to the index finger orientation.