Action-based multisensory integration in striking events

Many theories of multisensory integration focus on action-independent perception. This approach has limited ecological validity, because under everyday conditions we frequently act upon multisensory events through motor behavior. We present a novel methodology for the study of multisensory integration in action-directed perception, focusing on striking events. Observers repeatedly hit a virtual object with a target striking velocity, and are presented with various types of sensory information about the striking event: auditory and/or haptic and/or visual. For each of the experimental trials, they initially receive feedback on the tracking of the target striking velocity. In a second phase, feedback is eliminated. In a third phase, the sensory properties of the striking event are changed. We quantify the extent to which a variation in each of the sensory modalities disrupts performance in the tracking of the target striking velocity. Multisensory dominance hierarchies are inferred from the comparison of the patterns of performance disruption in unimodal and multisensory contexts.