Cross-Modal Spatial Remapping

Definition

Cross-modal spatial remapping is the brain’s ongoing conversion and reconciliation of visual, auditory, eye-position, and contextual signals so events can be assigned to stable locations and likely shared sources.

Current Synthesis

Jennifer Groh presents spatial perception as a coordinate problem rather than a direct copy of the world. Sound direction is inferred from tiny timing differences between the ears, level differences, and direction-specific filtering by the pinnae. Those head-centered cues must then be reconciled with where the eyes are pointing and with visual evidence about what is producing the event.

The result is useful but editable. A synchronized puppet mouth, movie screen, or phone image can pull a voice toward the visible source even when the loudspeaker is elsewhere. Groh’s laboratory findings add a possible low-level mechanism: eye position affects auditory processing across the pathway, and each saccade is accompanied by small, direction-sensitive eardrum movements. The measured coupling is stronger evidence than the proposed function; whether the ear signal directly improves localization remains open.

Key Claims

• Horizontal sound location uses interaural timing and level differences, while outer-ear filtering supplies additional direction-sensitive frequency cues.
• Eye position changes how auditory spatial information is represented, so sound maps are dynamically remapped rather than fixed.
• Temporal and contextual agreement can make the brain assign physically separated sights and sounds to one source.
• Ventriloquism and screen sound show that plausible visual evidence can override ear-based location estimates.
• Saccades are accompanied by precisely timed, direction-sensitive eardrum signals that may contribute to early audiovisual alignment.
• Development and learning matter because head growth and changing sensory geometry require localization cues to be recalibrated.

Evidence

• Localization cues: How Your Thoughts Are Built & How You Can Shape Them | Dr. Jennifer Groh describes interaural delays of up to roughly half a millisecond, acoustic shadowing, and pinna filtering.
• Visual source assignment: How Your Thoughts Are Built & How You Can Shape Them | Dr. Jennifer Groh uses ventriloquism, lip synchronization, theaters, and phones to show cross-modal binding.
• Eye-position coding: How Your Thoughts Are Built & How You Can Shape Them | Dr. Jennifer Groh reports auditory responses that shift with gaze and eye-position effects across auditory pathway areas.
• Peripheral coupling: How Your Thoughts Are Built & How You Can Shape Them | Dr. Jennifer Groh reports tiny eardrum movements time-locked to saccades and differing by movement direction and extent.

Counterevidence & Qualifications

The source is a structured interview summary, not the primary experimental literature. It provides no sample size, effect magnitude, anatomical-resolution limit, or competing functional account for the eardrum signal. Visual capture of sound is probabilistic rather than automatic and depends on timing, plausibility, context, and learning. The source’s predicted adaptation after outer-ear damage is explicitly a first-principles inference, not evidence reported from a study.

What Changed

• Created a unified page for gaze-dependent auditory maps, audiovisual source assignment, and saccade-linked ear signals.

Related Concepts

• Selective Auditory Attention - selects a target stream after localization and source inference make streams distinguishable.
• Acoustic Environment Perception - describes how reflections and room geometry change the spatial evidence being remapped.
• Visual-Vestibular Stabilization - neighboring visual-body coordinate problem used to stabilize gaze during movement.
• Perception as Biological Inference - broader account of perception as active interpretation under uncertainty.
• Reflex-Deliberation Control Hierarchy - places superior-colliculus multisensory selection within layered action control.