The McGurk effect.

So that interview I posted last week about synesthesia reminded me of a cool perceptual illusion I learned about in college called the McGurk effect. It turns out that we naturally integrate information from multiple sensory modalities without even realizing we're doing it, and that when those multiple sources of information conflict, crazy shennannigans ensue.

In the classic experiment, McGurk spliced video of a person producing one sound ("ga") with audio of them producing another ("ba"). The crazy thing that they found was that instead of perceiving just one sound or the other, the perception would often be an average of the two inputs, and actually come out closer to a "da" than anything else.

This makes a lot of sense phonetically, because the major difference between a "ba" and a "ga" is where you momentarily stop the outflow of air before letting it out in a little puff. With a "ba", you stop the outflow of air with your lips in the front of your mouth, while with "ga", you stop the air by pushing the body of your tongue back up against the roof of your mouth (right in front of your uvula). On the other hand, "da" is made by pushing the tip of your tongue up against the roof of your mouth towards the front. If you think about it, that spot is smackdab in the middle between the back of your mouth and your lips. Yay phonetics!

Here's a youtube video of the McGurk effect in action. I watched it a bunch of times, and for me, the "da" only pops out if I really stare at the lips. Otherwise it's pretty muddled or sounds more like a "ba" to me. Try closing your eyes too, or watching it with the sound off:

http://www.youtube.com/watch?v=aFPtc8BVdJk

I think it's pretty cool how automatic the integration of our senses is. If it wasn't so automatic, ventriloquists would have to get real jobs and they wouldn't be able to dub over swear words realistically in network tv versions of movies...

also, the very fabric of our shared reality would be ripped asunder, or something equally dramatic.

Free tip of the day: next time you're having a hard time hearing someone at a party, try looking really closely at their lips instead of cocking your head. It helps a bunch.