McGurk Effect – What You See is What You Hear

By Anupum Pant

Do Not Cheat

1. Close your eyes: Before you watch this video, you should know that, when you watch it for the first time, you have to watch it with closed eyes. Well, you can’t ‘watch’ with closed eyes. It simply means, you have to just hear the sound track first. I’m sure you can do it because closing eyes for 5 seconds is not asking for much. I don’t have any veiled interests here. It is for you. You won’t appreciate the effect if you keep your eyes open during the first go.
2. Open your eyes: Watch it again with your eyes open. Be calm. It happens to everyone.

So that was the McGurk effect. It is a perfect example to show that accurate perception of reality may involve more than one sense. This is called “Multimodal perception”. In simple words, our senses do not learn from the surroundings independent from each other, they work together and learn together to help us perceive information.

The video

The video shows a man moving his lips as if saying “Ga” or “Da”, although it is just a visual of him saying that, let us call this the visual for the first sound (“Da” sound). The second sound is the “Ba” sound that is actually playing – this is the second sound.

When your eyes are closed, you hear the correct sound, the second sound – the “Ba” sound.
When you watch it again with your eyes open, you hear the “Da” sound. The brain combines the visual and audio signals to make you believe that the sound you are hearing is actually “Da”. Even if you are aware of this illusion, your brain doesn’t correct it for you, no matter how many times you repeat this.

Some interesting things about McGurk effect

• You can focus anywhere on the face (not just the mouth) to perceive the same effect.
• Women show a stronger McGurk effect than men.
• Normally, people with mental disorders do not hear much difference in the sound with eyes open or closed.
• By the age of 4 months, infants are able to identify facial movements and relate them to sounds. So, even infants show this effect to a certain extent.
• Information rich perceptions are easier to remember. So, the next time you are trying to learn something, involve more senses. Probably involving a particular smell that goes with a piece of information will help you remember things well. Or a sound, maybe.

Note: Since my posts were getting longer everyday, I’ve tried to keep this one short. I don’t want the posts to be overwhelming for people who are not used to long reading. This blog is meant to make science sound interesting to everybody, not to scare them away from it. I’ll be waiting to receive your feedback on this. You can get in touch through twitter (@indigoanalysis)