Facial expressions of pitch structure

William F. Thompson, Frank A. Russo

Research output: Chapter in Book/Report/Conference proceedingConference proceeding contributionResearchpeer-review

Abstract

In two experiments, we examined whether the facial expressions of singers affect judgements of musical structure. In Experiment 1, a performer was recorded singing each of three intervals. Participants were shown the visual recording (no sound) and judged the size of the interval the performer was (imagined to be) singing. Judgments were made under five conditions of occlusion: no occlusion, occlusion of the mouth; occlusion of the eyes and eyebrows; occlusion of the mouth, eyes, and eye-brows; and occlusion of the entire face (only head movements visible). The results indicated that participants could decode pitch relations from facial expressions alone. Examination of the occlusion conditions indicated that participants could differentiate intervals based on eye-brow movements alone, and even based on head movements. In Experiment 2, we recorded a musician singing thirteen versions of the last phrase of “Silent Night.” Versions differed in the pitch of the final tone, which was either the (expected) tonic of the song, or one of the other tones of the chromatic scale, including the tonic one octave above the expected note of the song. Participants were shown the visual recordings of the performances (no sound) and judged the “goodness of fit” of final note, as conveyed in the facial expressions of the singer. Mean ratings closely corresponded to the standard major tonal hierarchy, suggesting that the singer successfully communicated tonal structure through the use of facial expressions.
LanguageEnglish
Title of host publicationProceedings of the 9th International Conference on Music Perception and Cognition (ICMPC9)
EditorsM. Baroni, A. R. Addessi, R. Caterina, M. Costa
PublisherSociety for Music Perception & Cognition
Pages1141-1143
Number of pages3
ISBN (Print)8873951554
Publication statusPublished - 2006
Externally publishedYes
EventInternational Conference on Music Perception and Cognition (9th : 2006) - Bologna, Italy
Duration: 22 Aug 200626 Aug 2006

Conference

ConferenceInternational Conference on Music Perception and Cognition (9th : 2006)
CityBologna, Italy
Period22/08/0626/08/06

Fingerprint

Facial Expression
Experiment
Singers
Song
Sound
Head Movement
Tonal
Performer
Chromatic Scale
Eye Movements
Musical Structure
Rating
Tonic
Musicians
Night
Octave
The Tonic
Goodness
Visible

Keywords

  • music and media
  • visual cues
  • facial expression
  • pitch structure
  • tonality

Cite this

Thompson, W. F., & Russo, F. A. (2006). Facial expressions of pitch structure. In M. Baroni, A. R. Addessi, R. Caterina, & M. Costa (Eds.), Proceedings of the 9th International Conference on Music Perception and Cognition (ICMPC9) (pp. 1141-1143). Society for Music Perception & Cognition.
Thompson, William F. ; Russo, Frank A. / Facial expressions of pitch structure. Proceedings of the 9th International Conference on Music Perception and Cognition (ICMPC9). editor / M. Baroni ; A. R. Addessi ; R. Caterina ; M. Costa. Society for Music Perception & Cognition, 2006. pp. 1141-1143
@inproceedings{ef4171fd67134273a398d5efcb2470c0,
title = "Facial expressions of pitch structure",
abstract = "In two experiments, we examined whether the facial expressions of singers affect judgements of musical structure. In Experiment 1, a performer was recorded singing each of three intervals. Participants were shown the visual recording (no sound) and judged the size of the interval the performer was (imagined to be) singing. Judgments were made under five conditions of occlusion: no occlusion, occlusion of the mouth; occlusion of the eyes and eyebrows; occlusion of the mouth, eyes, and eye-brows; and occlusion of the entire face (only head movements visible). The results indicated that participants could decode pitch relations from facial expressions alone. Examination of the occlusion conditions indicated that participants could differentiate intervals based on eye-brow movements alone, and even based on head movements. In Experiment 2, we recorded a musician singing thirteen versions of the last phrase of “Silent Night.” Versions differed in the pitch of the final tone, which was either the (expected) tonic of the song, or one of the other tones of the chromatic scale, including the tonic one octave above the expected note of the song. Participants were shown the visual recordings of the performances (no sound) and judged the “goodness of fit” of final note, as conveyed in the facial expressions of the singer. Mean ratings closely corresponded to the standard major tonal hierarchy, suggesting that the singer successfully communicated tonal structure through the use of facial expressions.",
keywords = "music and media, visual cues, facial expression, pitch structure, tonality",
author = "Thompson, {William F.} and Russo, {Frank A.}",
year = "2006",
language = "English",
isbn = "8873951554",
pages = "1141--1143",
editor = "M. Baroni and Addessi, {A. R.} and R. Caterina and M. Costa",
booktitle = "Proceedings of the 9th International Conference on Music Perception and Cognition (ICMPC9)",
publisher = "Society for Music Perception & Cognition",

}

Thompson, WF & Russo, FA 2006, Facial expressions of pitch structure. in M Baroni, AR Addessi, R Caterina & M Costa (eds), Proceedings of the 9th International Conference on Music Perception and Cognition (ICMPC9). Society for Music Perception & Cognition, pp. 1141-1143, International Conference on Music Perception and Cognition (9th : 2006), Bologna, Italy, 22/08/06.

Facial expressions of pitch structure. / Thompson, William F.; Russo, Frank A.

Proceedings of the 9th International Conference on Music Perception and Cognition (ICMPC9). ed. / M. Baroni; A. R. Addessi; R. Caterina; M. Costa. Society for Music Perception & Cognition, 2006. p. 1141-1143.

Research output: Chapter in Book/Report/Conference proceedingConference proceeding contributionResearchpeer-review

TY - GEN

T1 - Facial expressions of pitch structure

AU - Thompson, William F.

AU - Russo, Frank A.

PY - 2006

Y1 - 2006

N2 - In two experiments, we examined whether the facial expressions of singers affect judgements of musical structure. In Experiment 1, a performer was recorded singing each of three intervals. Participants were shown the visual recording (no sound) and judged the size of the interval the performer was (imagined to be) singing. Judgments were made under five conditions of occlusion: no occlusion, occlusion of the mouth; occlusion of the eyes and eyebrows; occlusion of the mouth, eyes, and eye-brows; and occlusion of the entire face (only head movements visible). The results indicated that participants could decode pitch relations from facial expressions alone. Examination of the occlusion conditions indicated that participants could differentiate intervals based on eye-brow movements alone, and even based on head movements. In Experiment 2, we recorded a musician singing thirteen versions of the last phrase of “Silent Night.” Versions differed in the pitch of the final tone, which was either the (expected) tonic of the song, or one of the other tones of the chromatic scale, including the tonic one octave above the expected note of the song. Participants were shown the visual recordings of the performances (no sound) and judged the “goodness of fit” of final note, as conveyed in the facial expressions of the singer. Mean ratings closely corresponded to the standard major tonal hierarchy, suggesting that the singer successfully communicated tonal structure through the use of facial expressions.

AB - In two experiments, we examined whether the facial expressions of singers affect judgements of musical structure. In Experiment 1, a performer was recorded singing each of three intervals. Participants were shown the visual recording (no sound) and judged the size of the interval the performer was (imagined to be) singing. Judgments were made under five conditions of occlusion: no occlusion, occlusion of the mouth; occlusion of the eyes and eyebrows; occlusion of the mouth, eyes, and eye-brows; and occlusion of the entire face (only head movements visible). The results indicated that participants could decode pitch relations from facial expressions alone. Examination of the occlusion conditions indicated that participants could differentiate intervals based on eye-brow movements alone, and even based on head movements. In Experiment 2, we recorded a musician singing thirteen versions of the last phrase of “Silent Night.” Versions differed in the pitch of the final tone, which was either the (expected) tonic of the song, or one of the other tones of the chromatic scale, including the tonic one octave above the expected note of the song. Participants were shown the visual recordings of the performances (no sound) and judged the “goodness of fit” of final note, as conveyed in the facial expressions of the singer. Mean ratings closely corresponded to the standard major tonal hierarchy, suggesting that the singer successfully communicated tonal structure through the use of facial expressions.

KW - music and media

KW - visual cues

KW - facial expression

KW - pitch structure

KW - tonality

M3 - Conference proceeding contribution

SN - 8873951554

SP - 1141

EP - 1143

BT - Proceedings of the 9th International Conference on Music Perception and Cognition (ICMPC9)

A2 - Baroni, M.

A2 - Addessi, A. R.

A2 - Caterina, R.

A2 - Costa, M.

PB - Society for Music Perception & Cognition

ER -

Thompson WF, Russo FA. Facial expressions of pitch structure. In Baroni M, Addessi AR, Caterina R, Costa M, editors, Proceedings of the 9th International Conference on Music Perception and Cognition (ICMPC9). Society for Music Perception & Cognition. 2006. p. 1141-1143