Robotic arm generative painting through real-time analysis of music performance

Richard Savery; Anna Savery; Justin Baird

doi:10.1145/3527188.3563913

Robotic arm generative painting through real-time analysis of music performance

Richard Savery, Anna Savery, Justin Baird

Research output: Chapter in Book/Report/Conference proceeding › Conference proceeding contribution › peer-review

4 Citations (Scopus)

Abstract

This paper describes a prototype audio-visual performance of a Ufactory Uarm Swift and a live musician. In this setting, the robotic arm was used as an AI agent to create a visual representation of a musical work in real-time. An A4 white canvas was gradually filled with a mixture of black, blue, red and yellow paints across the span of approximately eight minutes. The musician, performing on an acoustic violin, fitted with a custom built audio interface, performed multiple versions of an improvisatory work developed specifically for the prototype performance. The following sections discuss our technical approach to programming and implementing the Ufactory Uarm Swift as a painting arm, reflections of the musical process and propose future directions for this project.

Original language	English
Title of host publication	HAI '22
Subtitle of host publication	Proceedings of the 10th Conference on Human-Agent Interaction
Place of Publication	New York
Publisher	Association for Computing Machinery (ACM)
Pages	253-255
Number of pages	3
ISBN (Electronic)	9781450393232
DOIs	https://doi.org/10.1145/3527188.3563913
Publication status	Published - 5 Dec 2022
Event	International Conference on Human-Agent Interaction (10th : 2022) - Christchurch, New Zealand Duration: 5 Dec 2022 → 8 Dec 2022

Conference

Conference	International Conference on Human-Agent Interaction (10th : 2022)
Abbreviated title	HAI ’22
Country/Territory	New Zealand
City	Christchurch
Period	5/12/22 → 8/12/22

Keywords

artwork
audio
human-robot interaction
music
painting
sound

Access to Document

10.1145/3527188.3563913

Cite this

@inproceedings{c44b61b53304463983265cd01227bf72,

title = "Robotic arm generative painting through real-time analysis of music performance",

abstract = "This paper describes a prototype audio-visual performance of a Ufactory Uarm Swift and a live musician. In this setting, the robotic arm was used as an AI agent to create a visual representation of a musical work in real-time. An A4 white canvas was gradually filled with a mixture of black, blue, red and yellow paints across the span of approximately eight minutes. The musician, performing on an acoustic violin, fitted with a custom built audio interface, performed multiple versions of an improvisatory work developed specifically for the prototype performance. The following sections discuss our technical approach to programming and implementing the Ufactory Uarm Swift as a painting arm, reflections of the musical process and propose future directions for this project.",

keywords = "artwork, audio, human-robot interaction, music, painting, sound",

author = "Richard Savery and Anna Savery and Justin Baird",

year = "2022",

month = dec,

day = "5",

doi = "10.1145/3527188.3563913",

language = "English",

pages = "253--255",

booktitle = "HAI '22",

publisher = "Association for Computing Machinery (ACM)",

address = "United States",

note = "International Conference on Human-Agent Interaction (10th : 2022), HAI {\textquoteright}22 ; Conference date: 05-12-2022 Through 08-12-2022",

}

Savery, R, Savery, A & Baird, J 2022, Robotic arm generative painting through real-time analysis of music performance. in HAI '22: Proceedings of the 10th Conference on Human-Agent Interaction. Association for Computing Machinery (ACM), New York, pp. 253-255, International Conference on Human-Agent Interaction (10th : 2022), Christchurch, New Zealand, 5/12/22. https://doi.org/10.1145/3527188.3563913

Robotic arm generative painting through real-time analysis of music performance. / Savery, Richard; Savery, Anna; Baird, Justin.
HAI '22: Proceedings of the 10th Conference on Human-Agent Interaction. New York: Association for Computing Machinery (ACM), 2022. p. 253-255.

Research output: Chapter in Book/Report/Conference proceeding › Conference proceeding contribution › peer-review

TY - GEN

T1 - Robotic arm generative painting through real-time analysis of music performance

AU - Savery, Richard

AU - Savery, Anna

AU - Baird, Justin

PY - 2022/12/5

Y1 - 2022/12/5

N2 - This paper describes a prototype audio-visual performance of a Ufactory Uarm Swift and a live musician. In this setting, the robotic arm was used as an AI agent to create a visual representation of a musical work in real-time. An A4 white canvas was gradually filled with a mixture of black, blue, red and yellow paints across the span of approximately eight minutes. The musician, performing on an acoustic violin, fitted with a custom built audio interface, performed multiple versions of an improvisatory work developed specifically for the prototype performance. The following sections discuss our technical approach to programming and implementing the Ufactory Uarm Swift as a painting arm, reflections of the musical process and propose future directions for this project.

AB - This paper describes a prototype audio-visual performance of a Ufactory Uarm Swift and a live musician. In this setting, the robotic arm was used as an AI agent to create a visual representation of a musical work in real-time. An A4 white canvas was gradually filled with a mixture of black, blue, red and yellow paints across the span of approximately eight minutes. The musician, performing on an acoustic violin, fitted with a custom built audio interface, performed multiple versions of an improvisatory work developed specifically for the prototype performance. The following sections discuss our technical approach to programming and implementing the Ufactory Uarm Swift as a painting arm, reflections of the musical process and propose future directions for this project.

KW - artwork

KW - audio

KW - human-robot interaction

KW - music

KW - painting

KW - sound

UR - https://www.scopus.com/pages/publications/85144616040

U2 - 10.1145/3527188.3563913

DO - 10.1145/3527188.3563913

M3 - Conference proceeding contribution

SP - 253

EP - 255

BT - HAI '22

PB - Association for Computing Machinery (ACM)

CY - New York

T2 - International Conference on Human-Agent Interaction (10th : 2022)

Y2 - 5 December 2022 through 8 December 2022

ER -

Robotic arm generative painting through real-time analysis of music performance

Abstract

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this