Audiovisual granular synthesis: creating synergistic relationships between sound and image

Batty, J 2014, Audiovisual granular synthesis: creating synergistic relationships between sound and image, Doctor of Philosophy (PhD), Media and Communication, RMIT University.

Document type: Thesis
Collection: Theses

Attached Files
Name Description MIMEType Size Software Click to show the corresponding preview/stream application/zip 6.03MB
Batty.pdf Thesis application/pdf 57.11MB
Title Audiovisual granular synthesis: creating synergistic relationships between sound and image
Author(s) Batty, J
Year 2014
Abstract The aims of this research were to investigate how an audio processing technique known as granular synthesis can be translated to a visual processing equivalent, and to develop software that fuses audiovisual relationships for the creation of real-time audiovisual art. In order to carry out this project, two main research questions were posed. The first question was: how can audio processing techniques such as granular synthesis be adapted and applied to influence new visual performance techniques, and the second question was: how can computer software synergistically integrate audio and visuals to enable the real-time creation and performance of audiovisual art.

The project at the centre of my research was the creation of a real-time audiovisual granular synthesis instrument named Kortex. The research project involved a practice-based methodology and used an iterative performance cycle to evaluate and develop the Kortex prototype. These included performing iterations of the Kortex prototype at a number of local, interstate and international events.

Kortex facilitates the identification of shared characteristics found between sound and image at the micro and macro level. The micro level addresses individual audiovisual segments, or grains, while the macro level addresses post-processing effects applied to the stream of audiovisual grains. Audiovisual characteristics are paired together by the user at each level, enabling composition with both media simultaneously. This provides the audiovisual artist with a dynamic approach for the creation of new works.

Creating relationships between image and sound is highly subjective, yet an artist may use a mathematical, metaphorical/intuitive or intrinsic approach to create a convincing correlation between the two media. The mathematical approach expresses the relationship between sound and image as an equation. Metaphorical/intuitive relationships are formed when the two media share similar emotional or perceptual characteristics, while intrinsic relationships occur when audio and visual media are synthesised from the same source.

Performers need powerful control strategies to manipulate large collections of variables in real-time. I found that pattern-generating modulation sources created overlapping phrases that evolved the behaviour of audiovisual relationships. Furthermore, saving interesting aesthetics that emerged into banks of presets, along with the ability to slide from one to the next, facilitated powerful transformations during a performance.

The project has contributed to the field of audiovisual art, specifically to the performance work of DJs and VJs. Kortex provides a single audiovisual composition and performance environment that can be used by DJs and VJs for creative collaboration. Kortex has enormous potential for adoption by the DJ/VJ community to assist in the production of tightly synchronised real-time audiovisual performances.
Degree Doctor of Philosophy (PhD)
Institution RMIT University
School, Department or Centre Media and Communication
Keyword(s) Audiovisual
Granular Synthesis
Realtime Software
Digital Signal Processing
Creative Coding
Version Filter Type
Access Statistics: 295 Abstract Views, 819 File Downloads  -  Detailed Statistics
Created: Thu, 21 May 2015, 12:27:08 EST by Denise Paciocco
© 2014 RMIT Research Repository • Powered by Fez SoftwareContact us