Functional Sound Synthesis in Computer Animations


Timbre Tree
Until recently, sound has been given little attention in computer graphics and related domains of computer animation and virtual environments, although sounds which are properly synchronized to motion provide a great deal of information about events in the environment. Sounds are often not properly synchronized because the sounds and the phenomena that caused the sounds are not considered in an integrated way.

In this work, we present an integrated approach to motion and sound as it applies to computer animation and virtual environments. The key to this approach is synchronization by mapping the motion parameters to sound parameters so that the sound changes as a result of changes in the motion. This is done by representing sounds using a technique for functional composition analogous to the "shade trees" which we call timbre trees. These timbre trees are used as a part of a sound description language that is analogous to scene description languages like RenderMan. Using this methodology, we have produced convincing sound effects for a wide variety of animated scenes.

Collaborators
  • James HahnThe George Washington University
  • Tapio TakalaHelsinki University of Technology
  • Hesham Fouad, vrsonic
  • J. Won Lee, KOG Studios
  • Larry Gritz, Sony Imageworks
  • Publications
  • J. K. Hahn, J. Geigel, J. W. Lee, L. Gritz, T. Takala, and S. Mishra. An integrated approach to motion and sound. The Journal of Visualization and Computer Animation, 6(2):109–123, 1995.

  • J. K. Hahn, L. Gritz, R. P. Darken, J. Geigel, and J. W. Lee. An integrated virtual environment system. Presence, pages 353–360, 1993.

  • T. Takala, J. Hahn, L. Gritz, J. Geigel, and J. W. Lee. Using physically-based models and genetic algorithms for functional composition of sound signals, synchronized to animated motion. In International Computer Music Conference (ICMC), pages 180–183, 1993..
  • Animations
  • J. Hahn, L. Gritz, J. Geigel, J. Lee, and T.Takala. Timbre trees (animation). In SIGGRAPH Video Review – Issue 93. ACM SIGGRAPH, 1993.