Nvidia Unveils Fugatto, a 'Swiss Army Knife for Sound'

Nvidia introduced Fugatto, a generative AI model for audio that accepts text and audio inputs to create or modify any combination of music, voices, and sounds. The company describes it as a 'Swiss Army knife for sound.' It can generate music from text prompts, add or remove instruments in existing tracks, alter accents or emotions in speech, and produce sounds users have never heard before. Fugatto builds on Nvidia's earlier work in speech modeling, audio vocoding, and audio understanding. The model was trained exclusively on open-source licensed material using Nvidia's own chips over more than a year. Nvidia says Fugatto exhibits emergent capabilities and can handle tasks it was not explicitly trained for, such as generating high-quality singing voices from text, using a technique called ComposableART. Potential use cases include music production, adapting advertising campaigns with regional accents and emotions, and personalized language learning. Nvidia has not yet announced when Fugatto will be available to users.