Artificial intelligence is becoming an increasingly influential presence in the world of music technology. Recent years have seen AI integrated into a variety of plugins useful for everything from mixing (opens in new tab) to drum synthesis, while tools like Google’s MusicLM and Riffusion are making huge advances in their ability to generate sounds (and entire tracks, almost) at the click of a button.
In yet another quantum leap forward for AI-powered music-making, producers are using AI to transform their vocals into the sound of another artist’s voice. In a video posted to Twitter by entrepreneur and tech influencer Roberto Nickson, Nickson finds a Kanye-style beat on YouTube, writes eight bars and records them over the beat, before replacing his own voice with an AI-generated version of Kanye West’s.
The results are impressively realistic. The majority of the verse sounds absolutely spot-on and could easily convince the average listener of its authenticity, and there are one only or two words that sound a little off, early on in the recording. It’s worth noting, though, that the quality of the lyrics and delivery are inferior to Kanye’s – two things AI can’t successfully imitate, just yet.
Nickson also used the technology to create alternate versions of popular songs, producing versions of Justin Bieber’s Love Yourself, Frank Ocean’s Nights and Dr. Dre’s Still D.R.E. that feature AI Kanye (or Kany-AI, as we’ve taken to calling him) on the vocals.
To transfer the timbre of Kanye’s vocals to his own, Nickson followed a YouTube tutorial that demonstrates how to use Google Colab to access an existing AI model that’s been trained on Kanye’s voice. It’s not a complex process, but a little inconvenient: when this type of technology becomes streamlined and integrated into the DAW, it’s likely to have profound implications for the music industry.
“All you have to do is record reference vocals and replace it with a trained model of any musician you like,” Nickson says, considering where this type of AI could lead us in the future. “Keep in mind, this is the worst that AI will ever be. In just a few years, every popular musician will have multiple trained models of them.”
Though fascinating from a technical standpoint, the legal status of this kind of style transfer is unclear. The right of publicity, defined as “the rights for an individual to control the commercial use of their identity”, is protected in a number of countries and would likely prohibit artists from using AI versions of another artist’s vocals in commercia
