SongCreator AI
So essentially,
SongCreator makes high-quality songs with harmonized vocals and accompaniment from lyrics
Paper: SongCreator: Lyrics-based Universal Song Generation (23 Pages)
Github: https://songcreator.github.io/
Researchers from Shenzhen companies introduce AI in the field of song generation using AI has seen significant progress, but challenges remain, particularly in creating high-quality songs with both vocals and accompaniment from lyrics.
Hmm..What’s the background?
Previous research primarily focused on individual aspects of song generation. For example, Singing Voice Synthesis (SVS) focused on generating vocals from musical scores but couldn't create complete songs with accompaniment. Similarly, models like SongComposer could generate vocals from lyrics but lacked accompaniment. While Jukebox attempted lyrics-to-song generation, it produced unnatural vocals and lacked musicality.
Ok, So what is proposed in the research paper?
The main proposal in this research paper is SongCreator, a novel AI system designed to generate high-quality songs with harmonized vocals and accompaniment from lyrics.
Here's a breakdown of SongCreator's key feature
Dual-sequence language model (DSLM): Unlike previous models that treated vocals and accompaniment as a single entity, SongCreator uses a DSLM with separate decoders to model the information for vocals and accompaniment independently
Attention mask strategy: SongCreator employs an attention mask strategy for its DSLM, enabling it to perform a wider range of song generation tasks beyond lyrics-to-song generation
Through these innovations, SongCreator aims to be a versatile and powerful tool for song generation, pushing the boundaries of AI-generated music.
What’s next?
The researchers propose exploring the following for future work
Genre and Style Control: Currently, SongCreator cannot control the genre or style of the generated songs based on textual descriptions
Improved Vocal Clarity: The sources note that background accompaniment can interfere with the encoding of vocal information by BEST-RQ, potentially impacting the clarity of synthesized vocals
Ethical Considerations: The sources acknowledge the potential for misuse of AI-generated music, such as creating deepfakes or spreading misinformation
So essentially,
SongCreator makes high-quality songs with harmonized vocals and accompaniment from lyrics
Learned something new? Consider sharing with your friends!