Open music foundation models for full-song generation

(map-yue.github.io)

59 points | by selvan 4 days ago ago

20 comments

  • ssalka 3 hours ago ago

    Something interesting... the first 10 seconds or so of the "Death Growl" example[1] is basically copied verbatim from "Ov Fire And The Void" by Behemoth.

    More specifically, I think the part that seems copied is at 2:13 of the original[2], as it leads into a solo-ish bit which in the AI version sounds similar still, but goes on to do its own thing:

    [1] https://map-yue.github.io/music/moon.death_metal.mp3

    [2] https://youtu.be/vAmnsKKrt9w?t=133

    • someothherguyy 3 hours ago ago

      > Additionally, our memorization-effect experiments in Section 11 demonstrate that our design maintains creativity without plagiarizing, even under strong training set conditioning.

      https://arxiv.org/html/2503.08638v1#S11

    • vorgol 2 hours ago ago

      The youtube link is suddenly not available any more (at least in the UK)

    • amelius 2 hours ago ago

      Does Shazam think it is the same?

  • lotyrin 4 hours ago ago

    Very nice. Anyone know of projects that aren't tackling the full-song problem but rather instrument parts/loops/stems/acapellas? I'd like something that's more like "infinite AI Loopcloud/Splice" most of these full-song models don't do well to be asked for individual parts in my experience (though I will have to try it with this one).

    • vunderba 2 hours ago ago

      This gets discussed a lot but unfortunately there's just not much out there around this.

      The closest thing I've seen is virtual drummers in Logic X which will follow along with the structure of your song and generate a percussive accompaniment. It's no substitute for a real drummer but it's serviceable.

    • platers 3 hours ago ago

      https://suno.com/studio-waitlist Just a waitlist so far, but looks like this is the direction suno is going

      • lotyrin 3 hours ago ago

        Yeah... I hope this is what their plan is with that, but I'm not entirely certain.

    • HxokcPwi 2 hours ago ago
    • rwmj 3 hours ago ago

      Also live AI dueting would be interesting, like having a virtual guitarist you could jam/duet with.

  • bangaladore 2 hours ago ago

    What is the use case for music generation models? I see usecases for alot of the other foundation models like text, image, tts, sst, but why do I want AI generated music?

    • FridgeSeal 2 hours ago ago

      Now you don’t need to know how to make music! You’re finally free of all those pesky, elitist musicians gate-keeping music!!!!1!

    • frank_nitti 2 hours ago ago

      I’ve mostly used them for laughs with my friends. Sometimes generating “custom” songs with funny lyrics, but most fun so far is editing lyrics of existing songs to say ridiculous things for fun.

      No real clue how someone would use them for a more serious endeavor, only thing I could imagine would be to quickly iterate/prototype with song structures on a fixed seed to generate ideas for a real composition. Consider the case of an indie game developer or film maker getting some placeholder music to test the experience during early throwaway iterations.

    • bongodongobob 15 minutes ago ago

      An actual serious answer is to help musicians brainstorm while writing. It's so good at helping me come up with ideas, or converting an idea to another genre.

    • libraryatnight an hour ago ago

      Generating crappy background music for reality TV?

      • bangaladore 40 minutes ago ago

        Yeah, this seems the most likely. Just loads of royalty free background music for industries that want that.

  • scarecrowbob an hour ago ago

    yeah, but have yall made any progress in a model that can have sex with my partner for me?