Intro
This article about Spleeter is the first in a series here at PVC that’s going to focus on Artificial Intelligence / Machine Learning tools. There won’t be that high-level discussion of how AI works (or doesn’t). No discussion of corpus training. No mentions of Tensorflow or ML on a chip.
Instead, I’m going to focus on tools that are available to use – and look at what’s working now in a functional way for people in production/post-production.
Unbake a cake: Breaking apart music into stemlike elements:
Like you, I have my favorite music libraries.
I spend (too much time) finding music tracks. Often they have something great…if we only had the stems.
The rule has always been: you can’t unbake a cake back to it’s original ingredients”
Enter AI/Spleeter.
Deezer is a streaming music company. They developed a technology for research that they chose to open source. It’s called Spleeter.
It can take finished music and separate it back out into as many as five stems
* Vocals
* Piano
* Bass
* Drum
* Other
Suddenly, your music library can do more.
No, it’s not true stems (I guess you can’t truly unbake the cake) but, it means I get much more from my music library.
The AI tool has three models, 2 stems (vocals + everything else), 4 stems (vocals/drum/bass/other), and 5 stems as above.
How good is it? There’s a yearly AI contest and Spleeter is the standard that new tools have to beat.
Basics of how Spleeter works
There’s not much to this. Seriously.
* Give it a Music track
* Press button
* Two to Five stems.
Mix/mute/remix as you like in your weapon(s) of choice.
This is a command-line tool, but there are some intrepid coders who have wrapped it into a GUI
Already in some tools
This AI tech is already recognized as very useful – mostly by audio companies. Just to mention a few groups that use some/part of this ability include iZotope (Music Rebalance3) and SpectraLayers (Unmix) among others.
Command-line
For most of you, skip ahead to the Windows or Mac versions
Some of us are command-line people. Some of us aren’t. I don’t judge.
The original library for all of this is here
As far as CLI tools go? This is 100% a tool I’d feel comfortable as it’s simplicity is the key. Now, the install isn’t as easy as I’d like, but the command is.
The command looks like this:
spleeter separate -o audio_output -p spleeter:5stems YOURMUSICFILE.wav
Literally, you just copy this part:
spleeter separate -o audio_output -p spleeter:5stems
and put your music file name after it and press enter.
Deeper install here and Usage detailed here
Win version
For something a little more user friendly, this is the GUI that I’d strive to have. It’s called SpleeterGUI
It has a full set of bells and whistles available. It permits choosing how many tracks you want – along with the higher quality type of separation (up to 16Khz)
Mac
I found only one Mac tool
It’s super simple – only the five track version.
I’ll warn you that it’s all sorts of broken right now for the M1 chips – although there seems to be some ways to get the command line version working.
Actual usage
I’ve been using this for about the last two months. The actual splitting (splitting?)? Super easy.
My satisfaction is medium – 6.5/10 with this technology. It’s crazy useful to break apart music like this. I’m enjoying focusing on bass and drum hits that I can intensify on particular cuts — beyond just the beat.
Where it’s falling down for me? These aren’t really stems. About 40% of the music I’m trying it with, I end up with empty tracks – especially vocals. So little of the music I deal with has a vocal track at all.
I end up with nearly empty and empty piano. Bass and drums are generally well separated and nearly everything else in other.
But bass and drums are crazy useful.
My wish of course, is that it were to have true stems. I recently found a work I loved – but the sitar/guitar sound was just grating and I’d like to have just been able to remove it.
Other thoughts
I’m been watching a number of web tools open up into this space. Many of them are just putting a licensing system up in front of the CLI.
In other words, I think many of the sites out there are just selling you the same free tools, with a web interface.
If you find one that you think has something special – maybe toss it into the local version and see if their platform has much difference.
The key takeaway – the TL;DR
We might be able to unbake the cake enough that it gives enough flexibility to tweak our existing music. And in this case, it’s easy AND FREE.