The chorus from Radiohead's 'Sulk' as viewed in Infinity.
Infinity is no ordinary audio editor, but one that allows you to manipulate the individual instruments within a mixed audio file in endless ways.
It's a remarkable thing to see a mixed piece of music pulled apart and displayed as individually editable instruments. Through the use of its TrueSource source-separation technology, Infinity from Hit'n'Mix can identify the tonal qualities of instruments and split music into fully adjustable notes and percussion. It has the look and feel of Melodyne in some respects, but presents multiple instruments in a single view and allows their individual identities to be blurred, cloned, blended and transplanted. It's not an easy program to explain and it feels like it's yet to fully nail down its own identity, but there's some very clever stuff going on in here.
Hit'n'Mix Infinity does its magic through sinusoidal spectral analysis and resynthesis. What you see on screen are not audio tracks: they are resynthesized representations of musical data created with component sine waves, referred to by Hit'n'Mix as Rips. The process of ripping is not, as I had initially assumed, one of generating multiple individual audio tracks from mixed audio; rather, it is one of gathering and analysing audio data and identifying musical notes, timbres and tonal qualities. So, the first thing you need to understand is that you are not dealing with audio files but their component ingredients; and until you change something, these recombine perfectly to make an exact copy of the original audio file.
To Rip a file or files, you simply drag and drop them into the Infinity window. Unless you have a fast computer, the process can take a long time, but fortunately the ripper can do its work in the background.
Building Blocks
Developer Martin Dawe declines to explain what's going on behind the scenes; my closest point of reference is the recent electric piano releases from virtual instrument developers Sampleson. Instead of modelling circuitry or sampling the sound of, say, a Fender Rhodes, they have done deep-level spectral analysis, down to the frequency and amplitude of individual sine waves. The full range of sounds can be recreated by combining the right sine waves in the right way, resulting in a very authentic-sounding Rhodes from only a few megabytes of sine waves.
Martin says that his inspiration came from a David Attenborough piece about the Australian lyrebird, which mimics the sounds of other birds as part of its mating call. The remarkable thing is that the lyrebird will mimic anything it hears — not just birdsong, but all sorts of environmental sounds. The documentary shows it doing strangely convincing imitations of camera shutters, car alarms and chainsaws cutting down trees. Martin was inspired to figure out how to break down any sound to something simple and manipulable like sine waves, creating an algorithmic model based upon the way people perceive notes. He's been rolling this around for about 17 years and Infinity sees these ideas made concrete in a piece of software that's incredibly detailed, versatile and unlike anything else out there.
Infinity's ability to pull out individual instruments within mixed music is extraordinary.
So, what can you do with it? That is the question. In essence, Infinity unlocks the DNA of audio and gives you the flexibility to manipulate it at a molecular level. Sound is not being emulated and audio is not being streamed or processed: you are applying algorithms to the component elements of sound. These algorithms, or RipScripts as they are called, are written in the Python programming language, and you can get down and dirty with the code directly within Infinity. The RipScripts define the notes and describe the musical elements, dictating how the elements are put back together and what happens when you process them. This could be as simple as adding a vibrato effect to a vocal or as mind-bending as applying the tonal qualities and harmonic content of a clarinet to the sound of a guitar.
Multitrack Remixing
Infinity's ability to pull out individual instruments within mixed music is extraordinary. There's a lot of fun to be had with it, and there's much potential for fixing and tweaking things. It could be useful with live recordings where you want to fix a bum note or maybe remove noise and other artifacts. You can do interesting things like isolating and copying a vocal line from one song and placing it in another. You can then clone the pitch of the existing vocal and apply it to the transplanted one. As a way of isolating previously unobtainable samples it certainly has potential.
Most of the obvious uses are found in mucking about with mash-ups and remixes, but these have limited real-world applications in these copyright-twitchy days. Perhaps a more valid application would be to take a recording of you playing live and reharmonise your vocals. But as you use Infinity, a theme emerges: what it does is unbelievably clever, but it's not always clear what you should do with it. Having said that, fiddling around inside mixed music is only one approach to Infinity, and there are a lot more roads to travel down.
Not only can you isolate individual voices and instruments within mixed audio, but you can clone them and add harmony parts, as shown here.
Track Editing
The Infinity workspace displays pitch on the vertical axis, in the shape of a piano keyboard, and time on the horizontal axis. It detects tempo and displays bars for you (which can be adjusted) and places a measure-based grid onto the music. Notes are displayed in a colour corresponding to the detected instrument type, with brightness indicating amplitude. Infinity auto-detects from a list of real-life instruments such as violin, piano, bass, trumpet and so on, depending on the harmonic content it identifies within the sound. I found this worked to a degree, but whenever I dragged in a guitar track it always seemed to decide that it was a piano. It's not desperately important to get this right; what matters is more that Infinity correctly separates notes and assigns them consistently to the instruments.
The notes are not always in one piece; Infinity may detect a single note as multiple notes because of the nature of the playing, in which case you'll end up with a number of note fragments. You can stitch these back together manually so that when you are selecting them and moving them about they stay together. Around the notes like ghostly echoes across the whole frequency range is a cloud of grey and white material. These are the more percussive frequencies that have been detected and are not seen as part of the note. High-frequency energy within this category is labelled as Percussion whereas the low and cross-spectrum frequencies are reported as Drums. These get their own tracks on the instrument list and so you can turn them off to leave you with a pure instrument sound, though that doesn't sound entirely right; most real-world instruments contain some unpitched material, and killing those elements completely can make it sound a little strange. But it depends on the source material, what you're trying to achieve and the amount of work you want to put in. As an example, Infinity had no problem ripping out all four notes of an electric guitar playing some jazzy chords, and this worked quite well when I muted the Drums and Percussion tracks; in fact, the rattling and the hiss on the track were removed very neatly. If I approached things from the other way around and muted the guitar, I was left with quite an unusual percussive guitar sound.
The instrument list gives you a volume control for each instrument, so rather than removing elements entirely, you can just dial them down a bit. You can also delete elements directly from the high- and low-frequency bands as you can with any note. Infinity splits the Percussion up into eight bands of high frequencies and the Drums into two bands of low frequencies, which you can select, audition and delete independently. Clicking any note or percussion element will audition that part in isolation, whereas a double-click will play it back in the context of everything else.
Once you've ripped a track, you can move individual notes in both pitch and time in whatever way you fancy. You can change the chords, change the modes, transpose, fix timing issues, copy/paste, shorten and lengthen the notes... and we haven't opened a single menu yet! There's so much going on within Infinity that it's difficult to explain it all and not get distracted by one element when there are so many other things to talk about.
Watch our video from AES New York 2019.