How Virtual Instruments Work

We cut through the jargon to explain how software instruments achieve their incredible realism.

Reading some recent SOS reviews of flagship orchestral sample libraries from the likes of Spitfire, EastWest and VSL, I was struck by just how much jargon is involved in describing their features. Anyone new to the idea of creating a realistic orchestral composition in their computer, or whose most recent encounter with ‘fake’ musical instruments involved comparatively simple hardware ROMplers and workstation keyboards, would struggle to understand what they’d be getting from a piece of software that could cost hundreds of pounds/dollars and consume an entire SSD drive on its own. So, in this article, I’ll try and explain how these things work, so that the next time you see a review of Gigantic Trombones Vol 9, Enchanted Violas or Sensual Hollywood Banjos, you’ll know what the heck we’re talking about.

The Basics

Most virtual instruments are built using samples — that is, audio recordings of the instrument you’re ‘virtualising’. (There are some exceptions, most notably those that use physical modelling; for more info see the ‘Getting Physical’ box later on). So let’s start with a simple hypothetical virtual instrument: a solo violin. Because we want this to be the best virtual violin it can possibly be, we’ll hire a world‑class session player to bring his Stradivarius into our studio, and play the clearest, crispest, most mellifluous Middle C known to humanity for us to record. Now, we take that sample and load it into a software sampler like Native Instruments’ Kontakt. Hit Middle C on your keyboard, and you’ll hear that lovely note again. Congratulations on your first virtual instrument!

You might eventually consider playing notes other than Middle C, but don’t worry: all software samplers have the ability to pitch‑shift audio, so that our violin sample can be played back at any other pitch. This works well enough for notes a few semitones either side of the recorded note, but stray too high or too low and it’ll start to sound quite unnatural. So let’s call in our session violinist again, and ask him to play another C, this time an octave higher. If we map the first C to Middle C on our keyboard, plus all the notes half an octave either side, and then do the same an octave up for the higher C, we’ll have two octaves of pretty convincing violin to play with.

Twenty years or so ago, when hard‑disk space was expensive and limited, that’s exactly how it was done. But with storage space so cheap and fast nowadays, we can afford to be uncompromising, so why not go the whole hog and ask our session player to play every single note in the violin’s range, for maximum realism?

Dynamics

This is exactly what most modern sample libraries do, and it allows us to play some relatively convincing solo violin lines from our MIDI keyboard, across the entire instrument’s range. But what if you want some notes to be played quietly and others loudly? You have two options. The first is to apply your MIDI keyboard’s velocity data to the synth‑style controls in your software sampler, to modulate the volume at which the samples play back. You can even add a low‑pass filter too, setting it so that lower‑velocity notes sound duller than higher‑velocity ones, mimicking the softer timbre you get when playing a real violin more quietly.

Again, this is very much how dynamics were handled many years ago, and it can work well enough, but since we’ve all got multi‑terabyte drives and our sample library is still only a few megabytes, why not get our session player in again and ask him to play every single note once again, but at three different volumes? Then we’ll tell our sampler to play the quiet version of each note for MIDI velocities below, say, 50; the medium‑volume samples from 50 up to 100; and then the loud samples for anything above that.

These are what’s known as velocity layers and, again, most modern sample libraries use this technique to enhance realism. If we were selling our virtual violin, we might boast about it having ‘three velocity layers’, but you can use more — indeed, virtual drum instruments in particular tend to use a great deal more (BFD Drums, a division of InMusic, use up to 80 velocity layers per drum kit element). You can start to see why some of them ship on their own hard drives!

Virtual drum instrument BFD3 features up to 80 velocity layers per kit piece.

To get the most from our humble three velocity layers, though, we can even combine them with the MIDI‑controlled filtering scheme I mentioned earlier. And, to avoid an obvious ‘switch’ in samples between notes played at a velocity of 49 and those played at 50, we can crossfade between them, for a smoother transition.

Round Robin

Our virtual instrument is now quite usable, but try playing repeated notes at the same velocity and your ears will tell you instantly that you’re just hearing the same sample over and over again, rather than a real violin. If this were a virtual drum instrument we’d complain about the ‘machine gun’ effect (it’s particularly audible on drum hits, but our virtual violin isn’t immune to the problem).

One way around this would be to put our programming hat on again and tell the sampler to apply a tiny amount of random pitch‑shift to successive samples, to mimic the imperfections that our human violinist might exhibit if we told him to play E7 eight times in a row. Better still, though, would be to drag our patient session violinist back into the studio, and get him to play every sample — each note on the violin, at each of our three velocity layers — all over again. Then we tell our sampler to alternate between samples from the first session and the new ones we’ve just recorded, and now the machine‑gun effect is eliminated.

This is known in the trade as a round robin. Our virtual violin has one round robin per note, because our session player got bored and went home before we could ask him to record more, but a fully-featured orchestral instrument could easily have half a dozen round robins — and some of EastWest’s virtual string instruments have as many as 16.

Now our virtual violin — which started with just a single sample — already has close to 300 (six distinct samples for each of the 50‑odd semitones in its range).

One potential, albeit usually pretty minor, issue with round robins is that when you hit Play at the start of your song, you won’t know for sure which of the round robin samples will play back. In practice this is rarely an issue (because sample library makers are pretty meticulous types and will have been consistent in recording one mezzopiano Middle C after another), but if you simply must hear exactly the same thing every time you play your composition, some orchestral instruments feature a ‘round robin reset’ button, to allow you to do just that.

EastWest’s Hollywood Strings 2. On the right you can see faders for the different microphone options, and on the left is the ‘round robin reset’ button, which ensures consistent playback.

Take...