Physical Modelling Synthesis Explained

Modelling is the current Big Thing in digital synthesis, and it's being used to recreate the sounds not only of traditional acoustic instruments, but also the analogue synth timbres electronic musicians know and love. So just how are the manufacturers making numbers behave like nose‑flutes and maths sound like Moogs? Super modeller Martin Russ provides the beginners' guide.

We live in a world that is increasingly described by numbers. Most bartering of goods was long ago replaced by metal and paper tokens. Since the Ordnance Survey started mapping Great Britain in 1791, even the shape of the country itself has been represented by heights and positions on a grid. From the early 1980s onwards, audio has been held as numbers on digital Compact Discs. In the 1990s, the most important part of your passport is arguably the machine‑readable strip of numbers on the back page — which looks increasingly like an identity card (or will the ubiquitous credit card take over that role if it ever gains a photograph of the holder?).

To deal with all the numbers that are used in today's world, we are becoming dependent on computers. Digital processing of numbers calculates our wages; deducts our taxes; enables us to register a National Lottery number; controls the manufacturing of hi‑fi equipment; reduces the distortion in loudspeakers; produces music, and much more. With the right knowledge and programming, computers can be used to simulate how a nuclear reactor works, how the economy should be developing, and even how a musical instrument makes sounds. They do this by using a model of the item being simulated. The model is usually a series of mathematical rules which describe how the individual parts interact — so it's often called a 'mathematical model'.

Starting Simple

Let's take a very simple example — a model of a balloon. There are three basic categories of balloon: empty; blown‑up (full of air); and burst. So how about some rules for the behaviour of a balloon?

• You can turn an empty balloon into a blown‑up one by putting air inside.
• You can turn a blown‑up balloon into a burst one by pricking it with a pin or by trying to put too much air into it.
• You can't turn a burst balloon into either of the other categories — once burst, it's burst forever.

We also need to know something about the properties of the things we are using, so we need to know that the balloon has a limit to the amount of air that it can hold before it bursts. We might also need to know that if you add air to a balloon and then release it, all the air will rush out of the balloon and restore it to almost its original emptiness.

Believe it or not, this simple example illustrates many of the important things about mathematical models.

• Firstly, you need to be able to describe what is happening — in this case, whether the balloon is empty, blown‑up or burst.
• Secondly, you need to be able to describe what changes can be made. For example, adding air to the balloon changes it from empty to blown‑up.
• Thirdly, some of the changes can be irreversible — the pin can permanently burst the balloon.
• Fourthly, you need to know something about the properties of the things which are used in the model.
• Finally, some changes can be unexpected: for example, if you add too much air to the balloon, it can burst, because its capacity for holding air has been reached.

Once armed with a model, we can then use it to make predictions, and to answer questions. For example, what happens if we add water to an empty balloon? Does it become blown‑up? Will a pin still have the same disastrous effect? In order to know, we need to know something about the properties of water, and if we decide that it behaves like thick air, then we can make predictions based on this — if you add water, then the balloon will be blown‑up, and a pin will still burst it. But if you fill a balloon with sand, does a pin still burst it? The simple model we have for the balloon may not work in some situations, especially those outside the scope of our description of how a balloon works. The model might need additional information on the substance used to fill it, for example.

We live in a world that is increasingly described by numbers.

Models of musical instruments allow us to make sounds which have many of the characteristics of real instruments, but only when we understand enough about the instrument to be able to describe how it works in sufficient detail. Balloons are relatively easy to describe, while musical instruments are usually more complex. Of course, if we understand how something works, we can often extend, enhance or even just alter the way in which the parts interact, and this allows the creation of imaginary instruments: ones which might be impossible to actually make! These 'impossible' instruments are often rather like filling a balloon with sand, because they go beyond the normally expected boundaries of the model.

Putting It Together

A mathematical model is just a way of plugging together all of the things we've learned about how a musical instrument works. These are normally expressed as equations connecting all the individual parts. So an equation that describes a string driver needs to know things such as the tension in the string, how far it's pulled from its rest position, and whereabouts, along the length of the string, it's being pulled. A resonator equation is more concerned with the frequency response of the body or tube part of the instrument, where the resonances are. There may also be equations which describe how the driver and resonator are connected together, or how the energy decays away or is coupled to the air. The basic model runs something like this:

• Energy is added to the instrument.
• Something vibrates because of the extra energy.
• Something resonates with the vibration.
• The vibration is coupled to the air by some means.
• The energy is gradually transferred to the air.

That's about it for modelling. We now have a mathematical model of how a physical musical instrument works — and this is the basis for all of the modelling‑based synthesis techniques. The rest of this article examines the different types of models.

Pre‑Digital Modelling

At the risk of being sectioned for madness, I'll let you into a secret. Analogue synthesizers are nothing more than pre‑digital versions of the latest digital 'virtual' synthesizers — and they use modelling. You don't actually need to have a digital synthesizer for it to be called a 'modelling' instrument, although many manufacturers would have you believe that. In fact, the earliest analogue synthesizers were arguably things called analogue computers, which used voltages and currents to represent numbers, and processed them using circuits like amplifiers and filters.

What was once an obscure topic of interest to a few music/physics researchers is now a valuable commodity.

The important thing is that the way in which an analogue synthesizer is put together is nothing more than a convenient representation of how to make sounds. The Oscillator is the source of the raw vibration: the driver. The Filter is the resonator. The Envelope Generator and Amplifier effectively control damping and volume, and the loudspeaker converts the electrical signal into sound by vibrating the air. You may never have thought of an analogue synthesizer as a model of how sounds are produced (or as an analogue computer!), but that's all it is.

One difference between real‑world musical instruments and synthesizers is the way in which energy is added to the driver. Whereas a synthesizer's Oscillator produces sound all the time, a conventional musical instrument normally has energy added in bursts: plucks, hits, and blowing — with the exception of the digeridoo, of course. In an analogue synthesizer, the envelope generator and amplifier perform this function instead, and this is arguably one area where the modelling is rather unrealistic. In fact, the whole of an analogue synthesizer is an idealised, simplified view of how a musical instrument makes sounds.

Virtuality

Once you've worked out how to convert an understanding of how something works into a physical model, the same principles can be applied in many ways, and all you need is a suitable means of turning the model into reality. Analogue synthesizers do it using currents and voltages which represent the vibrations directly, but today's technology tends to replace most signals with digitised numerical versions instead. If you take an analogue synthesizer and convert it into a digital form, you have what can be called a 'Virtual' synthesizer.

The Clavia Nord Lead is one example of the technique of using digital technology to represent an analogue synthesizer, and manipulating numbers to make sounds. But FM synthesizers and more recent Sample & Synthesis (S&S) instruments, such as the Emu Morpheus, also use numbers and equations to represent how frequency modulation and complicated filters work. But the underlying model they use, of sound source, filter and envelope is a crude one, which is not very representative of the real workings of a musical instrument.

Digital technology, however, allows a model to be implemented with as much depth as is required. In the case of virtual analogue synthesizers, this includes detail such as the imperfections of oscillators, filters and amplifiers. Some possible areas where it is possible to describe (and thus model) this non‑ideal behaviour include:

• Oscillators which vary in pitch slightly to simulate the effects of power supply loading, temperature, humidity, and even just time.
• Oscillators whose modulation inputs are non‑linear, or non‑exponential.
• Waveforms whose shapes are merely rough approximations of the names used to describe them (Square, Triangle, Sawtooth, Sine, etc).
• Waveforms whose shape changes with frequency to simulate the effects of bandwidth‑limited/non‑linear/distorting amplifiers.
• Filters that distort audio signals.
• Filters whose characteristics change as they approach self‑oscillation.
• Filters that add noise to audio signals passing through them.
• Amplifiers that distort/compress/add noise to audio signals passing through them.
• Envelope Generators which have linear/exponential or other curves.

Just as it is often the detail of a real instrument that provides the uniqueness and interest in the timbre it produces, so these analogue 'imperfections' can be built into the model to provide the distinctive sound of individual 'retro' analogue instruments. If we understand exactly what gives analogue instruments their character, it's then possible to model them.

Interactions

One way of producing a better model of a real instrument is by trying to take account of the interactions between the sound source and the resonator. In an analogue or virtual synthesizer, the two are normally kept entirely separate, whereas in a real musical instrument, the driver and the resonator are part of the same instrument, and changing one affects the other. If you remove the tubing from a brass instrument and play just the mouthpiece, you realise just how important all that metal tubing is! Or imagine a guitar where the body and the strings are not connected. In fact, separating these two parts is so alien to how a real instrument works that it is often difficult to imagine a driver without the associated resonator, and the interaction between the two can be crucial in determining the sound of the instrument. Think about the difference between an acoustic guitar and an electric guitar when played without amplification — and then compare the sustain performance of the two...

One method for representing these interactions between the driver and resonator is to change the driver in some way. The S&S approach would be to provide different samples, although producing a smooth change between them would be difficult. Technics use a different method in their SX‑WSA1 Acoustic Modelling synthesizer: here the driver is connected to the resonator by a delay line, which is used as a simple model of a string or tube (see the 'What's the Delay?' box, below left). The driver waveform is used to drive the delay line, and the output of the line is then fed to the resonator. The 'positions' of the input and output can be changed dynamically. The delay line thus provides a rough analogy to the real‑world connection between the driver part of an instrument and the resonator.

If we understand exactly what gives analogue instruments their character, it's then possible to model them.

Rather than just using an audio sample as the driver waveform, Technics have attempted to 'reverse engineer' the output of real instruments by removing the effect of the resonator, so that the raw, unfiltered driver waveform can be used rather than a sample of the complete instrument. The actual driver waveforms that this process produces sound like thumps and bangs with extreme treble boost applied, but the final result once they have been 'connected' to a resonator is impressive — filtering removes the high‑frequency emphasis, and the result sounds like a sampled instrument. Of course, since the actual connection and filter settings may be completely different from the usual ones associated with the instrument, the final timbre produced can be very different from the driver sample itself.

The resonances which are found in most musical instruments do not translate to simple low‑pass or band‑pass filters: one method for experimentally discovering the sort of resonances that an acoustic guitar body might produce is merely to sing into the sound‑hole of a guitar. Technics have, presumably, determined the major resonances of a number of different instruments, and then used this information to work out what the raw driver sound was like before the resonator modified it. The results take the form of band‑pass filters whose bandwidth, Q and centre frequency can be altered — rather like a parametric equaliser. But in order to simulate the multiple resonances that are present in a real instrument, several of these filters can be combined in parallel.

Since this technique emulates the interaction between the driver and the resonator, I've called it 'Interaction Emulation' (see diagram, below right). It's important to differentiate between this and S&S synthesis, because although both appear to use a sample followed by filtering, the Interaction Emulation technique does not use a straightforward sample, nor is the filtering as simple as that typically found in S&S synthesis. It thus represents a halfway stage between S&S and Physical Modelling.

Physical Models

Physical Modelling takes the ideas behind Interaction Emulation to their logical conclusion. Instead of using just a simple model for the connection between the driver and the resonator, it attempts to produce models for the driver, the resonator, and their interactions. This provides a feedback path which is not present in simpler models. One of the best current examples of this technique can be found in Yamaha's VL1 and VL1m physical modelling synthesizers.

The connection and resonator models are similar to those already described: the connection can be modelled using a simple delay line, and if you use a more complex delay line, this can even produce part of the resonator as well. It's the complexity of the mathematical models required for the driver that pose the major technical hurdle for physical modelling, since here the task is to describe complex moving systems of air, strings, bows, lips and mouthpieces.

Let's look at the resonator first. Possibly the simplest resonator is a string or a tube. You pluck a string and it produces a sound whose frequency is related to the length of the string. If you blow across the end of a tube, the note that is produced is, again, related to the length of the tube. In both cases, the plucking or blowing adds energy into the string or tube, and the resulting vibration is emphasised at those frequencies whose wavelengths are related to the length of the tube or string. If the length of the tube or string changes, so do the frequencies which are emphasised, and the resonant frequency changes. If there are holes in the tube, or the width of the tube changes along its length, extra resonances can occur, while for a string the rigidity of the end‑points of the string can have similar effects. In digital circuitry, a delay line can be thought of as a model of a tube, so multiple resonances can be produced merely by adding in feedback paths at different points along the delay line. Some tape echo units produce multiple echoes in exactly this way, but they're behaving like tubes tens or hundreds of metres long!

Drivers are more of a challenge. There are many types, each with its own special characteristics. Plucking a string requires a sudden stretching of the string, followed by a release, whereas bowing a string involves lots of smaller stretches and releases as the rough bow catches on the string. With a flute mouthpiece, a stream of blown air hits the far side of the mouthpiece hole, whilst a recorder mouthpiece has a sharp edge that the air is directed against. Oboes and accordions use moving air to vibrate reeds, whilst in a trumpet or trombone it is the player's lips which vibrate inside a tiny mouthpiece enclosure. In a piano, the hammer hits the string at a fixed position, but the dynamics of the acceleration and deceleration are very complex. The mathematical descriptions of each of these is necessarily detailed, and well beyond the scope of this article.

The unrealistic freedom that you have with an analogue synthesizer is replaced by the natural‑sounding limitations of physics.

What is significant about this level of physical modelling is that because lots of real‑world behaviour is built into the model, the results it produces have many of the restrictions that you find in real instruments. So, whereas the analogue synthesizer makes a sound all the time, and artificially imposes an envelope on it, a physical model produces sounds only when you add energy into it by 'blowing', 'bowing', or 'plucking' — just like a real instrument. Trying to persuade a real or virtual reed to vibrate with a gradually increasing volume isn't easy either — at some point it suddenly jumps in and starts making a sound. The unrealistic freedom that you have with an analogue synthesizer is replaced by the natural‑sounding limitations of physics! In practice, though, you can usually explore modifications to drivers and resonators that would be difficult or impossible to achieve in reality, as well as mixing them — using a bow driver to drive a tube resonator, for example.

In case you were thinking that this sort of physical modelling is exclusively digital, remember that it is possible to connect an echo unit with a non‑linear amplifier (a compressor will do) to act as feedback, and then produce some very unusual sounds by tweaking the delay time and feedback level. All it needs is something to set things going — the inherent noise will do at a pinch, but a trigger sound into the delay line acts much more like the transient energy bursts that you might associate with plucking or blowing. Digital technology just improves the control, the repeatability and the depth of implementation of synthesis techniques.

What Next?

Despite appearances to the contrary, we don't actually understand how all musical instruments work. There are good models for plucked or hit strings, and these can be relatively easily implemented on affordable hardware, but any model that involves jets of air is considerably more complicated, and requires huge processing resources to calculate. Which is where simplification and compromise come in. Any resemblance between an affordable real‑time physically‑modelled flute and the real thing is a consequence of the fact that the model is a very simple digital representation of a tube, rather than a completely detailed model of how a flute really works. A simple analogy might be to consider a ventriloquist trying to produce the sound 'p': he can't move his lips, and yet by providing a similarly 'plosive' sound using his teeth and tongue, he can make the end result sound like a 'p', even though it isn't.

For some instruments, there just don't seem to be any models at all yet. In these cases, the ventriloquist analogy can be used again to show how it is possible to make sounds which appear to come from a specific instrument, even though we're not using the right model. I'll avoid taking this line of argument to its logical conclusion, where all models are declared to be so crude that they aren't at all usable, and that a serendipitous process of sound‑making is all there is to physical modelling. The process of 'misusing' sound synthesis techniques has worked very well for analogue synthesizers, and I see no reason why modelling‑based instruments should be any different. If it sounds like the timbre you want, the method used to produce the timbre may well be irrelevant to you.

The explosion of interest in physical modelling has changed the perceptions of many people. What was once an obscure topic of interest to a few physics/music researchers is now a valuable commodity, with electronic musical instrument manufacturers all investigating how they can incorporate this technology into their own products. The current range of instruments represent only the initial phase of development, since the synthesis possibilities opened up by physical modelling are far from being fully explored. More sophisticated drivers, better resonators, and understandable user interfaces should all contribute to improving the useability of modelling‑based instruments.

Perhaps the biggest casualty of the launch of modelled instruments will be the simple S&S synthesizer, where the static sound generation offered by a sample replay oscillator is coupled with a crude low‑pass/band‑pass filter: 'painting by numbers' is one possible description. But selling instruments with few points of similarity to the analogue or S&S metaphors which have been built up over many years, could prove to be very difficult. Yamaha's success with FM was tempered by its failure to persuade all but a few die‑hard enthusiasts to learn how to actually program it!