26 April 2007

Adaptive Music

Piano taken by  Andreas Reinhold

I was reading an article on Defining Adaptive Music which is well worth a read if you have a few minutes. I've studied a bit of music in my past and, I guess, my interest in dance invariable knocks on to an interest in music too. The article, as its title suggests, attempts to define adaptive music; much more in the theory rather than practice.

A few years ago I had a Gravis Ultrasound (what am I talking about it's well over 10 years ago now). That soundcard had much, much, better MIDI handling than my current onboard sound card. Decent MIDI synth is expensive; requiring a chunk of dedicated hardware. People who want it, for music development work, spend a lot on a top of the range soundcard; everyone else just gets some off the shelf card that has fairly poor support. Games now tend to use MP3 or similar files for their tracks; they are small so you can fit a lot of them in a reasonable sized game (though they are no where near as small as MIDI data); there is a CPU cost associated with playing them but it's relatively inexpensive. This means unless you want your game to sound poor on most peoples hardware you want to go for compressed audio like MP3.

MP3s are fairly difficult objects to play with though. They aren't very well suited to adaptive music. Sure you can fade one out and fade another in; you can play one over another to, say, keep the baseline while you change the treble (assuming three audio files here). You can change the pitch and such but your options are fairly limited. It's not to say you can't do it at all. The article mentions Mozart’s Musikalisches W├╝rfelspiel describing it as:

In Mozart’s combinatorial Musical Dice Game, parts are generated a measure at a time by rolling dice to pick randomly from a table listing multiple potential versions. The number of potential variations of this piece of music is so large that any waltz you generate with the dice and actually play is almost certainly a waltz never heard before. If you fail to preserve it, it will be a waltz that will probably never be heard again.

In this case each piece could be recorded onto different files and the same random method could be used to generate the given track for, umm, a Waltz scene. In a similar way perhaps it is possible to include the fast bass line of some battle music with the music of a safe location such as a town as a juxtaposition between what the player recognises as a safe location and the fact that, for some reason, a battle is going on there. Of course the audio engineers might just write another score that mixes the two and has it as one file: until there is a large permutation it is probably more efficient to have multiple files with only one playing at a time rather than have less files (possibly) with multiple files playing at the same time.

It could well be that the best place for innovation in this field is in consoles like the Nintendo DS which, as far as I know, uses a synthesizer. So what do you think? Do you think that we've missed the boat for major innovation in the area?


Matthew said...

I don't think the boat's been missed. Rather I think the primary protagonists inexplicably leapt from the boat into waters infested with the terrible shreaking eels. After all Lucas Arts were doing a lot of this sort of thing with their point-and-click adventures.

They started doing MIDI stuff using the oh-so-wonderfull FM synth. However by "Curse of Monkey Island" they were using sampled sound. As you point out, they lost a lot with this move but they gained realistic sounding instruments. Instead of doing clever interpolation between MIDI scores they had to create the music with cut points at which you could switch track without a nasty glitch.

There's a nice little site which has a load of history and samples to illustrate the various techniques.

Gary said...

Thanks for that link it was interesting. I've known about LucasArts iMuse stuff but it's interesting to see that they did make the transition between MIDI and MP3.

Cool stuff.