Stretching Audio for Fun and Radio
Over the past two weeks, whilst taking some much needed time off from work, as well as enjoying the festive seasonal cheer I spent some time brushing up on my C skills. Part of the reason for this is because I’m embarking on a long running project involving pic chips, modular synths and actual hardware will need to get back to some good old fashioned embedded C. My other plan was to start on an idea I’ve had for a while and decided since it would involve quite a bit of DSP, I’d do it in C and use it as a learning exercise.
By now most people should have heard the Justin Bieber 800% slower ”remix” that flew around the internet a little over a year ago. It uses Paul’s Extreme Sound Stretch to slow the song down without affecting the pitch, turning a fairly trite pop song into a truly amazing slab of monolithic, ambient noise. Essentially the program just takes many overlapping windows of audio, calculates the frequency spectrum using an FFT, randomizes the phase value for each frequency bin and then does the inverse FFT. There’s a little bit more magic to it than that, but that’s essentially it.
I’ve decided to create a radio station, much like PatchWerk Radio, that will download CC licensed audio from the internet at large, stretch the songs, and then stream them back out. To help matters along, Paul’s Stretch is open source software so I’ve been able to dig into the internals and find out how it all fits together. At the moment the code is up on github and can be used to stretch audio and write it out to a new file. It doesn’t sound quite as good as Paul’s version because there’s a bit more DSP voodoo in his that I haven’t yet implemented but I’ll be getting on with that in the next couple of days.
Once I have that sounding good, the next step will be to get on with the audio finding and downloading, then interfacing it with shoutcast/oggcast. Given how quickly I’ve managed to hammer everything out so far I don’t think it’s too far off.
Patch-a-Day November 2011 – Day 30 – Bouncing Ball
For the last (very late) patch of the month I’ve decided not to do any more with the sequencer, but to go off on a bit of a tangent and create a simulation of a bouncing ball. It’s not the most thorough physical model, and there are a couple of issues with it, but it gives a pretty good idea of how to go about taking mathematical equations and transferring them over to PD.
This patch will model dropping a ball from a chosen height and simulating how far it falls or rises within fractional lengths of time. The downwards direction gets changed to an upwards direction on the bounce but the co-efficient of elasticity (bounciness) can be changed to decide how much energy is lost when this happens.
I’ll be using two equations here.
These are two of the standard suvat equations of motion. The first says that in a given amount of time 
The second says that the velocity of an object after time
In the patch below I’ve broken the model down into three parts, one for the first equation, one for the second and then another part that checks when we’ve hit the bottom.
Bouncing ball patch
At the top there’s the metro to run the patch and some setup. There are five variables here and I’m storing them in value objects.
- The time period
- The initial starting height
- The current velocity
- The acceleration due to gravity
- The bounciness
This patch works with a relative time model instead of an absolute one. We’re calculating the relative change in height since the last calculation, not the overall height since we started. This means that the velocity change also needs to be calculated relative to the last velocity. This is done by the left hand collection of objects which calculate the second equation. The section gets given the sample length and multiplies this by the acceleration to get the change in velocity. This gets added to the current velocity to give us the new value.
The middle section calculates the values for the first equation. It takes the current velocity and multiples this by the sample length then adds on the 
The final part on the right hand side checks to see when the ball hits the ground. When the height is < 0 it reverses the velocity and multiplies it by the bouncyness. On the way back up the velocity will gradually decrease due to the effects of the negative gravity, and eventually it will stop and come back down. Assuming the bouncyness was a less that 1 value, the ball won’t reach the same height, so the time until it hits will decrease.
There are a few issues with this model, mostly around the the section that checks for the bounce. The height will actually go negative for one iteration and only go back up again on the next calculation, after the velocity has been reversed. This can cause issues when the height gets very small and ideally there needs to be something smarter here to work it out.
Also if the sample time is too large then there is a lot of unstable behaviour so it’s best to keep it to values around or below 0.1.
That said, this serves pretty well as an example so should teach you the basics of doing this sort of physical mechanical modelling.
But no1 that’s it for Patch a Day month 2011. I’ll make sure that all the example patches are in the repository for downloading and I’ll leave it at that for now. There will be more PD stuff going up here, I’ll make sure I;m better about it than last time.
Patch-a-Day November 2011 – Day 29 – Launchpad lights
Sending output to the launchpad is pretty easy. Sending the right output, in a way that’s easily configurable and fits in with the standard is a little trickier. I’ve had more of a read of the Launchpad programming guide and have a better handle on the double buffering and so on and I’ll put some of that into practise here. The updates to the sequencer here are, I feel I should point out, as pretty or high performance as they could be. There’s definitely a lot more tweaking needed.
One of the things I’ve found a bit of a problem is sending the large number of update messages to the launchpad. There’s a slow down in PD whilst it’s going. I don’t know if it’s due to the MIDI output, or just trying to send a mass list of 64 messages and needing to make the process more efficient but more investigation is needed.
Step sequencer track with light output
This is the updated track abstraction with the output for the light states. The left hand section that updates the data structure will now also send out a message with the new value of the updated step. The right hand sections are for dumping out the state of all the steps in the data structure. This is used when changing tracks as we want to update the launchpad with the new values and clear the old ones. By dumping out the state of all on or off steps we clear the old data and fill the grid with the new. This could be made more efficient by first clearing the grid, then just sending the steps that are on but i decided this is initially simpler to deal with.
Full sequencer with light state output
The main sequencer section has been updated as well to parse the data now being output by the tracks. The objects right at the bottom will take the step messages, convert the step number back into the XY coordinate and change the 1 or 0 value into the correct note velocity for the colour we want. The clr abstraction works differently now just to point out. It takes two arguments, one is the colour value to send out when it gets a 1 input, the second for when it gets a 0 input. I’m following the Launchpad programming guide by sending out a value of 12 for off. This apparently makes it easier to add in double buffering later which might well be necessary.
There’s also a small section in the middle at the top that sends out the light on/off messages for the mode lights. The tracklight patch just keeps track of the current track number and, when the track changes, will send out an off message for the old button and an on for the new one.
As I said, there’s actually a fair bit of work that needs to happen on this to make it fast enough but it’s not bad as a start. I’ll probably concentrate on tacking on the drum sounds for the next patch-a-day and then call it finished for the moment.
That will also actually be my last patch-a-day patch for this round. I’m going to be continuing work on this sequencer and I’ll be writing more of it up, just not worrying about doing it once a night though heh.
Patch-a-Day November 2011 – Day 28 – Launchpad sequencer improvements
Just a shortish one today, Yesterday I said that the step sequencer we’d built was mostly working but had a fairly vital flaw, we couldn’t actually save values in the data structure because we were getting not on and off messages from the launchpad. There are a few ways to fix this but I think that the way I do it here is the simplest and most flexible.
First things, we only want note on grid messages, that can be fixed with a spigot in the main sequencer patch.
Sequencer only accepts note on grid messages
Note off messages will now get blocked and won’t go through to the track abstractions.
Track abstraction now checks current value
There’s a few less objects in the track abstraction now but a few new ones as well. The biggest change is that the input is now actually only a single number. This number is used to calculate the pointer to the step position as normal, but it then gets sent to a get object as well as the set object. The abstraction will now check the current value in the data structure at that step and then update it with the opposite value, so we’ve successfully toggled the value in the data structure.
It’s easy to change this behaviour if needed, you might want to increment it or do something else, that’s up to you. Obviously we cant just send in a value for the abstraction to store now but that’s also easy to change, or just use the old version.
Anyway, the launchpad can now update the data structure and the sequencer actually sequences if you turn the clock on. Of course some lights on the Launchpad would be good, but that’s a task for next time.
Patch-a-Day November 2011 – Day 27 – Launchpad sequencer
Right, time to hook the launchpad up to the sequencer. I’ve modified the Launchpad IO abstraction to neaten it up and make it actually deal with output and input. The sequencer patch has been changed so that incomng grid messages are converted to a stop number and the mode buttons select the track number to update and the main clock is now on the outside.
Let’s run through them and I’ll explain a bit about what’s going on.
First launchpad sequencer
The sequencer is now the steps abstraction and the first argument tells it how many steps we want to have. The basic metro clock is still there, entirely unchanged. The launchpad_io abstraction now takes the channel as an argument as well. All simple and a bit dull.
Improved launchpad IO
The launchpad IO now just deals with outputting the MIDI in messages and sending the MIDI out messages. All IO goes through a single inlet or outlet and all messages are prefixed with either grid, mode or ctl to show where it’s from or going to. There’s also the reset option available which sends the clear message.
Updated step sequencer
The step sequencer now has a bit more functionality. It takes the input direct from the launchpad and pulls out the grid and mode button messages. The mode messages control which track we are updating and do that by pre-pending the track name onto the front of the grid message. The tracks then route for this as they do normally. Note the spigot on the mode section, this is there so that only button on messages get through. We don’t actually care about button off messages here so we just stop them coming through.
The grid messages are converted from their XY format into a single value that gives the step position. At this point it may be clear that there is a problem with the current setup. The grid buttons also have button up and down messages coming through, so when we press a button the 1 value gets stored, but as soon as it’s released the 0 is stored over the top of it.
For the moment I’ll let you think about how you might solve this, tomorrow I’ll show you how I do it.
Patch-a-Day November 2011 – Day 26 – Launchpad IO
I said I’d do this once I’d done a bit more work on the sequencer, but I realised that it’s probably a better idea to get this done first as it will make testing much easier. This patch is really just taking the MIDI input from the launchpad, formatting it into messages that are easier to deal with, and then translating our simple messages back into correct MIDI for us to send back out to it. As a quick refresher, here’s what the Launchpad looks like.
Novation Launchpad
During these posts, I’ll refer to three sections of the button layout. The eight buttons on the top row are the control buttons, the eight down the right hand side I call the mode buttons, and the 8*8 grid is the grid. Explaining that should make it easier to follow what I’m talking about.
The input from the Launchpad for these is pretty basic, but it’s not as easy to deal with as it could be, the grid and eight mode buttons all have MIDI note numbers assigned to them, but they’re essentially mixed together. Ideally I want to separate the mode button messages from the grid messages, and have the grid messages be in a simple (X Y value) format. The control button input is all as midi control messages, which is easy enough to deal with, but they start at value 104 and it’s much easier just to deal with 0 to 7.
One other thing to note, the input values from all button presses are either 0 or 127, either as note velocity or controller value. For input I find it much easier to just deal with 0 or 1 for button presses, output we actually need to use certain values to set the colours but that can be simplified easily enough. Here’s the patch as it currently looks.
Launchpad IO
Top left is dealing with grid and mode note in values. Straight out of the notein object I’ve already converted the velocity values to 0 or 1 and the notein object itself is filtering for midi channel 1. This is the channel my Launchpad appears under in PD though it might be different depending upon what’s plugged in. When I turn this into an abstraction, all the channel messages will be an abstraction argument to make integration easier.
The route object specifically selects for the eight note values of the mode buttons and then uses a fairly cludgy way to turn these into 0 to 7 numbers with the on/off value. Anything that isn’t one of these numbers is from the grid which can be turned into the XY coordinates I’m after using some basic maths. The layout of the note numbers on the launchpad follows the equation (row * 16) + column, so by dividing by 16 we can get the row number, and finding the modulo remainder of 8 we get the column. These get packed with the note on/off and that’s the grid message.
The control messages come from the ctl in object, situated middle top, and the channel filter has to be done externally because of the way ctlin works. The values also come in in the format (value, ctlnumber) which we want to be the other way round, the swap object handles that, then the ctl number has 104 subtracted from it and the data gets packed into a nice simple format to deal with.
The output sections basically do this in reverse. One thing to note is that I’ve added in a reset control. Sending the launchpad a zero value on control zero will turn off all the LEDs, a useful function to have.
The right hand side is just to take our input messages and convert them into output messages so the patch can be tested with a launchpad. The clr abstraction gives an easy way to set the output velocities/ctl values so we can select the colour we want on the launchpad. Internally they;re pretty simple.
Launchpad colour abstraction
The creation argument will set a default value, or the right inlet can be used to select and change it. The incoming value gets multiplied by the selected value so note off messages are still zero when they go out. The colours are red low, red full, amber low, amber full, yellow, green low and green full. I’ve just realised that technically all output to the launchpad should always be the off value (12) due to the internal buffering and so on it has available, I’ll need to check that out, but for the moment, have a play around with this if you have a Launchpad and see how it works.
This kind of input filtering and manipulation helps to make life much easier when creating larger apps that interface with MIDI controllers and I hop this abstraction will be pretty useful in future.
Patch-a-Day November 2011 – Day 25 – Step Sequencer Clock
OK, carrying on from yesterdays patch, lets get the stepping and clock for the sequencer.
Sequencer clock and output
The setup here is really pretty simple, but there’s a couple small things to note. I’m routing for clock and reset messages, just to make things obvious when you;re controlling the sequencer. The reset message gives the pointer a traverse message which resets it to the head of our list of scalars. The clock message will trigger a next message to be sent to the pointer which will then send it’s value to the series of daisy chained objects. This will cause all of them to output their values and they can be seen printed out to the console.
The trigger object is there because of how the pointer deals with coming to the end of a list. The pointer will output a bang, upon receiving a next message once it has already output the last list element. we want that final bang to send it back to the beginning, and then immediately output the first list element. To do that, when the pointer sends out its bang, it first triggers a new traverse message, and then triggers another next message, meaning the first step values get sent out.
Next I’m planning to neaten this up a bit, and then we can start dealing with Launchpad input.
Patch-a-Day November 2011 – Day 24 – Step Sequencer Basics
I’ve just noticed I think I missed out a day somewhere…. This patch is patch 25 in my repository, but it’s day 24…. Looks like I made a patch titled Improved presets and haven’t written it up but oh well. I might go back and slot it in if I think it’s missed out anything vital.
On with the step sequencer for the moment though. I’m designing this to run on my Novation Launchpad, the plan being to have eight tracks with 64steps each. Track selection will be done by the left hand row of function buttons and all track steps displayed on the grid. The first thing to do is work out how the pattern will be stored, and I’ve decided that the data structures are the right way to go about doing it. Below is the beginnings of the patch.
Beginning of the data structure sequencer
Structure and datawindow patches in the top left corner, struct has eight float fields named a to h. The central section just clears our window and creates the data for the 64 steps. The section on the right is just to that I can write out the data to a file for debugging and won’t be there in the final patch.
The group of daisy chained objects on the left hand side are dealing with saving and loading the data about individual tracks. The left hand inlets/outlets deal with setting individual steps for specific patterns. The middle is so the object can receive a pointer to a specific step and load the value from it, the right hand inlets/outlets are where this data comes from. For the moment only the saving works, though the loading doesn’t need much more done.
Basic track abstraction
I’ve realised that in patch comments are probably a good idea heh, so I’ve started adding them. As they show, the creation arguments for the abstraction are the name of the data window, the name of the struct and the name of the field to set or get.
One of the things I decided for this was that the functionality for getting the right pointer when setting a value should be inside the abstraction, but when getting the value the pointer should come from outside. My reasoning was that when getting the value from a step we’re actually going to want to get the value from all the steps and, because they would all use the same pointer, it would be wasteful to repeat that functionality. When we’re setting a value we’re usually just doing it for one track so it’s simpler to contain that inside the abstraction.
The abstractions all use the passthrough design so when a message comes into the left hand inlet it is routed to the correct place by checking for the field name. From here the step position triggers a standard traverse->until>next style setup and then the value is set using the set object. Try sending some messages in and then saving the data to a file so you can have a look. This works pretty well and should mean the entire sequencer itself will be pretty compact.
Tomorrow I’ll get the the retrieving side of it done, then time to start on interfacing it with the launchpad.
Patch-a-Day November 2011 – Day 23 – Generic Preset abstractions
Back on the case finally. Too many things on the go and fairly behind on Patch-a-Day. I’m planning on getting quite a few patches built this weekend so I should be finished with the 30 days by mid way through next week, hopefully.
Anyway, this evenings patch is an extension of the preset saving that we’ve been looking at. I realised that actually it was very easy to move the get and set objects into their own abstractions for each preset value you’re interested in saving. It’s using the same sort of passthrough design I’ve talked about and it results in much less clutter and much more useful code I feel.
Individual preset parameter abstractions
The bottom section is what’s actually in the preset-param abstraction. This time the loading and saving are being triggered by a pointer sent into either the left side for saving, or the right side for loading. The param objects still get and set their values using the send and receive objects and now it’s very simple to add more parameters to store in our preset.
I’m probably going to turn the file loading/saving and structure clearing section into an abstraction as well, at which point this should all work as a fairly generic solution. There are a few other things I think could be added but I’m probablly going to be leaving this for the time being.
I’ve decided over the weekend I’m going to build up a sequencer based on my Novation launchpad and using the data structure and preset stuff I’ve been learning. Tomorrow is a new day, full of patching!
Minor delays again!
Becoming a bit of a busy week again, probably won’t have a chance to write any more patches until Friday unfortunately, but I have been granted an entirely free weekend, which makes a nice change. November has almost finished, but I will get to patch 30, never fear!