After doing initial template my next aim was to figure out what kind of info I could get out of sound signal + spectrum and how I could exploit it to effect colors, movement and shape of visualization.

First thing I decided to do was to go and see what kind of things people already made to figure out what kind of stuff could be extracted.

Cool audio visualizers

First place for me to stop was to go trough flight404 gallery at vimeo. I remember I was mesmerized many tiems by his works some of which are happen to be audio visualizers and here are two I picked out of his gallery:

Just click the picture to go and see videos on Vimeo. Check other his videos t00.

Then here are few random finds:

Now this one includes 3 visualizations:

I think I like third the most. Seems simple and yet kind of dancing along with music. Also I guess it would be possible to make such one in Flash but that’s not what I aim for. Even though ways it works behind the scene are interesting for any visualization.

And tastiest for the dessert. I seen it before but completely forgotten about. But it falls very close to what I want to achieve:

Now on representation part there are many elements I do not like. Mainly colors and that many effects are “pre-rendered” for exactly that song. But in core mechanics it exactly what I seek to make. Something even more abstract where user is responsible for part of visualization and also more colorful and closer to what Flight404 does. Big aims, not sure I will deliver but its fun journey anyways.

First experiments

In parallel with checking stuff from above I started to play with spectrum, listening various songs and trying to understand what kind of stuff I could get out of spectrum. Pretty quickly understood that I will not be able to extract information about “pitch” of music. Even though pitch is related sound frequency and sound signal spectrum is all about showing present frequencies you can’t really easily get various “pitches” out of it. I am not sure but it seems that pitch we hear is related to harmonics or signal fundamental frequencies. As far as I understand it works like that, we have harmonic of 2, then in spectrum we see frequency 2, 4, 6 and any other integer multiple of 2 present.  Now complex music contains many such base frequencies for instruments and human voice that all overlap and you can’t easily retrieve them. Anyways reading about it and thinking a little I decided to forget it, too hard and new for me to do it in few weeks. Also decided to search Amazon for books about math/physics and humans brain things behind magic of music and found those two books. I guess I will order one of them to read for self education. Should be interesting read.

After giving up on harmonics simple idea came to my mind pretty quickly on how alternatively draw spectrum. Results pleasantly surprised me (just love when my own ideas surprise me :D ). Here is small collection of experiments based on that idea along with various effects from my previous works(it may be processor intensive, did not invest much in optimization yet as it is work in progress):

It is interesting that out of all of them I like first the most. It is most simple but at the same time in in no way distracts from music in any way and seems to move best along with it while other visualization add their own movement that has nothing to with music…

Well that’s all for today’s post.