After a tough year I finished my graduation project . The result is a browser which translates html and style properties into sounds. Unfortunately it is not possible to make the current application webbased. A lot of research needs to be done to increase the accuracy of the sonification of the web. In the future I hope to post some video’s to demonstrate the working, for now the sonification can be heard here.

Documentation

The full paper

Abstract

Is it possible to make an interpretable sonification of the web? To answer this question the Musical Browser was build, The Musical Browser makes it possible to explore the web, and sent the HTML to the parser. The parser translates this model into data which a sound synthesizer understands. The synthesizer software turns this model into sound. This separation made it easy to experiment with the sonification.

Several tests were conducted with custom created, preselected and random webpages. These test provided a granular approach to interpretation. From webpages with a distinct property, (very dark webpage) to more average webpages (colorful webpage). For the last test predictions were made about webpages after hearing the sonification of these webpages.

The result is that the musical browser’s sonification is interpretable for the HTML tags and some style properties of a webpage. But the sonification is not sufficient to make an exact prediction of the visual display of the webpage.

During my graduation project I needed to get a 2 dimensional list from Processing to Max/MSP. I knew my best option was Open Sound Control. First you need to CNMAT objects for Max/MSP, these objects make Max/MSP work with OSC. Processing needs the oscP5 library

OSC does not make it possible to send multidimensional arrays, therefore this only works with multidimensional arrays with the same data type. Now the code, processing sends an Integer array of an index and 4 values.

Processing Code

import oscP5.*;
import netP5.*;

OscP5 oscP5;
NetAddress myRemoteLocation;

void setup()
{
  size(400,400);
  frameRate(25);
  /* start oscP5, listening for incoming messages at port 12000 */
  oscP5 = new OscP5(this,5555);

  myRemoteLocation = new NetAddress("127.0.0.1",5555);
}

void draw()
{
  background(0);
}

void mousePressed()
{
  /* in the following different ways of creating osc messages are shown by example */
  OscMessage myMessage = new OscMessage( "processing" );

  myMessage.add(new int[] { 1, 2000, 1000, 5, 5, 2, 400, 500, 5, 6, 3, 200, 560, 5, 6 }); 

  /* send the message */
  oscP5.send(myMessage, myRemoteLocation);
}

Max/MSP part

Now in Max/MSP we need to update the objects to their maximum buffersize and their maximum listsize. To extract the data we need to slice the list to remove the “processing” String and then divide the list in chunks of 5, the last step is to unpack these object to get to the data.

As you can see you need to set the buffersize for the OpenSoundControl object in bytes. For the zl object you can set the maximum list size.

update ( 09-03-2010 )
You can download a sample application with a processing patch as well as a max/MSP patch.